We also are required to restrict Mail and ran into this issue. Our workaround was to leave the "Message" section in the restricted software item. It then just kills the process/blocks Mail from opening without displaying a message. We also remove Mail from the dock for our users.
@CJeffery
Are you restricting Mail or Mail.app?
If it’s the latter, you’ll see the behavior you’re describing as the app is simply called Mail now, without the .app (this is a change in Sierra). Even with the Restrict exact process name box checked, it should still just be Mail to avoid the issue.
Thanks!
Amanda Wulff
Jamf Support
I have just checked.
I have mail in the process name field and set to kill process.
I had to turn messages off because it was coming up everytime at startup.
Shall I try dropping the wildcard characters?
Gabriel, did you add Message or Messages?
Also do you kill process or restrict exact process name?
thanks,,,,
@CJeffery
From what we found in testing here, simply having it set to Mail (with a capital M) with the Restrict exact process name box checked should make it work without having the message pop up at random times.
If the OS or the user tries to do something that would, by default, try to open Mail, they would get the message again, most likely.
When we were testing, we found that this was because the process name in 10.12 changed from Mail.app, which is what it was in previous MacOS versions, to just Mail and it seemed to cause it to kind of freak out if the restriction said Mail.app.
If you've got the Restrict exact process name box checked, there is no need to use wildcards, and using wildcards could cause unexpected behavior in that case as it will assume they're part of the exact process name.
Unless you're looking to restrict all apps with the word "mail" anywhere in the process, we shouldn't be using wildcards anyway as it may cause odd behavior if the user or the system opens an app or starts a process that has "mail" anywhere in the app or process name, even if it has nothing to do with the Mail app.
If you're just looking to restrict Apple's Mail application, just using Mail and having the Restrict exact process name box checked should do that.
We didn't have the issue of the message coming back at startup when testing, so I can't vouch for why adding Messages to the restricted list would cause the popups to stop as we were not able to replicate that behavior in house.
Thanks!
Amanda Wulff
Jamf Support
@CJeffery This is what we have for Messages and haven't had any issues. 
