For several years now, Apple has highlighted the important role that partners and Admins play in testing beta versions of their upcoming operating systems and submitting feedback for any issues that can (and do, particularly in the early versions) appear. They also provide some great tools that make it easy for this to be done, but those tools are only half of the story.
The other half is an effective testing strategy that, when executed properly, will both provide Apple’s engineers with invaluable data for resolving unforeseen issues prior to public release and also give you ample opportunity to ensure your readiness (and confidence!) to deploy the latest OS version upon public release. This benefits your organisation and users alike by offering the latest software features to enhance workflows and maintaining security standards across your Apple fleet.
This post aims to provide an overview of the tools Apple provides to Admins for beta testing at scale, as well as highlighting best practises that can form a solid basis for your own organisation’s testing programme.
AppleSeed for IT
You can think of AppleSeed for IT as the starting point of your beta testing journey, much like how ABM/ASM are the foundation of the device management process. AppleSeed for IT provides you with complimentary access to beta releases on the Developer cycle (the faster channel that receives beta updates earlier and more frequently than the Public cycle). A Managed Apple ID is required to log-in, which you will almost certainly have if you’re reading this. AppleSeed for IT also includes a change log for each beta OS release, a discussion board and an area for submitting feedback to Apple - although beta OSes also feature the Feedback Utility app that generally offers the best experience for doing so.
Currently, enrolling a device into Apple’s beta programme is achieved by downloading and installing configuration profiles. However, this is due to change as Apple retires this process and instead moves to tying device beta enrolments to approved Apple IDs. In the near future, this almost certainly means that you’ll need to sign-in to the test device using the Managed Apple ID you use for AppleSeed for IT, rather than grabbing a profile from there and installing it. The good news is that you can sign-in to just the Software Update area of the device using a different Apple ID to the one that’s primarily in use, just like you can for the App Store, if the primary Apple ID doesn’t have Developer beta programme entitlements.
Here are a couple of resources from Apple to get you started:
Testing is a year-round task
Chatting to Apple’s SE team, it’s clear that the best practise for beta testing is to treat it as a marathon rather than a sprint. It’s a task that should be undertaken from the moment a new major beta release is made available, and then kept up-to-date as newer releases come so you can see if any former issues have been resolved and whether new ones have appeared.
In this way, you can also constantly check for compatibility with your organisation’s most key apps and have a much more accurate idea of whether mass-deploying the OS upon public release is going to have undesirable effects on them. You can even go a step further and raise any such major issues with app vendors directly, so they have a better chance of being fully compatible upon public release.
For those who are new to beta testing and perhaps unsure of what to look at, I personally like to test a fresh ADE enrolment in environments where advanced automated build assets are used (such as branded Jamf Connect configs, Enterprise SSO, complex scripts and third-party network security tools). You can also test your personal workflows and those of your colleagues, to gauge how usable the OS already is at this early stage. Be careful if you use endpoint protection/security software, as you could well run into false-positives during the early stages of a major new OS or otherwise unexpected behaviour.
Feedback
A key part of beta testing is having the ability to directly feedback to Apple about anything of note, whether it be things not working as expected with an app or something more foundational. As mentioned, you can do this from within AppleSeed for IT itself (it opens a link to feedbackassistant.apple.com) or, for the best experience, via the included Feedback Assistant app (see screenshot). Whichever option you choose, you can view both your own feedback and any feedback originating from your organisation. This is ideal in organisations where an individual may be the ‘lead’ beta tester, as they can see everything from a single pane of glass. Apple may even directly respond to your feedback submission, though this is usually either to ask for more information or to request a sysdiagnose capture in my experience.
In terms of ‘how’ to give feedback, simplicity and clarity are key. Step-by-step instructions on how to replicate the issue are great, as are screenshots, videos or logs (definitely check out macOS’ Unified Logging commands in Terminal, or the integrated Console app for a more graphical experience). It can also be handy to include how the issue impacts your workflow and wider context in why the issue is a pain-point, because this can help Apple determine if it’s likely to affect lots of users. Lastly, be sure you are using Managed Apple ID from AppleSeed to file your feedback, which will result in the feedback going into the queue for Enterprise & Education reviewers.
For more information, take a look at the Bug Reporting Guide on Apple's Developer Portal.
Closing thoughts
Far from being viewed as ‘just another thing for our team to do’, beta testing should absolutely be embraced and maintained constantly throughout the year. As this article has shown, there are many benefits to doing so: your IT team will be empowered to deploy major OS updates with complete confidence on day one, giving your users the benefit of having the latest features and your Apple endpoints the best security baseline, and you directly help influence the development of Apple’s OSes. Plus, those things aside, it’s always fun to play with new software and features before anyone else!
7 Comments