Does anybody have formal test plan they use when testing a new OS in your environment?

[RyanN]

Looking for some suggestions from the community on how others work through testing new OS versions releases. I am looking to create something to be used as a baseline and just want to tap the expert knowledge the community might be able to provide.

Thanks
Ryan

[SQR]

Good question and would love to hear other ideas.

I try to have a few test plans in place. modular and upgrade.
I work in a creative department, so we're up against drivers and constant adobe updates among others.
So i like to make an image of a few of our users current state systems and perform upgrades and test against that. see what basically doesnt work in the new environment - some are okay with it, some want theyr data off, thats fine. were pretty laid back so its not much of an issue but it gives me an idea if a full upgrade path will work, or do we bare metal them and modular upgrade apps.

thats where I like to nuke a system and deploy the bare image, drop apps adobe and office and test against the paces there. generally we have goodluck with bare+modular imaging but can be time consuming since anything after 10.9 is probably already a fresh install, so we do like upgrading where possible, but if theres a lot of software updates or driver changes we tend to do complete wipes and modular image from there. but so far using real live instances and modular/out of the box is my method of testing

[gachowski]

I asked many many years ago and there was no response : )

I was looking at something like

http://www.testplant.com/eggplant/testing-tools/

And there is an open source one too...

However back then it's was just faster to check by hand... I have a check list that I check and then I have our onsite team test too... also when we run in to a new issue it goes on the check list..

Super low tech.

C

[powellbc]

We maintain a spreadsheet with 2 tabs: one for common functions in the environment (e.g. connecting to wireless infrastructure) and one for applications. We start with a base template and modify it each year as needed. The client management group uses it to test and then fills it in accordingly.

We then share this out with users so they know exactly what has been verified to work or not work.

[damienbarrett]

@powellbc Would you be wiling to share that spreadsheet? I am just about to begin designing out "testing matrix" for El Cap and wouldn't mind seeing what others have already created...

Of course, environments differ and tasks will be different, but we're all capable of excluding from the spreadsheet things that don't apply, and including others...

[powellbc]

It is not fancy, but here you go: Testing Results Template Mac OS X

I edited out the stuff specific to us, but hopefully it can get you started.

[RyanN]

Thanks everyone for the responses as the ideas have started me in the right direction.

@powellbc - Thanks for sharing your spreadsheet and if anyone else is willing to share it is much appreciated.

[m_stirrup]

Does anyone have a copy of this, or something similar, the link above has been disabled.

[gachowski]

My current plan is use an update version of this in our enroll script, so it gets checked in real time.

https://github.com/jamf?utf8=%E2%9C%93&q=CIS&type=&language=

Right now we check for the FV key and a few profiles after FV is enabled.

While this won't check if MS Office is running correctly or other GUi setting I think it's the best place to start..

C

[mbezzo]

I'd be super interested in seeing what others do too. This is one of those things that smaller orgs can really struggle with because there's not really staffing for it - but it's really important.

We implement a pilot group post official release and let that group use it for a few weeks and sort of work through any issues that show up. Historically, I haven't been able to test with end users any earlier than Public Release cuz there's always been a piece of critical software (nearly always Security/AV related) that just doesn't work on the betas. This year seems to be better! (knocks on all the wood!).

But yeah, would love to see any docs teams use to more officially test. I know lots of these docs would be highly org-specific, but I think we all understand that'd be the case - and just seeing the sort of framework is the important part. We'll obviously need to customize them for our own unique environments.

Thanks!