What (if any) test plans do you use when testing a new OS in your environment

There are a great many tests that we perform given our specific operating environment. First and foremost, I have a constantly updated document that spells out the various specific program requirements that have been submitted to our IT Department over the years. It's really important to have this sort of documentation and that you remove anything that is no longer supported. Because every institution is different, and every department within said institution may have different requirements this will be critical in preparing ANY deployment.

Now with all that said there are a few fun test's that I run AFTER ensuring that my images or self service items meet the above requirements.

1) User Test -For full images I generally find a problem user in said group to beta test for me. Note, that I find the user most likely to break things but still be eager to help

2) Application Torture test - Open the Applications Folder, expand every directory containing apps within that folder, select all and open them all at once. This simply allows me to see that under severe duress, each application may open without directly interfering with other apps who's combinations I may not have predicted. -Leave all app's open overnight and then see if I can individually operate and quit said applications normally. -Open each individual app thereafter to make sure that no preferences or databases have been corrupted.

3) MOST important to me. -I actually force myself to use a laptop that I've based on the main image my company uses rather than all the various apps/tweaks that I prefer to use. This allows me to better understand the situations that my users experience as well as pointing me to trouble spots my users don't report.

Documentation is critical as you understand. I always prepare full documentation for each full image configuration as well as each separate package. It's easy to document the various packages within Casper Admin. I prepare a separate document for each complied image out side of the documentation mentioned above. I'm sure there will be many great other items mentioned here, but these were the first thing to jump into my head. After I finish I also document any issues noticed after deployment and what I've done to correct them so as to be prepared for the next major image.

I've also used something similar to Chris' user test, which has always been part of a test or pilot group. In that group I've solicited volunteers that included both technically savvy users and complete technophobes, but all eager to help and, importantly, agreeing to the fact that they are using something new and there will likely be issues to work through. I also made sure to include users from different departments to get a good read from the different use types. A Creative Suite user will likely have different problems than someone who lives in Excel most of their day.
The tech savvy users often don't make the best test subjects since they have a tendency to "fix" stuff or work around problems and not report them, hence the inclusion of the technophobe types.

I like the Application torture test outlined above, and may crib something like that for our own tests going forward. :)