Office 2016 updates and network bandwith

[bbot]

In our ~600 Mac environment, deploying updates for the Office 2016 suite has been a huge pain.. mainly on the network side.

I'm finding everytime I push more than 50 of a single update (like Word or Excel) within a one hour window, I get a page from our Network team that the network utilization is over 80% and I need to stop as it may impact our call center.

As you can imagine, having 600 Macs, and deploying 1 update for each Excel, Word, PowerPoint, Outlook, and OneNote is 3000 deployments. If I can only do 50 at a time, my full-time job now consists of only doing Office updates...

How are you guys deploying Office updates? Does anyone else have network constraints with deploying updates that are larger sized? (300mb to 800mb)

[scottb]

We usually split up this sort of rollout via smart groups and stagger them.
Sometimes it's based on site or country, which has worked well for us.
I know many folks are rolling out the entire installer for Office as opposed to the single updates as they take up more space. There was a good thread breaking things down, I'll see if I can find it.

[Nix4Life]

Hey @bbot Are you able to run or schedule the updates at night or on the weekends? how about staging (sharding)? also have you seen the Office 2016 two-part presentation on you tube presented by Paul Bowden and Bill Smith? I am going to link to part 2 where they discuss applying updates. using the most recent installer may help with your issues, it's at the 45 min mark Video

[thoule]

You have a couple options.

1. Put your packages somewhere externally available and have external clients download from there. You'd have to do some redirection for internal and external clients, but support for that is built into the JSS.

2. Precache updates on machine. You can schedule update caching between certain hours. Then have a smart group of users that have the update cached and offer the update to them. This solution is a pain, but it would allow you to minimize network usage.

3. Write a script to have the packages downloaded from the vendors, then applied. Think autopkg-like solution on each workstation, then applied. This solution would take some coding, but should be possible.

[mconners]

Here with over 800 Macs, we are limiting the desktops to night updates, which occurs for us every month. The laptops are trickier, but have removed our lab Macintosh laptops form the update process for various reasons. Employee laptops is really the last group and are limited to less than 100, 95 out of all of those. This has really helped us a lot.

I know the network team did some QoS, quality of service, settings on the network so our telepresence isn't impacted by imaging or deployments.

[May]

Hi,

I cache then install the latest Office installer rather than each individual Office update,
this helps reduce the size by a fair bit

Office 2016 update = 1.58GB

Word update = 1.02GB
Excel Update = 887MB
Powerpoint update = 826MB
Outlook update = 719MB
Onenote update = 442MB

Downloads here

It's advised somewhere in Administering Office 2016 for Mac, Part Deux

[bbot]

@scottb I've been using smart groups and limiting the number of groups by utilizing the "JSS ID". I normally limit it down to below 100, then update.

@LSinNY @mconners Unfortunately 99% of our Mac population are user laptops, and the users are instructed to take their laptops home with them. Our Casper environment is on-prem, so they'd have to be on VPN at home to be able to receive packages.

@thoule I like #1 and #3. I'll have to dig deeper into these. I'm using #3 for most of my other patches like Adobe updates and Java updates where I'll go out to the vendor to download the packages.

@May I'll look into this. By pushing the entire Office suite, doesn't this force quit the user's Office suite? I like the idea of being able to push updates while the program may be open so it doesn't interrupt the user as they work.

[May]

@bbot

Hi,

We're caching the suite first then once the package is cached the user gets a notification telling them that the update is available via Self Service, and within Self Service the description tells them to quit all Office applications before proceeding.

I could also add a check in there to be sure that Office is closed first, notify if it's not closed or trigger the install policy if it is closed.

Are you able to install the updates whilst the application is open ?

[bbot]

@May Gotcha.

Yes, the individual updates allows you to install it while the application is open. I heard it's not recommended, but we've been doing it here since 15.19 with no complaints. The next time they close and open their Office app, it'll be updated.

[mconners]

@bbot We haven't had issues doing the individual updates either. In fact, this is the first time I have seen this isn't recommended as we really have had good success, knock on wood. Any idea why this isn't recommended? Is there a gotcha in there somewhere I am not aware of?

[bbot]

@mconners Now that I think back, I think I'm getting 2011 and 2016 mixed up where they had the "all_quit" pkg in there where it forces you to quit all the office. This might not apply since 2016 now that the app is now self contained.

[talkingmoose]

@mconners, the folks at Microsoft themselves don't recommend updating any code while it's running. According to Erik Schwiebert in the MacAdmins team on Slack:

basically, updating things while they are in use can cause really weird and unpredictable problems. For example, if we’ve mapped a file into memory and the file gets updated while it’s open, then when we go to read it the new data currently in the file may not match what the old (currently running) code expects. At best, nothing goes wrong. The app might just crash. Or, we might write bad data out to a file and end up in a corrupted state.

He later said (paraphrasing):

You assume the risk if you update the apps while they're running and corrupt a user's work.

[mconners]

Thank you @talkingmoose! Fortunately, I have had all of my update policies set to run only on logout.

[talkingmoose]

@bbot, you've got some options (now and forthcoming).

If you can use Microsoft AutoUpdate (MAU) in your network environment, then that will aways decide the smallest updates needed to get to current. The delta updates are a lot smaller than pushing the individual apps. If must deploy the updates yourself and you're pushing three or more apps, push the 1.5 GB installer as an update. That's fully supported.

As of MAU 3.6 (I think), updates are silent and don't require admin privileges to install. The already installed Privileged Helper Tool has the elevated privileges it needs to download and update if an app is currently quit. If an app is running, it'll prompt the user to either quit and install now or wait 'til later.

With MAU 4.0 (in beta), you'll have command line control. You can turn off the "Auto" and then use Casper to install when you prefer.

A feature coming later to MAU is the ability to run your own internal Office update server. It should work over HTTP/S, which means you could run this on an internal web server or even a repository on your JSS. (That's all to be seen and tested before recommending that approach.)

[mconners]

Thank you for the great news @talkingmoose. Any place I can read up about the various versions of MAU that you mentioned? I am intrigued and excited to see these things moving us admins forward.

[bbot]

+1 on the MAU version. I'm interested to learn more.

I figure I can also test this when I have time, but do you know off the top of your head if you're able to deploy the 1.5 GB combined installer if a user has the program open without force quitting their app?

I use to do the log out and log on update, but we found our regular user only reboots 1-2 a month if that. Being in a finance company and being heavily audited, we have to have our machines up to compliance all the time.

[scottb]

@LSinNY - THAT video is where I saw the info on the individual packages vs full installer. No wonder I couldn't find the thread!
It's a great watch, and very informative. I also forgot about caching, which we do utilize for things that don't change as often as Office does these days.

[talkingmoose]

@mconners, I'm learning about all the new stuff coming in Office for Mac by participating in the MacAdmins Slack team. That group now has nearly 7000 Mac admins signed up and right now I see 675 signed in. It's a great resource for asking questions and learning new things.

If you subscribe to the #microsoft-office and #mau4 channels, you'll find a list of pinned items (you may have to enable the sidebar to view them). Any nuggets of wisdom or knowledge bombs get pinned there. You can read through the history or search.

All of this goes along with the http://macadmins.software site that one of the Office developers maintains for us. You'll find a history of Office 2016 updates as well as links at the top to documents for our review before they become KBs and a list of the Office developers interacting with us in Slack.

@bbot, not sure what else I can say other than what I've already posted from the Microsoft developers themselves. Users should quit their apps before updating. Period.

Computers are not appliances like toasters with one function that works the same for 20 years. They're devices that require periodic maintenance to keep them functioning well.

[Nix4Life]

@talkingmoose Thanks for picking this up and as always providing good information. I was typing on an iPhone and trying to condense as much as possible before i stepped into the subway

Larry

[bbot]

@talkingmoose Thanks. Sounds like I need to do some thinking on how to re-work this update policy. Our management wants everything pushed as opposed to self service. (We tried the self service method and was only getting about 20% compliance)

On log off and log on wouldn't work as we've seen uptimes of well over 30 days before reboots.

I'm thinking of using jamfhelper and having a popup that forces them to update using the full office package and can have it quit out of the updates before running. I can work in delay/postpone feature.

[itupshot]

@talkingmoose I love the last paragraph of your last post. I will use that to try to get people to understand why they need to log off their desktops at EOD so we can run updates.

As for the topic at hand, we use the VL installer and serializer for the initial installation of Office 2016 during imaging. After that, we use the Office 365 installer for updates. As others have mentioned, it's smaller than trying to use the individual update installers.

We added a "no quit" and a "no MAU" key to the choices XML. This allows us to install the suite while the apps are running, and we don't install MAU. It may not be recommended, but it's extremely difficult to get people to quit their apps to run updates here.

The policy has a Complete Message notification that reads, "Microsoft Office was updated in your Mac. Please quit and re-launch Outlook, Excel, PowerPoint, OneNote and Word."

We schedule our desktops after office hours with smart groups broken down by model; for example, mac minis first, iMacs next, and Mac Pros last.

We schedule laptops during office hours, again broken down by model: MacBooks (Retina) first, MacBook Airs next, and MacBook Pros last.

So far this has worked. However, I'd love to host an internal update server for Office for Mac updates. Silent updates running from that would be nice. We have one for Windows updates. We also use internal Apple SUS, and Adobe SUS, so Office for Mac SUS is the missing piece.

[talkingmoose]

@bbot, have a look under the User Interaction tab of your policy. Enable the Allow Deferral feature on that page and you'll see some more options come alive. This may be what you're needing (already built in).

[mpermann]

@talkingmoose the "Allow Deferral" feature is not working currently. I don't believe it has worked correctly since 9.81. Have a look at this thread for more information. It doesn't sound like they are going to fix the issue until version 10 is released.

[clburlison]

Quick correction MAU 3.6 does include a PrivilegedHelper. It does not include the silent backup updating. That is a main feature point for MAU 4.0.

The only task that the PrivilegedHelper does is allow non-admin users to install updates.

MAU 4.0 will have all the fun features: cli options for driving the updates, automatic caching of updates and installation when the applications aren't running and maybe more...we'll just have to see.

[talkingmoose]

@mpermann, thanks for the update on the Allow Deferral feature. I've helped a small handful of customers configure this feature, but that could have very well been before v9.9.

And thanks, @clburlison, for correcting the version number I posted.

Since our posts yesterday, Ian McDowell with Microsoft has provided new information about Microsoft AutoUpdate features in the #mau4 channel on Slack. Be sure to check out the pinned items in the channel.