Software Update Server Updates automatically disabling

[Matt_Ellis]

I didn't have any luck on the apple forums. So im hoping to have more here.

I am currently setting up a Software Update server for my company and im having a strange issue. I enable all updates from 2103 - Current. They show as enabled and downloaded. But when i come in the next day all updates are disabled. has anyone ever seen this issue before?

System specs:
Mac Mini Late (2012)
OS X 10.10.5
Server app 5.0.4

Mac mini, OS X Yosemite (10.10.5), Server App 5.0.4

[gskibum]

I didn't have any luck on the apple forums.

Join the club! :-)

Hopefully someone chimes in with an easier solution than mine.

For many years Software Update Server would frequently break on me on different servers at many different locations. When it would break I quit banging my head against the wall with trying to fix it and would just reset it back to default. 3-4 years ago it stopped breaking on me so I haven't had to reset to defaults in a long time. Now I don't remember how to reset it.

I tried to search for the procedure just now and was finding old info that may be outdated.

[Matt_Ellis]

Dang seems like when they removed server admin, OS X server when to hell :( thanks for the reply though ill update this if i get anywhere.

[gskibum]

Ha I just found this by modifying my keyword search a bit.
https://support.apple.com/en-us/HT200272

[Matt_Ellis]

Isn't quite in keeping with my senario but ill give it a go this server isn't in production yet. thanks

[bentoms]

@Matt.Ellis i wonder if it's an issue with the available space?

I've detailed what I saw when that was my issue here

[GabeShack]

Ill 2nd the available space piece,
I found that it suddenly stops download updates and then appears to not do anything else. But if you check the server logs for softwareupdateserver you will see messages for disk full (it usually saves a certain percentage of the drive for free space).

Gabe Shackney
Princeton Public Schools

[Matt_Ellis]

In my case it shouldn't be a space issue. I have about 600gb available on the server. Ill double check when i get back into town just finished up my cca today :)

[Matt_Ellis]

So i did verify it, i have 600gb of space still open, i can try upgrading to the newest Server App, and see if that helps? im also going to search thought the log today as @gshackney recommended

[GabeShack]

Also I've found that setting the server to store the data on a separate drive, aside from the boot drive, had issues. I eventually got this working, and I think the newest Server app helped this a bit.

Gabe Shackney
Princeton Public Schools

[Matt_Ellis]

So everything is stored on one drive, i upgraded to 10.11.1 and updated the server app to 5.0.15, Enabled the updates i wanted. Was able to update my computer with them. a few hours later all show as disabled in server app.

Since this server isn't in production yet im half tempted to wipe the whole thing and start again maybe something got corrupted in an update.

[dgreening]

I am seeing this as well - OS X 10.10.5 with Server 5.0.4 - updates saving to the standard place on the system volume. I have set the system volume reserve to 10%, and the permissions on the config files are good. I go in and enable updates (10.11 updates in this case) and a couple of hours later they are disabled again.

[Matt_Ellis]

Glad its not just me, i am going to put in a call with Apple enterprise support later today. if i get any info from them i will post it here.

[Matt_Ellis]

So they have no initial clue, there asking me to run a log capture after the updates disable themselves so hopefully i can get that to then in a few hours.

[francksartori]

Hi from France. FYI, I confirm that the issue is still there with Server 5.0.15 on OS X 10.11.1. The enabled updates are disabled after some minutes or hours. That is really annoying. Did someone yet open an bug report at Apple and have received a "duplicate" status ?

[Matt_Ellis]

I havent put in a bug report yet as im working with there Enterprise support/ engineering team

[claven]

We are seeing the same thing. I've opened up case 976946712.

I've wiped and reinstalled with the same behavior after letting it sync up overnight. For us it is every update since 10/1/2015 that goes to the disabled state.

Mac Mini Late 2012
OS 10.11.1
Server 5.0.15

[dgreening]

I did a permissions repair on one of my Mini DPs first thing this morning, and so far the enablement status of 10.11 updates is holding steady. Stay tuned.

[Matt_Ellis]

Ill give that a shot, still havent heard back from apple. So i can't run disk permissions as im on 10.11 and it does that automajically :( but System integrity protection is on and should be maintaining that.

[dgreening]

Scratch that the updates just disabled themselves again. Here is what I am seeing in the swupd log file for the sync:

Nov 2 09:45:34 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: ========== Sync Started ==========
Nov 2 09:45:34 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Checking service data location: /Library/Server/Software Update/Data/html
Nov 2 09:45:34 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving catalog list
Nov 2 09:45:34 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving deprecated updates list
2015-11-02 09:45:34.738 swupd_syncd[25388:1711160] Failed to obtain sandbox extension for path=/var/empty/Library/Caches/swupd_syncd. Errno:2
Nov 2 09:45:35 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving remote catalog (STANDARD): index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Nov 2 09:45:37 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving remote catalog (STANDARD): index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Nov 2 09:45:40 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving remote catalog (STANDARD): index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Nov 2 09:45:41 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving remote catalog (STANDARD): index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Nov 2 09:45:42 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving remote catalog (STANDARD): index-lion-snowleopard-leopard.merged-1.sucatalog
Nov 2 09:45:44 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Retrieving remote catalog (STANDARD): index-leopard-snowleopard.merged-1.sucatalog
Nov 2 09:45:46 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: Updating local catalog: index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Nov 2 09:47:38 bosjssdp1.sapient.com swupd_syncd[25388] <Info>: *

Nothing out of the usual...

[Matt_Ellis]

If you want to open a ticket with apple my case# is 974711947. Claven i gave my rep your case number so they could compare. as it looks like we are on identical software / hardware.

[dgreening]

I upgraded one of my DPs to 10.11.1 and am going to watch and see if this problem still pops up...

[dgreening]

Installing 10.11.1 did not fix this issue. Updates still disabled... SIGH

[dgreening]

Something interesting that I found in the swupdate settings (by running /Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin fullstatus swupdate):

swupdate:state = "RUNNING"
swupdate:lastChecktime = 2015-11-02 21:30:05 +0000
swupdate:autoMirrorOnlyNew = no
swupdate:syncServiceState = "RUNNING"
swupdate:startTime = 2015-11-03 15:38:26 +0000
swupdate:lastProductsUpdate = 2015-11-03 15:37:56 +0000
swupdate:logPaths:swupdateServiceLog = "/Library/Server/Software Update/Log/swupd_syncd.log"
swupdate:logPaths:swupdateErrorLog = "/Library/Server/Software Update/Log/swupd_error.log"
swupdate:logPaths:swupdateAccessLog = "/Library/Server/Software Update/Log/swupd_access.log"
swupdate:pluginVers = "10.11.111 (111)"
swupdate:checkError = no
swupdate:autoEnable = no
swupdate:updatesDocRoot = "/Library/Server/Software Update/Data/"
swupdate:hostServiceState = "RUNNING"
swupdate:autoMirror = yes
swupdate:numOfEnabledPkg = 1011
swupdate:servicePortsAreRestricted = "NO"
swupdate:numOfMirroredPkg = 1207
swupdate:setStateVersion = 1
swupdate:syncStatus = "DONE"
swupdate:readWriteSettingsVersion = 1

I have enabled all but two updates in the GUI, so the enabled count should be 1205, not 1011. That is very strange! I submitted a bug report with Apple...

[Matt_Ellis]

Mines even more fun after enabling about 30 updates i see this: it has to be some sort of bug not showing the correct amount of whats enabled then a process runs about 3 hours later and fixes the UI issue that shows them as enabled.

swupdate
swupdate:state = "RUNNING"
swupdate:lastChecktime = 2015-11-03 11:00:05 +0000
swupdate:autoMirrorOnlyNew = no
swupdate:syncServiceState = "RUNNING"
swupdate:startTime = 2015-10-29 20:41:02 +0000
swupdate:lastProductsUpdate = 2015-11-03 11:14:13 +0000
swupdate:logPaths:swupdateServiceLog = "/Library/Server/Software Update/Log/swupd_syncd.log"
swupdate:logPaths:swupdateErrorLog = "/Library/Server/Software Update/Log/swupd_error.log"
swupdate:logPaths:swupdateAccessLog = "/Library/Server/Software Update/Log/swupd_access.log"
swupdate:pluginVers = "10.11.111 (111)"
swupdate:checkError = no
swupdate:autoEnable = no
swupdate:updatesDocRoot = "/Library/Server/Software Update/Data/"
swupdate:hostServiceState = "RUNNING"
swupdate:autoMirror = yes
swupdate:numOfEnabledPkg = 0
swupdate:servicePortsAreRestricted = "NO"
swupdate:numOfMirroredPkg = 1205
swupdate:setStateVersion = 1
swupdate:syncStatus = "DONE"
swupdate:readWriteSettingsVersion = 1

[dgreening]

Hummm appears Server 5.0.15 is out as an update for 5.0.4...

[dgreening]

Server 5.0.15 with OS X 10.11.1 still disabling the updates.... Sigh... Does anyone know where ASUS stores the enablement status of updates?

[Matt_Ellis]

So spoke to apple they cant reproduce the issue, but believe a combination of reformat / reinstall of the system plus having my network guys setting up a A record and PTR record in DNS might resolve the issue.

Waiting on my network team to do its part. will let you know once i get it all up and running again.

[dgreening]

Yeah reformat of the system isn't going to fly. Our servers resolve both forward and reverse, and are not accessible outside of our network. They all have A records, but not PTR records.

Here is a thread over on Apple Support about this: Apple Support Discussion

Maybe time to look at Reposado more closely...

[Matt_Ellis]

Hehe i started that thread, there first got no help so opened this one up here.

[dgreening]

So I did a bit more digging through the config files for ASUS and found that the enablement status <key>enable</key> in /Library/Server/Software Update/Status/com.apple.sever.swupdate.plist is not being set to true when enabling an update (10.11.1 in this case). You enable it in the GUI and look at that file, and you will see that the status is still disabled.

It seems like there is some disconnect between what you do in the GUI and the status in this file. When I enable an update I do see this file as modified, but it doesn't set the enabled update as true...

I compared the file permissions for for the config files with an older SUS server (10.8.5, server 2 based) with the newer ones, and everything seems to line up. No lock flags (system or user) on that plist...

[claven]

I heard back from Apple support on this. Hopefully a fix is around the bend.

Here is the CliffsNotes version:

After researching this exhibited symptom, it is related to a similar report currently being investigated by Engineering. I have associated this report to the case and provided the logs to the report to help further the investigation. In testing, it has been reproduced on OS X 10.10.5 / OS X Server 5.0.4 as well. Once there is information to offer you, I will do so promptly.

[dgreening]

Excellent. Thanks for the update. I expect to be speaking with Apple on this tomorrow as well.

[sean]

If you are only just setting up a SUS, then you might want to look at Reposado instead:

Reposado overview

Reposado github repo

and if you like a web interface over command line:

Margarita

It is more flexible than Apple's, with multiple branches for different machines, allows for deprecated updates if you desire and you aren't limited by the server OS version.

Intro to Reposado

[dgreening]

Yep, I have used Reposado in a prior job, and it's quite nice. We are a ASUS environment here, and will continue to be for the foreseeable future.

[Matt_Ellis]

My guy is working with engineering as well, he couldn't reproduce it but they did tell me it was a know issue. ill look into those other programs as im not tied to just Apples SUS

[Matt_Ellis]

Ok so my issue is resolved. not sure why this worked but so far for 24 hours my updates have not been disabled

Keep in mind this was a test machine but here are the steps i took:

1. Setup a fully static ip and A/ PTR record
2. Fully reformatted and install / updated 10.11.1
3. Downloaded Server app and setup DNS (all i did was turn it on) and software update service
4. downloaded and enabled all updates from 2015 minus the voice packs.

And 24 hours later i still have enabled updates!

[dgreening]

Interesting. Unfortunately wiping and re-installing on our DPs is not an option, as we have 30+ around the globe. Apple suggested turning off automatic downloads, but that didn't have any effect.

[kishjayson]

Ran into this exact same issue on two separate servers today, both of which are running OS X Yosemite (10.10.5). One server is currently running Server 5.0.4 while the other is 5.0.15. I contacted AppleCare Enterprise Support today and they were able to reproduce the issue on their test system as well. Case Number is 986233263.

Thank you @dgreening for figuring out where the Enable flag is being stored. I'm going to see about scripting something to change this flag on my test server as an interim workaround.

I'll let everyone know if it's successful.

[kishjayson]

UPDATE

I stopped services on both servers, then deleted the contents of /Library/Server/Software Update/. I then configured the Software Update Server to Manual mode, and left "Automatically download new updates" unchecked.

I then selected all of the available updates, then chose "Download and Enable". This allowed each of the updates to be downloaded successfully and then properly set the enabled flag correctly in /Library/Server/Software Update/Status/com.apple.server.swupdate.plist.

While not yet tested, I presume that stopping services, then manually updating this flag from <true/> to <false/> should effectively disable the applicable update as well.