Apple SUS suddenly not working

@rharrington If you enter that URL in a browser do you get the catalog from your SUS?

@StoneMagnet , no I get the error:

Not Found

The requested URL /index.sucatalog was not found on this server.

However, I don't know if it makes a difference but if I go to, http://servername.ourdomain.com:8088/catalogs.sucatalog I get this:

CatalogsList http://servername.ourdomain.com:8088/content/catalogs/others/index-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-lion-snowleopard-leopard.merged-1.sucatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog PrimaryCatalog http://servername.ourdomain.com:8088/content/catalogs/others/index-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog

Just had the same issue crop up yesterday. I am still trying to figure out what went wrong.

Just had the same issue crop up yesterday. I am still trying to figure out what went wrong.

@rharrington The index.sucatalog URL should return the catalog appropriate for the OS version loading it. Would it be possible for you to set up another system with Server.app running SUS to see if it exhibits the same problem? I run a NetSUS rather than Server.app for my SUS, and it's working normally, so I'm wondering if there's an issue in what Apple is downloading to Server.app SUSes.

Just made a new OS install and fresh Server install and I see the same thing. What is odd, and I'm wondering if this is the problem, is that the index files are all created as aliases and they appear to be broken. @rharrington are you seeing the same thing?

@gabriel_martinez Same thing here:

Spent some time with Apple on the phone. We got it working by deleting the broken fixing the broken aliases in the html folder. The index-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog file should be an alias of /Library/Server/Software Update/Data/html/content/catalogs/others/index-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.composite.

We then had to point the clients to the new url of "http://servername.domain.com:8088/index-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1 2.sucatalog" and it worked.

Hope that helps.

@gabriel_martinez Thanks for posting the workaround for Server.app. I forced an update on my NetSUS, and it seems whatever Apple did to break the Server.app SUS doesn't affect Reposado. Did Apple by any chance acknowledge it as a known problem, or have an ETA on a true fix?

@StoneMagnet They wouldn't acknowledge anything.

@gabriel_martinez Hmmm, that did not work for me, I still received the "Can't load data from the Apple Software Update server." error. Also, after a reboot, the Alias changes back to the original one with the broken link.

Hello, maybe not quite related to the original topic but it's been a few days I am having problems with our SUS becoming unusable...
Each time SUS syncs, all updates are deleted and downloaded again with errors
There seems to be a problem with catalog updates ("Product file URL contains possible security violation." plus a couple of updates are showing "* Internal error creating cache data for product XXX-XXXXX")

Part of the service log at the bottom

The only custom setting is "service data" location is not the boot disk but another one.
Auto-copy (ALL) option is ON
Auto-enable option is OFF
Tried both with OSX 10.12.5/Server 5.3 (16S4123) and OSX 10.11.6/Server 5.2 (16S1195) and reinstalling from scratch.

Any help would be greatly appreciated!
Many thanks for your time
Ciao
Carlo

Jul 19 14:59:44  swupd_syncd[20449] <Info>: * Auto-copy (ALL) option is ON.  Applying option to eligible products.
Jul 19 14:59:44  swupd_syncd[20449] <Info>: * Auto-enable option is OFF. No products were changed.
Jul 19 14:59:44  swupd_syncd[20449] <Info>: * Manual mode - auto-download is ON, auto-enable option is OFF. Canceling pending enable requests only.
Jul 19 14:59:44  swupd_syncd[20449] <Info>: Previously copied product 091-00008 missing from local storage. This product will be re-download.
Jul 19 14:59:44  swupd_syncd[20449] <Info>: Previously copied product 091-00006 missing from local storage. This product will be re-download.
Jul 19 14:59:44  swupd_syncd[20449] <Info>: Previously copied product 091-21496 missing from local storage. This product will be re-download.
Jul 19 14:59:44  swupd_syncd[20449] <Info>: Previously copied product 091-22844 missing from local storage. This product will be re-download.
Jul 19 14:59:44  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 091-00004.
Jul 19 14:59:44  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 031-95676.
Jul 19 14:59:45  swupd_syncd[20449] <Info>: Updating local catalog: index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Jul 19 15:01:34  swupd_syncd[20449] <Info>: * Auto-copy (ALL) option is ON.  Applying option to eligible products.
Jul 19 15:01:34  swupd_syncd[20449] <Info>: * Auto-enable option is OFF. No products were changed.
Jul 19 15:01:34  swupd_syncd[20449] <Info>: * Manual mode - auto-download is ON, auto-enable option is OFF. Canceling pending enable requests only.
Jul 19 15:01:34  swupd_syncd[20449] <Info>: Previously copied product 091-00008 missing from local storage. This product will be re-download.
Jul 19 15:01:34  swupd_syncd[20449] <Info>: Previously copied product 091-00006 missing from local storage. This product will be re-download.
Jul 19 15:01:34  swupd_syncd[20449] <Info>: Previously copied product 091-22844 missing from local storage. This product will be re-download.
Jul 19 15:01:34  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 031-95676.
Jul 19 15:01:34  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 091-00004.
Jul 19 15:01:36  swupd_syncd[20449] <Info>: Updating local catalog: index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Jul 19 15:02:47  swupd_syncd[20449] <Info>: * Auto-copy (ALL) option is ON.  Applying option to eligible products.
Jul 19 15:02:47  swupd_syncd[20449] <Info>: * Auto-enable option is OFF. No products were changed.
Jul 19 15:02:47  swupd_syncd[20449] <Info>: * Manual mode - auto-download is ON, auto-enable option is OFF. Canceling pending enable requests only.
Jul 19 15:02:47  swupd_syncd[20449] <Info>: Previously copied product 091-00008 missing from local storage. This product will be re-download.
Jul 19 15:02:47  swupd_syncd[20449] <Info>: Previously copied product 091-00006 missing from local storage. This product will be re-download.
Jul 19 15:02:47  swupd_syncd[20449] <Info>: Previously copied product 091-22844 missing from local storage. This product will be re-download.
Jul 19 15:02:47  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 091-00004.
Jul 19 15:02:47  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 031-95676.
Jul 19 15:02:47  swupd_syncd[20449] <Info>: Updating local catalog: index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Jul 19 15:03:55  swupd_syncd[20449] <Info>: * Auto-copy (ALL) option is ON.  Applying option to eligible products.
Jul 19 15:03:55  swupd_syncd[20449] <Info>: * Auto-enable option is OFF. No products were changed.
Jul 19 15:03:55  swupd_syncd[20449] <Info>: * Manual mode - auto-download is ON, auto-enable option is OFF. Canceling pending enable requests only.
Jul 19 15:03:55  swupd_syncd[20449] <Info>: Previously copied product 091-00008 missing from local storage. This product will be re-download.
Jul 19 15:03:55  swupd_syncd[20449] <Info>: Previously copied product 091-00006 missing from local storage. This product will be re-download.
Jul 19 15:03:55  swupd_syncd[20449] <Info>: Previously copied product 091-22844 missing from local storage. This product will be re-download.
Jul 19 15:03:55  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 091-00004.
Jul 19 15:03:55  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 031-95676.
Jul 19 15:03:57  swupd_syncd[20449] <Info>: Updating local catalog: index-lion-snowleopard-leopard.merged-1.sucatalog
Jul 19 15:05:51  swupd_syncd[20449] <Info>: * Auto-copy (ALL) option is ON.  Applying option to eligible products.
Jul 19 15:05:51  swupd_syncd[20449] <Info>: * Auto-enable option is OFF. No products were changed.
Jul 19 15:05:51  swupd_syncd[20449] <Info>: * Manual mode - auto-download is ON, auto-enable option is OFF. Canceling pending enable requests only.
Jul 19 15:05:51  swupd_syncd[20449] <Info>: Previously copied product 091-00008 missing from local storage. This product will be re-download.
Jul 19 15:05:51  swupd_syncd[20449] <Info>: Previously copied product 091-00006 missing from local storage. This product will be re-download.
Jul 19 15:05:51  swupd_syncd[20449] <Error>: *** Internal error creating cache data for product 091-00004.
Jul 19 15:05:53  swupd_syncd[20449] <Info>: Updating local catalog: index-leopard-snowleopard.merged-1.sucatalog
Jul 19 15:06:36  swupd_syncd[20449] <Info>: * Auto-copy (ALL) option is ON.  Applying option to eligible products.
Jul 19 15:06:36  swupd_syncd[20449] <Info>: * Auto-enable option is OFF. No products were changed.
Jul 19 15:06:36  swupd_syncd[20449] <Info>: * Manual mode - auto-download is ON, auto-enable option is OFF. Canceling pending enable requests only.
Jul 19 15:06:38  swupd_syncd[20449] <Info>: Updating product updates information
Jul 19 15:06:39  swupd_syncd[20449] <Info>: Starting product update downloads
Jul 19 15:06:40  swupd_syncd[20449] <Info>: Downloading: Gatekeeper Configuration Data 1.0 (ID: 091-22844, posted: 2017-07-18 18:55:37 +0000)
Jul 19 15:06:44  swupd_syncd[20449] <Info>: Finished downloading product: 091-22844

and

Jul 19 15:06:44  swupd_syncd[20449] <Info>: Downloading: Chinese Word List Update 5.31 (ID: 091-22078, posted: 2017-07-18 17:00:14 +0000)
Jul 19 15:06:44  swupd_syncd[20449] <Info>: Finished downloading product: 091-22078
Jul 19 15:06:44  swupd_syncd[20449] <Info>: Downloading: Safari Technology Preview 11.0 (ID: 091-21439, posted: 2017-07-12 16:47:30 +0000)
Jul 19 15:06:53  swupd_syncd[20449] <Info>: Finished downloading product: 091-21439
Jul 19 15:06:53  swupd_syncd[20449] <Info>: Downloading: Safari Technology Preview 11.0 (ID: 091-21957, posted: 2017-07-12 16:47:30 +0000)
Jul 19 15:06:59  swupd_syncd[20449] <Info>: Finished downloading product: 091-21957
Jul 19 15:06:59  swupd_syncd[20449] <Info>: Downloading: Core Suggestions Configuration Data 800 (ID: 091-21496, posted: 2017-07-11 17:30:05 +0000)
Jul 19 15:07:00  swupd_syncd[20449] <Info>: Finished downloading product: 091-21496
Jul 19 15:07:00  swupd_syncd[20449] <Info>: Downloading: Canon Laser Printer Software Update 3.2 (ID: 091-00006, posted: 2017-07-06 15:56:15 +0000)
Jul 19 15:07:39  swupd_syncd[20449] <Info>: Finished downloading product: 091-00006
Jul 19 15:07:39  swupd_syncd[20449] <Info>: Downloading: EPSON Printer Software Update 3.3 (ID: 091-00008, posted: 2017-07-06 15:56:15 +0000)
Jul 19 15:16:55  swupd_syncd[20449] <Info>: Finished downloading product: 091-00008
Jul 19 15:16:55  swupd_syncd[20449] <Info>: Downloading: Brother Printer Software Update 4.1.1 (ID: 031-95676, posted: 2017-07-06 15:56:15 +0000)
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Product file URL contains possible security violation.
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Product ID: "031-95676"; file URL: "(null)"
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Local path: (null)
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Reason: file URL contains invalid scheme (should be "http")
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** The suspect product file will not be downloaded.
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Product file URL contains possible security violation.
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Product ID: "031-95676"; file URL: "(null)"
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Local path: (null)
Jul 19 15:16:55  swupd_syncd[20449] <Error>: *** Reason: file URL contains invalid scheme (should be "http")

We are seeing the same thing here, and rebuilding the links did not work either.

Well, in light of this not working and Apple SUS being depreciated shortly, I have switched everything over to the caching server. Seems to be working great so far!

@rharrington Are all your Macs running macOS Sierra? It's incredibly unclear in what cases Caching Server is used by the softwareupdate tool, especially for pre-Sierra systems, so you might want to consider setting up a Reposado instance for SUS emulation (or a NetSUS instance which incorporates Reposado). That is still working. If you go the NetSUS route, note that the getting_started.md document is out of date - you'll need 500GB of disk space rather than the 300GB that document lists. Another caution on the NetSUS route, the version of reposadocommon.py included with NetSUS 4.1.0 is out of date compared to the version in the main Reposado repository, so you'll have to manually merge the changes if you want your NetSUS to download Sierra updates.

@StoneMagnet 10.11 and 10.12. I have been watching the debug log for the past 24 hours and it appears that all of my systems are in fact able to both cache updates and also grab updates from the cache as well.

@rharrington Interesting. I wonder if Apple finally pulled the plug on the SUS in Server.app? I know they want people to move to Caching Server, but until that allows enabling/disabling of specific updates a lot of organizations aren't going to want to make the switch. Not to mention the lack of an analog like Reposado to allows hosting the service on non-Apple hardware. My Reposado instance runs under Ubuntu on a high-end Lenovo VM host with 10Gb network connectivity while my caching servers are 2012 Mac minis with 1Gb connections. It should be clear which system I'd prefer to serve updates from. (Yes, I could get a 10Gb adapter for a mini, or use 802.3ad to combine multiple 1Gb connections, but I like the redundancy of multiple minis)

I revisited this today and it turns out that Symantec Endpoint was blocking all the inbound connections. For anyone else having this issue, I would look first at your virus software and firewall logs.