High Sierra Supplemental update breaks recon

[remyb]

Hi!
Yesterday apple released a supplemental update in order to fix the password leak for encrypted APFS disks. Around the same time all the High Sierra macs in our environment started failing their recons. Here's what I've noticed:

Recon's resulting in an Unknown error

sudo jamf recon
Password:
Retrieving inventory preferences from ***...
Finding extension attributes...
Locating applications...
Locating accounts...
Locating package receipts...
Searching path: /Applications
Locating hard drive information...
Locating software updates...
Locating printers...
Locating hardware information (Mac OS X 10.13.0)...
Submitting data to ***...

There was an error.

     Unknown Error - An unknown error has occurred.

JAMF server log full of SQL errors:

2017-10-06 09:44:18,991 [WARN ] [Tomcat-15  ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001
2017-10-06 09:44:18,992 [ERROR] [Tomcat-15  ] [SqlExceptionHelper       ] - Data too long for column 'version' at row 1
Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?)
Query is:
insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?)
2017-10-06 09:44:18,993 [ERROR] [Tomcat-15  ] [lientCommunicationServlet] - Error processing communication content - org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement

Software updates seem to be the culprit here.... Let's test that theory:

softwareupdate -l
Software Update Tool

Finding available software
Software Update found the following new or updated software:
   * macOS High Sierra 10.13 Supplemental Update-
    macOS High Sierra 10.13 Supplemental Update ( ), 893081K [recommended] [restart]

There is an update available! Lets run it

5 minutes and a reboot later, the machine is updated. Let's run a new recon:

sudo jamf recon
Password:
Retrieving inventory preferences from ***...
Finding extension attributes...
Locating package receipts...
Locating hard drive information...
Locating accounts...
Locating applications...
Searching path: /Applications
Locating software updates...
Locating printers...
Locating hardware information (Mac OS X 10.13.0)...
Submitting data to ***...
<computer_id>84</computer_id>

Presto!
Has anyone else run into this so far?

[Niels_Illem]

Updatet this morning, but I still can Recon.... We're on 9.101.0

sudo jamf recon
Retrieving inventory preferences from https://xyz.jamfcloud.com:443/...
Finding extension attributes...
Locating accounts...
Locating hard drive information...
Locating applications...
Locating package receipts...
Locating software updates...
Searching path: /Applications
Locating printers...
Locating hardware information (Mac OS X 10.13.0)...
Gathering application usage information...
Submitting data to https://xyz.jamfcloud.com:443/...
<computer_id>902</computer_id>

jamf version
version=9.101.0-t1504998263

[remyb]

@Niels.Illem The issue presents itself before installing the Supplemental Update, and is fixed after updating

[JPDyson]

Sounds like somebody's not scrubbing their inputs...

[nimitz]

@JPDyson What does that mean exactly?

[dgreening]

I am still on 9.90 and I am not seeing this before or after the supplemental.

[cskj]

I am seeing this as well. I'm following up with Support to make sure they are aware of the problem.

@nimitz regarding the comment about scrubbing inputs. The assumption being made is the JAMF code that's handling the "macOS High Sierra 10.13 Supplemental Update-" line is failing due to special characters not being handled correctly.

[dgreening]

Ah perhaps I am not seeing this because I am not collecting software updates...

[JPDyson]

@cskj Is correct; my assumption is that the name of the supplemental update is breaking a query.

[Sonic84]

Looks like it's trying to pass standard in to the database via the erroneous '-'?

Does this qualify as a mySQL injection attack vector?

[cskj]

Hey guys, after looking at the logs a bit deeper it looks like it might just be a simple parsing issue.

2017-10-06 19:56:33,447 [DEBUG] [Tomcat-61 ] [SQL ] - insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values (?, ?, ?, ?, ?, ?) ... 2017-10-06 19:56:33,448 [TRACE] [Tomcat-61 ] [BasicBinder ] - binding parameter [6] as [VARCHAR] - [, 412289K [recommended] [restart]] ... 2017-10-06 19:56:33,448 [ERROR] [Tomcat-61 ] [SqlExceptionHelper ] - Data too long for column 'version' at row 1

", 412289K [recommended] [restart]" is a pretty unusual version to try and store in the database.

Also, as a workaround so that your systems continue to check in, you can disable the "Collect available software updates" from the Inventory Collection settings. Just remember to re-enable it after a patch from JAMF has been released.

[cskj]

@Sonic84 after looking at this some more, I would not classify this as a vulnerability to SQL Injection; they are passing via parameter and the error we are seeing isn't a broken SQL Query, but an error stating that the value is too long to insert into the database.

[remyb]

JAMF Support just notified me that they are aware of the issue and are investigating. I will update this when I have more info.

[remyb]

The answer JAMF support sent me was basically 'create a policy to apply the update manually'. While this is a nice workaround, and something we obviously already did, it's not quite the fix I was hoping for.

[tcandela]

received this unknown error also when running recon on 10.13

running softwareupdate -ai. supplemental update is installing right now, will re-run 'jamf recon' after update installs and computer restarts.

so after the supplemental update gets installed and computer restarted 'sudo jamf recon' fully runs without error. Computer is listed in inventory with OS as 10.13

[alexjdale]

It looks like the new 10.13.2 supplemental update is doing the same thing for us, on 9.101.

[maurits]

Yes, same errors with 10.13.2 supplemental update, JSS 10.0.0: jamf recon fails

[donmontalvo]

Well, thought this did not effect us (JSS 9.101.4-t1507816971)...opening a ticket with Jamf now...

bash-3.2# jamf recon
Retrieving inventory preferences from https://server.domain.com:8443/...
Finding extension attributes...
Locating accounts...
Locating applications...
Locating package receipts...
Locating hard drive information...
Searching path: /Applications
Locating software updates...
Locating plugins...
Locating printers...
Searching path: /Library/Internet Plug-Ins
Locating hardware information (Mac OS X 10.13.2)...
Submitting data to https://server.domain.com:8443/...

There was an error.

     Unknown Error - An unknown error has occurred.

bash-3.2#

[KyleH]

There is some more information available in this post:

10.13.2 Supplemental Update Workaround

[donmontalvo]

We opened a ticket with Apple...looks like their coding elves bonked the update name:

Note the trailing minus sign Update-, and the empty open/close parenthesis in Update ( ) where a version or build number belongs.

# softwareupdate -l
Software Update Tool

Finding available software
Software Update found the following new or updated software:
   * macOS High Sierra 10.13.2 Supplemental Update- 
    macOS High Sierra 10.13.2 Supplemental Update ( ), 138293K [recommended] [restart]
#

Compared to a 10.12.4 computer's list of updates:

# sudo softwareupdate -l
Password:
Software Update Tool
Copyright 2002-2015 Apple Inc.

Finding available software
Software Update found the following new or updated software:
   * OS X El Capitan Update-10.11.6
    OS X El Capitan Update (10.11.6), 1510350K [recommended] [restart]
   * RAWCameraUpdate6.21-6.21
    Digital Camera RAW Compatibility Update (6.21), 7695K [recommended]
   * 031-99115-5.1
    HP Printer Software Update (5.1), 159247K [recommended]
   * iTunesX-12.7.2
    iTunes (12.7.2), 264184K [recommended]
#

Opening ticket with Jamf as well, to see if they can do something to keep Jamf Pro from choking on Apple's missteps.

So Jamf Pro chokes when trying to pull the list of available updates...and that prevents us from scoping the patch to computers that show it in the softwareupdate -l list:

# jamf recon
Retrieving inventory preferences from https://server.domain.com:8443/...
Finding extension attributes...
Locating accounts...
Locating applications...
Locating hard drive information...
Locating package receipts...
Searching path: /Applications
Locating software updates...
Locating plugins...
Locating printers...
Searching path: /Library/Internet Plug-Ins
Locating hardware information (Mac OS X 10.13.2)...
Submitting data to https://server.domain.com:8443/...

There was an error.

     Unknown Error - An unknown error has occurred.

#

[timmy]

I don't have anything else to add except we're experiencing the same problem. Ran through everything above and can confirm it's the same issue. Just wanted to add "me too".

[ventura_torres]

Hi Everyone!

I spoke to Jamf support and they were able to provide me with a solution/workaround for the issue with 10.13.2 Macs being unable to report inventory after the supplemental update has been installed. In fact there are two solutions.

1. Go to All Settings > Computer Management > Inventory Collection > under the General tab uncheck the box next to option "Collect available software updates" > save. This will cause the Jamf Pro to stop collecting available software updates from every computer. If you need this option enabled option 2 may be better suited for you.

2. Adjust the JAMFSoftwareServerDatabaseSchema.xml to allow for the longer length. This is how it can be done.

   -1. Navigate to /usr/local/JSS/Tomcat/webapps/ROOT/WEB-INF/xml/ (path may differ if you are running Jamf Pro on a Windows Server or Mac)
   -2. Open JAMFSoftwareServerDatabaseSchema.xml in your text editor of choice -3. Make the following change:

<table> <table_name>available_software_updates</table_name> <column> <name>version</name> <type>varchar</type> <size>31</size> </column>

to

<column> <name>version</name> <type>varchar</type> <size>255</size> </column>

-4. Save and restart Tomcat

Hope this helps!

[guidotti]

Thanks.
This was driving me crazy.

[cubandave]

Same thing happens with 10.13.3 update actually. @ventura.torres do think your option 2 workaround will work there as well?

[remyb]

@cubandave We have adjusted the database schema and I can confirm that this workaround works perfectly

[donmontalvo]

@cubandave Apple Enterprise Support escalated our ticket, they admitted the empty parenthesis are supposed to contain the version number. Since they don't it causes problems, including bonking the display of those updates in Jamf Pro.

Note the "Version" for the 10.13.3 Combo Update, which was just released, and has the same empty parenthesis. Somewhere in a third world country, an Apple dev team manager is getting yelled at.

Luckily Jamf's ace team fixed it with the database schema tweak. :)

[remyb]

For those that applied this workaround please note:
Updating to a new version of JAMF pro will result in your server not starting back up as in: https://www.jamf.com/jamf-nation/articles/349/troubleshooting-the-jss-startup-suspended-issues

Don't panic, the value in the JAMFSoftwareServerDatabaseSchema.xml file was reverted to it's original state. Stop the JAMF service, re-edit the value back to 255 and start JAMF. Use this at your own risk.

[acaveny]

Does the 10.2.0 update fix the root of the problem? I was looking through the release notes and open bugs and couldn't find it....

[PhillyPhoto]

@remyb Is that only for the schema workaround and not the "Collect available software updates" workaround? I've only done the latter and plan on upgrading to 10.2.0 next Friday night.

[remyb]

It does not fix the root of the problem, it just reverts the schema to the original state. Which in turn 'breaks' the startup process as there are already values stored in the database that are bigger than the defined max value of 31.

I can't speak for the other workaround, but I suppose if that is just disabling the collection of available software updates, it won't cause any issue.

EDIT: After further investigating this, the upgrade to 10.2 in fact DID solve the root cause of this problem, as 'new' recon's now properly populate the database even if the macos update syntax is screwy

[donmontalvo]

@remyb wrote:

Don't panic, the value in the JAMFSoftwareServerDatabaseSchema.xml file was reverted to it's original state.

I thought Jamf put an end to stomping on our customized configuration files.

¡Ave María Purísima!

[mthakur]

A recent update — 041-62886 iTunes Device Support Update — also triggers this problem:

On the Mac endpoint (note the update's name "MobileDeviceSU- " has a space after the dash at the end, and also note the absence of a version number inside the parentheses):

$ sudo softwareupdate --list
Password:
Software Update Tool

Finding available software
Software Update found the following new or updated software:
   * 031-48032-4.1
    Xerox Printer Software Update (4.1), 73250K [recommended]
   * MobileDeviceSU- 
    iTunes Device Support Update ( ), 105795K [recommended] [restart]
$ jamf version
version=9.101.0-t1504998263
$

In the /var/log/install.log file:

$ less +'?softwareupdated.*updates found:' /var/log/install.log
May 20 18:59:50 softwareupdated[309]: 3 updates found:
        031-48032        | Xerox Printer Software Update 4.1
        041-56834        | Gatekeeper Configuration Data 166
        041-62886(R)     | iTunes Device Support Update
May 20 18:59:50 mec094321 softwareupdated[309]: SoftwareUpdate: 041-62886 (MobileDeviceSU- ): install-later not supported for restart-required update (options={
            customize = never;
            hostArchitectures = "i386,x86_64";
        })
$

On the JSS (after endpoint runs recon):

$ less '+?SQL Error' /Library/JSS/Logs/JAMFSoftwareServer.log
2019-05-20 15:39:23,972 [WARN ] [Tomcat-19  ] [SqlExceptionHelper       ] - SQL Error: 1406, SQLState: 22001
2019-05-20 15:39:23,972 [ERROR] [Tomcat-19  ] [SqlExceptionHelper       ] - **Data too long for column 'version' at row 1**
Query is: insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, versi
on) values (?, ?, ?, ?, ?, ?)
Query is:
insert into available_software_updates (computer_id, display_name, name, recommended, restart_required, version) values
 (?, ?, ?, ?, ?, ?)
2019-05-20 15:39:23,973 [ERROR] [Tomcat-19  ] [lientCommunicationServlet] - Error processing communication content - or
g.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.
hibernate.exception.DataException: could not execute statement
$

Workarounds:
- Same as above: Navigate in JAMF Pro to Computers --> Management Settings --> Inventory Collection -and uncheck "Collect available software updates".
- Run $ sudo softwareupdate --install 'MobileDeviceSU- ' on affected endpoints.

Fix:
- Upgrade JAMF Pro to latest version; apparently, this was fixed.

[a_holley]

Ok, we are encountering the same error on two laptops. We do not have "Collect available software updates" ticked, and I have checked for software updates and there are none available. I am completely stuck, I don't know what is causing the issues on these machines, but I need them to recon correctly. Any suggestions? Something I've missed?