Posted on 01-26-2021 07:00 AM
Hello all, Been a while since I last posted anything, but this has become a head scratcher for me.
I have a 10.26.1 Jamf Pro instance on a Windows VM in AWS. We have had this in place there for a while with no issues, and we haven't made any changes to the server or its configuration lately. The database is hosted locally on this server using MySQL 8.0.
The problem is that there is high resource usage that has been operating continuously for at least a day now. We noticed this because the server began having issues deploying software (policies). Upon inspection, the C drive was about 95% full, and lots of disk activity. I added additional storage to that drive, but the usage persists. We have reboot, and even restored to a snapshot from the previous day, all with no change. The policies are working again, but the resource usage is still very concerning.
I've attached screenshots here. In the Network section you can see that the tomcat OUTBOUND traffic is the same as the mysql INBOUND. MySql.exe is directed to the FQDN on this server. And unless I'm mistaken about IPv6 abbreviation, Tomcat8.exe is directed from the LOCALHOST address of the server. Both Tomcat and MySql are pegging the CPU. The local disk IO seems concentrated on "c:programdatamysql server 8.0dataibdata1" and "c:programdatamysql server 8.0dataib_logfile1".
I really appreciate any help with this. Thanks in advance,
-Pat
Posted on 01-26-2021 08:13 AM
@ferrispd If you go here:
C:/ProgramData/MySQL/MySQL Server 8.0/Data
Do you see a lot of files building up that have a format similar to:
JAMF.bin.0001
Please let me know.
Posted on 01-26-2021 12:24 PM
Chuey, I checked and didn't find any those. There are 2 of these undo_000X files, about 50 MB each.
Update: we found that one of the admins set something that was causing a policy to run over and over again on all devices. Resolving that improved the server responsiveness. Those network speeds between TomCat and MySQL both spike high numbers, but it isn't persistent like before. I'm still curious if that is normal behavior.
-Pat
Posted on 01-27-2021 04:43 AM
Good find -- Depending on how many devices that policy was going out to and how big could def. be typical behavior.
Posted on 01-27-2021 06:50 AM
it was about 600-700 devices performing the task every 15 minutes. Insane.