I've been struggling with the JDS for a few days now and I was wondering if anyone had any tips on tuning or tweaking any settings to improve the performance. My JSS and root JDS are on the same xserve running 10.8.5. My master distribution point is an AFP server in the same rack, which has performed quickly and reliably pre-9.x.
From my observation, the JDS replicates in an interesting manner. It seems when I replicate using Casper Admin, the packages transfer from my master distribution point into the JSS mysql database (downloadable_file_chunk_data table), then once the file is completely transferred the JDS is notified to download it. While downloading, the package appears to sit in /tmp and move to the JDS share once it is fully "downloaded" from the local database.
My issue is that in attempting to perform my full first replication to the root JDS, the mysql database balloons up to 10-50 GB and mysql becomes mostly unresponsive. When this happens, the JAMFSoftwareServer.log gets flooded with repeated mysql connection errors and all JSS tasks slow to a snail's pace. These are the log entries I'm seeing, grep-ed to show only the error lines:
2013-10-10 09:58:37,372 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 5
2013-10-10 09:58:45,690 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 4
2013-10-10 09:58:52,693 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 3
2013-10-10 09:58:59,697 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 2
2013-10-10 09:59:06,704 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 5
2013-10-10 09:59:13,729 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 5
2013-10-10 09:59:20,732 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 4
2013-10-10 09:59:27,751 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 3
2013-10-10 09:59:34,755 [ERROR] [atch-thread] [BoneCP ] - Failed to acquire connection to jdbc:mysql://localhost:3306/jamfsoftware?characterEncoding=utf8&useUnicode=true&jdbcCompliantTruncation=false. Sleeping for 7000 ms. Attempts left: 2
If I interrupt the Casper Admin replication, the JDS eventually catches up overnight and retrieves the packages living in the database and JSS performance is back to normal. If I run another replication to get the remaining packages into the JDS, the files copy into the database but the JDS never starts to "download" them from the local database until I optimize the mysql tables. Optimizing seems to kickstart the JDS to download the packages waiting in the database. I changed mysql's max packet size to 256 MB and performance seems improved, but it might just be a placebo.
I would appreciate any feedback or tips that might improve my situation. I've been working for over a week to try and replicate about 250 packages (188 GB) to my root JDS instance. I'd be happy to provide any other relevant information about our setup to assist troubleshooting.
