Thank you for your help everyone! I figured it out working with the Jamf support. The solution was that there were duplicate Configuration Profile Entries in the Database. Here is how to solve it:
Check if you have those duplicates with the following MySQL Command:
SELECT COUNT(uuid) AS 'dupes', uuid FROM complex_payload_sub_payloads GROUP BY uuid HAVING COUNT(uuid) > 1;
If you have them continue like this:
1. Create a database backup for security
2. Stop Tomcat
3. Remove duplicates with
DELETE c1 FROM complex_payload_sub_payloads c1 INNER JOIN complex_payload_sub_payloads c2 WHERE c1.id > c2.id AND c1.uuid = c2.uuid;
4. Check if duplicates are gone:
SELECT COUNT(uuid) AS 'dupes', uuid FROM complex_payload_sub_payloads GROUP BY uuid HAVING COUNT(uuid) > 1;
5. Start Tomcat