I can't seem to find clear instructions anywhere for doing an in place upgrade from MySQL 5.6 to 5.7. The community installer shows no upgrade from 5.6.35 to 5.7.x. Do I need to remove and install, or install in parallel, and then import the database from a backup? Am I missing some obvious steps somewhere?
Solved! Go to Solution.
I would not recommend an upgrade to MySql 5.7.
I could not do any backups after upgrading, JAMF support told me that is not supported.
They need to update the documentation, because it states that 5.7 is required.
@isradame, so you updated to 9.98 and cannot do backups? Jamf support still said 5.7 is NOT supported even though the documentation specifically states MySQL 5.7 compatibility and is recommended? Are you backing up with the Backup Utility?
@ryanstayloradobe , yes. I upgraded to 9.98 and MySql 5.7. I could not do manual and also using the JSS Database backup utility.
I told Jamf support that their document states MySql 5.7 is recommended.
(From page 14 of the JSS Installation and Configuration Guide for Mac 9.98PDF document. "The JSS Database Utility requires a server with MySQL 5.6.x or 5.7.x (MySQL 5.7.x is recommended) installed.").
JAMF Support is aware about the situation.
@isradame I could make and restore backups just fine with MySQL 5.7 on Windows. I had to comment out line 62 in MySQL's my.ini configuration file, as well as change line 287 as follows (depending on your config, they may be on different line numbers:
This file's default location is C:ProgramDataMySQLMySQL Server 5.7my.ini
This may be a Mac side issue with macOS Sierra 10.12.4, MySql 5.7 and JAMF Pro 9.98.
I had to remove MySql 5.7 and re-install the 5.6 version.
I also have a specific username and password for the 'jamfsoftware" database. Not using the default.
@isradame have you tried making the changes suggested above to your my.cnf file? As far as I know the settings are stored in the same way irrespective of platform. If it doesn't exist in /etc/my.cnf then @bentoms describes how to create it here: https://macmule.com/2013/12/13/jss-database-utility-change-mysql-settings-greyed-out/
If you find a line saying no-beep, comment it out with a preceding # and up your max packet size to 512M.
Do you get a specific error message when you try to create a backup? The other thing I had to do was make the JSS Database Utility aware of the MySQL 5.7 binaries location - it prompted the first time it was launched after migrating.
@neil.martin83 yes I did.
Not only the issue with not able to do backups, but the MySql service was restarting automatically.
Here is the error I would get when I tried to perform a backup, thru the JAMF Utility or manually.
"Encountered an error during applications repair: mysql: [Warning] Using a password on the command line interface can be insecure.Error 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error:0"
MySQL 5.7 is supported for 9.98 and does work. Due to security fixes for previous version going to MySQL 5.7.14+ is recommend.
If your backup is failing with the JSS Database utility I would try running the command locally with the actual mysqldump binary, which the utility uses, and see what errors its presenting:
/path/to/mysql/bin/mysqldump -u root -p --lock-tables DatebaseName > /path/to/where/you/save/mybackup.sql #or windows version path omysqlinmysqldump.exe -u root -p --lock-tables DatebaseName > C:path owhereyousavemybackup.sql
Sometimes installing a newer version of MySQL is more of a second installation vs what we normally think of as an upgrade so you have the possibility of files (normally the my.ini/my.cnf) from previous versions of mysql remaining and being respected but some of their settings might be out of date or the new files might not have the settings you have previously configured.
"No-beep" was removed prior to version 5 of mysql I believe so that would likely cause issues if it was still there. You may have to check all instances of the my.cnf/my.ini on the system as they could be installed and respected in multiple locations depending on versions and installation methods. This KB on using Option Files can help you locate them.
Sometimes you need to increase your max_allowed_packet if you have a packet in your database that is larger than the default 16mb, but typically this is not the case. Normally the main thing to increase this for is when uploading printer ppds, and you will see an error in logs if this is the case. DO NOT just go increase this value unless you are having issues uploading files or are getting errors with your mysql dump because of a previously uploaded packet. If you are getting errors with the dump, look at the logs and see what size it says its failing for and increase to that value and not an arbitrary value. It is set at this default value for a reason and could cause problems if increased for no reason. I have seen rouge clients with built up logs due to proxies peg tomcat servers sending massive file submissions and since mysql was set to allow it, it took that large amount of data and in turn brought the server down. Also adjusting max_allowed_packet is not for scale or performance, it is literally the cap on the largest data packet that can be put in the database.
The way to confirm if your settings are being respected after the changes made and service restarted is to run the 'show variables' command while logged into mysql like:
#For All Variables show variables; #For specific variables show variables like 'max_allowed_packet';
@isradame the issue you are mentioning with MySQL starting/stopping in System Preferences will cause errors while connecting via command line or doing backups when its stopped, and is generally seen on a Mac and has been seen with previous versions of MySQL as well. We have had some success with modifying the table_open_cache values in the my.cnf but I would reach out to support to have them work with you on that.
Here is one other discussion on that issue on MySQL on macOS: https://www.jamf.com/jamf-nation/discussions/18352/mysql-5-7-9-on-10-11-2-showing-weird-behavior-whe...
@mike.paul I also ran into a forum to run mysqldump command, and I also got the same error.
On the issue with MySql database restarting in the system preference is not the same I had. I have seen this before, and yes you are correct. JAMF support told me the same thing. But my problem was different.
I would run /usr/local/mysql/bin/mysql -u root -p
and this error would come up.
Enter password: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)
I also do the command "ps uxa | grep mysql" and the only thing that will come back was.
ladmin 18266 0.0 0.0 2432804 816 s000 S+ 3:32PM 0:00.00 grep mysql
I tried even restarting the service manually and nothing. The only solution was to restart the Mac server.
@kirkmshaffer After a clean reload of the OS 10.12.4, a clean install of JAMF "9.98", latest Java "1.8" and the latest MySql "5.7". I still had the same problem.
Could not do manual or use the JSS Database utility for backups.
Also the mysql service was crashing.
Here is what I did to fix it. I do not deserve the credit. @zachary.fisher provided me with the instructions.
Step 1. I changed the Database.xml file located in /Library/JSS/Tomcat/webapps/ROOT/WEB-INF/xml/
Step 2. I created an empty my.cnf file in the etc folder.
sudo vi /etc/my.cnf
Then using the JSS Database Utility, I changed the settings.
The last three lines I added them manually.
My my.cnf file
max_allowed_packet=1024M # Updated by JSS Database Utility
max_connections=150 # Updated by JSS Database Utility
expire_logs_days=0 # Updated by JSS Database Utility
max_binlog_size=16M # Updated by JSS Database Utility
innodb_file_per_table = OFF
table_definition_cache = 400
table_open_cache = 400
Step 3. Ran the commands to repair the database.
/usr/local/mysql/bin/mysqlcheck -u root -p --repair NAMEOFDATABASE
@isradame apologies for the delay, here's my my.ini - bear in mind it's from the Windows version so I don't know what differences there might be between it and a my.cnf from Linux/macOS besides the paths to the database, logs etc. This is also from my dev/test JSS server - on the real one I have the MySQL Data folder located away from the C: drive (it's all hosted on virtual servers so not done for performance, but it helps a lot if something causes a drive to fill unexpectedly, which did happen once).
Other than my tweaks that I blogged about, one thing I noticed that's different from the 5.6 version is that query_cache is set to only 1MB by default, where it was previously much larger (I think 2GB).
# Other default tuning values # MySQL Server Instance Configuration File # ---------------------------------------------------------------------- # Generated by the MySQL Server Instance Configuration Wizard # # # Installation Instructions # ---------------------------------------------------------------------- # # On Linux you can copy this file to /etc/my.cnf to set global options, # mysql-data-dir/my.cnf to set server-specific options # (@localstatedir@ for this installation) or to # ~/.my.cnf to set user-specific options. # # On Windows you should keep this file in the installation directory # of your server (e.g. C:Program FilesMySQLMySQL Server X.Y). To # make sure the server reads the config file use the startup option # "--defaults-file". # # To run run the server from the command line, execute this in a # command line shell, e.g. # mysqld --defaults-file="C:Program FilesMySQLMySQL Server X.Ymy.ini" # # To install the server as a Windows service manually, execute this in a # command line shell, e.g. # mysqld --install MySQLXY --defaults-file="C:Program FilesMySQLMySQL Server X.Ymy.ini" # # And then execute this in a command line shell to start the server, e.g. # net start MySQLXY # # # Guildlines for editing this file # ---------------------------------------------------------------------- # # In this file, you can use all long options that the program supports. # If you want to know the options a program supports, start the program # with the "--help" option. # # More detailed information about the individual options can also be # found in the manual. # # For advice on how to change settings please see # http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html # # # CLIENT SECTION # ---------------------------------------------------------------------- # # The following options will be read by MySQL client applications. # Note that only client applications shipped by MySQL are guaranteed # to read this section. If you want your own MySQL client program to # honor these values, you need to specify it as an option during the # MySQL client library initialization. # [client] # pipe # socket=0.0 port=3306 [mysql] # no-beep default-character-set=utf8 # SERVER SECTION # ---------------------------------------------------------------------- # # The following options will be read by the MySQL Server. Make sure that # you have installed the server correctly (see above) so it reads this # file. # # server_type=2 [mysqld] # The next three options are mutually exclusive to SERVER_PORT below. # skip-networking # enable-named-pipe # shared-memory # shared-memory-base-name=MYSQL # The Pipe the MySQL Server will use # socket=MYSQL # The TCP/IP Port the MySQL Server will listen on port=3306 # Path to installation directory. All paths are usually resolved relative to this. # basedir="C:/Program Files/MySQL/MySQL Server 5.7/" # Path to the database root datadir=C:/ProgramData/MySQL/MySQL Server 5.7Data # The default character set that will be used when a new schema or table is # created and no character set is defined character-set-server=utf8 # The default storage engine that will be used when create new tables when default-storage-engine=INNODB # Set the SQL mode to strict # sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" # Enable Windows Authentication # plugin-load=authentication_windows.dll # General and Slow logging. log-output=FILE general-log=0 general_log_file="DL-JSSDEV-01.log" slow-query-log=1 slow_query_log_file="DL-JSSDEV-01-slow.log" long_query_time=10 # Binary Logging. # log-bin # Error Logging. log-error="DL-JSSDEV-01.err" # Server Id. server-id=1 # Secure File Priv. secure-file-priv="C:/ProgramData/MySQL/MySQL Server 5.7/Uploads" # The maximum amount of concurrent sessions the MySQL server will # allow. One of these connections will be reserved for a user with # SUPER privileges to allow the administrator to login even if the # connection limit has been reached. max_connections=181 # Query cache is used to cache SELECT results and later return them # without actual executing the same query once again. Having the query # cache enabled may result in significant speed improvements, if your # have a lot of identical queries and rarely changing tables. See the # "Qcache_lowmem_prunes" status variable to check if the current value # is high enough for your load. # Note: In case your tables change very often or if your queries are # textually different every time, the query cache may result in a # slowdown instead of a performance improvement. query_cache_size=1M # The number of open tables for all threads. Increasing this value # increases the number of file descriptors that mysqld requires. # Therefore you have to make sure to set the amount of open files # allowed to at least 4096 in the variable "open-files-limit" in # section [mysqld_safe] table_open_cache=2000 # Maximum size for internal (in-memory) temporary tables. If a table # grows larger than this value, it is automatically converted to disk # based table This limitation is for a single table. There can be many # of them. tmp_table_size=358M # How many threads we should keep in a cache for reuse. When a client # disconnects, the client's threads are put in the cache if there aren't # more than thread_cache_size threads from before. This greatly reduces # the amount of thread creations needed if you have a lot of new # connections. (Normally this doesn't give a notable performance # improvement if you have a good thread implementation.) thread_cache_size=10 #*** MyISAM Specific options # The maximum size of the temporary file MySQL is allowed to use while # recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE. # If the file-size would be bigger than this, the index will be created # through the key cache (which is slower). myisam_max_sort_file_size=100G # If the temporary file used for fast index creation would be bigger # than using the key cache by the amount specified here, then prefer the # key cache method. This is mainly used to force long character keys in # large tables to use the slower key cache method to create the index. myisam_sort_buffer_size=704M # Size of the Key Buffer, used to cache index blocks for MyISAM tables. # Do not set it larger than 30% of your available memory, as some memory # is also required by the OS to cache rows. Even if you're not using # MyISAM tables, you should still set it to 8-64M as it will also be # used for internal temporary disk tables. key_buffer_size=8M # Size of the buffer used for doing full table scans of MyISAM tables. # Allocated per thread, if a full scan is needed. read_buffer_size=64K read_rnd_buffer_size=256K #*** INNODB Specific options *** # innodb_data_home_dir=0.0 # Use this option if you have a MySQL server with InnoDB support enabled # but you do not plan to use it. This will save memory and disk space # and speed up some things. # skip-innodb # If set to 1, InnoDB will flush (fsync) the transaction logs to the # disk at each commit, which offers full ACID behavior. If you are # willing to compromise this safety, and you are running small # transactions, you may set this to 0 or 2 to reduce disk I/O to the # logs. Value 0 means that the log is only written to the log file and # the log file flushed to disk approximately once per second. Value 2 # means the log is written to the log file at each commit, but the log # file is only flushed to disk approximately once per second. innodb_flush_log_at_trx_commit=1 # The size of the buffer InnoDB uses for buffering log data. As soon as # it is full, InnoDB will have to flush it to disk. As it is flushed # once per second anyway, it does not make sense to have it very large # (even with long transactions). innodb_log_buffer_size=1M # InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and # row data. The bigger you set this the less disk I/O is needed to # access data in tables. On a dedicated database server you may set this # parameter up to 80% of the machine physical memory size. Do not set it # too large, though, because competition of the physical memory may # cause paging in the operating system. Note that on 32bit systems you # might be limited to 2-3.5G of user level memory per process, so do not # set it too high. innodb_buffer_pool_size=8M # Size of each log file in a log group. You should set the combined size # of log files to about 25%-100% of your buffer pool size to avoid # unneeded buffer pool flush activity on log file overwrite. However, # note that a larger logfile size will increase the time needed for the # recovery process. innodb_log_file_size=48M # Number of threads allowed inside the InnoDB kernel. The optimal value # depends highly on the application, hardware as well as the OS # scheduler properties. A too high value may lead to thread thrashing. innodb_thread_concurrency=8 # The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full. innodb_autoextend_increment=64 # The number of regions that the InnoDB buffer pool is divided into. # For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency, # by reducing contention as different threads read and write to cached pages. innodb_buffer_pool_instances=8 # Determines the number of threads that can enter InnoDB concurrently. innodb_concurrency_tickets=5000 # Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before # it can be moved to the new sublist. innodb_old_blocks_time=1000 # It specifies the maximum number of .ibd files that MySQL can keep open at one time. The minimum value is 10. innodb_open_files=300 # When this variable is enabled, InnoDB updates statistics during metadata statements. innodb_stats_on_metadata=0 # When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table # in a separate .ibd file, rather than in the system tablespace. innodb_file_per_table=1 # Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none. innodb_checksum_algorithm=0 # The number of outstanding connection requests MySQL can have. # This option is useful when the main MySQL thread gets many connection requests in a very short time. # It then takes some time (although very little) for the main thread to check the connection and start a new thread. # The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily # stops answering new requests. # You need to increase this only if you expect a large number of connections in a short period of time. back_log=80 # If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and # synchronize unflushed data to disk. # This option is best used only on systems with minimal resources. flush_time=0 # The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use # indexes and thus perform full table scans. join_buffer_size=256K # The maximum size of one packet or any generated or intermediate string, or any parameter sent by the # mysql_stmt_send_long_data() C API function. max_allowed_packet=512M # If more than this many successive connection requests from a host are interrupted without a successful connection, # the server blocks that host from performing further connections. max_connect_errors=100 # Changes the number of file descriptors available to mysqld. # You should try increasing the value of this option if mysqld gives you the error "Too many open files". open_files_limit=4161 # Set the query cache type. 0 for OFF, 1 for ON and 2 for DEMAND. query_cache_type=1 # If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the # sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization # or improved indexing. sort_buffer_size=256K # The number of table definitions (from .frm files) that can be stored in the definition cache. # If you use a large number of tables, you can create a large table definition cache to speed up opening of tables. # The table definition cache takes less space and does not use file descriptors, unlike the normal table cache. # The minimum and default values are both 400. table_definition_cache=1400 # Specify the maximum size of a row-based binary log event, in bytes. # Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256. binlog_row_event_max_size=8K # If the value of this variable is greater than 0, a replication slave synchronizes its master.info file to disk. # (using fdatasync()) after every sync_master_info events. sync_master_info=10000 # If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk. # (using fdatasync()) after every sync_relay_log writes to the relay log. sync_relay_log=10000 # If the value of this variable is greater than 0, a replication slave synchronizes its relay-log.info file to disk. # (using fdatasync()) after every sync_relay_log_info transactions. sync_relay_log_info=10000 # Load mysql plugins at start."plugin_x ; plugin_y". # plugin_load # MySQL server's plugin configuration. # loose_mysqlx_port=33060
@isradame Thanks a lot for sharing your solution. We ran in the same problems after updating MySQL from 5.6.x to 5.7.17, JSS 9.88, macOS 10.12.4 and the latest JDK. After rebooting the Mac we could ran the JSS Utility to make a manual backup. Once. The second try ended with the ERROR 2013. Only a machine reboot resolved the issue until the next backup try.
So we had first to modify the MaxPoolSize from 90 to 251 at /Library/JSS/Tomcat/webapps/ROOT/WEB-INF/xml/DataBase.xml (the other values were fine). Then we used @bentoms Reminder to change the MySQL Settings (was greyed out) and used the JSS Database Utility to change the max_allowed_packet Value from 16M to 1024M (a huge bump) and finally manually added the 3 lines at /etc/my.cnf.
innodb_file_per_table = OFF table_definition_cache = 400 table_open_cache = 400
After repairing the database with
/usr/local/mysql/bin/mysqlcheck -u root -p --repair jamfsoftware
our JSS did it's first automatic backup since the MySQL update.
Thank's again @zachary.fisher
Thanks @neil.martin83 for your MySQL upgrade procedure and documentation. Good stuff. Jamf should pay you for your effort in filling in the holes in their MySQL documentation :0)
Two things that caught my eye in your documentation:
1) You gave explicit details on stopping the JSS process (i.e.; Apache Tomcat) proir to installation, but you dont mention stopping the actual MySQL56 process before performing the upgrade. Was this step intentionally left out?
2) When launching the MySQL GUI installer, you indicated that you cancelled the prompt to do any available in-series updates (i.e.; Oracle's term for minor "dot" updates). However, Oracle recommends this:
"Upgrading one release level is supported. For example, upgrading from 5.6 to 5.7 is supported. Upgrading to the latest release series version is recommended before upgrading to the next release level. For example, upgrade to the latest 5.6 release before upgrading to 5.7."
(This was cited from https://dev.mysql.com/doc/refman/5.7/en/upgrading.html under "Supported Upgrade Paths")
Did you do this? I'm asking because my Windows JSS is running MySQL 5.6.27, but the final series version for 5.6 was 5.6.36. I'm wondering if I really need to do these smaller delta updates before installing 5.7.x...?
Worth mentinging to JSS admins whom are not super familiar with the inner workings of the Windows Server OS: The C:ProgramData directory (where the my.ini file typically lives) is usually hidden in the Explorer GUI by default. Some may confuse ProgramData with Program Files - they are not the same!
Thank you so much for your feedback!
1) There was no need for me to to do this as I uninstalled 5.6 first, so the service was stopped and removed as part of that step.
2) I wasn't aware of that, thank you. I had no issues migrating my database which was previously hosted in MySQL 5.6.34. I'd follow Oracle's advice if I ever did it again.
Dumb question: Since MySQL 5.7 is recomended (not mandatory), there's no reason why I can't complete Jamf 9.99 upgrade (from 9.93), first and then upgrade MySQL from 5.6 to 5.7 later on (i.e.; perform the Jamf/Tomcat updates and then upgrade MySQL in a few weeks if everyhing is running smooth).
I figure if I divide these two (fairy major) upgrade tasks in half, I also reduce the odds of running into problems in half.
I also have some Windows Server updates to apply, as well as Java 8 JDK (& JCE too). I typically dont like making multiple big changes like this in a single time window (unless I have no choice).
Has anyone else staggered their JSS upgrades like this?
Just to add to the conversation I am running 9.101, clustered, on windows server 2008r2. I used @neil.martin83 's post and everything went pretty smoothly (Thank you sir). I did get an error when attempting the restore with the database utility. I ended up unzipping the backup and running a manual restore.
So, does anyone have instructions on how to do the MySQL 5.6 to 5.7 in-place upgrade for Mac? I'm trying to test 5.7 before I upgrade to Jamf 10.
Some of the steps in the Oracle instructions (https://dev.mysql.com/doc/mysql-installation-excerpt/5.7/en/upgrading-strategies.html#upgrade-methods) don't really apply for the Mac version. For example, running mysqladmin -u root -p shutdown doesn't stop the service since it's managed by the preference pane.
mysql -u root -p --execute="SET GLOBAL innodb_fast_shutdown=0"
- Stop MySQL from the preference pane.
- Install 5.7
- copy the jamfsoftware folder from the old mysql data folder to the new one
- make sure the ownership is correct (_mysql:_mysql)
- start MySQL from the prefence pane
- change the root password from the asinine random one created by 5.7 on install
- run mysql_upgrade. A few errors pop up about some jamfsoftware tables not existing (jamfsoftware.jss_cache_configurations, jamfsoftware.jss_memcached_endpoints, jamfsoftware.key_pairs, and jamfsoftware.keystores) which I expected considering what they are.
- Recreate the jamfsoftware account (since it's not there after the 5.7 upgrade) and give it access to the database.
And after a probably unneeded restart of MySQL I get a database error when launching the web page.
I feel like I am being "one of those people" who don't really answer your question with this response, but, you could always set up a new Linux system now that Apple really has no "server" hardware. If you are set on sticking with Apple hardware then the @neil.martin83 method might still be fairly useful. The idea is to back up your JamfPro database, remove MySQL 5.6, Install 5.7, then bring your database back up from the backup. Honestly I tried several times to do an in place upgrade and the only clean path from 5.6-5.7 was following his recommendation.