The slow query log is the trusted old method of recording slow query, so the database administrator can determine which queries are in the most need for optimization. Since MySQL 5.6, it has to some extend been overshadowed by the Performance Schema which has lower overhead and thus allows collecting statistics about all queries. The slow query log has one major advantage though: the data is persisted. In MySQL 8.0.14 which was recently released, there is an improvement for the slow query log: additional statistics about the recorded queries.
The slow query log with log_slow_extra enabled.
Contribution
Thanks for Facebook for contributing a patch for the new feature.
The additional information is not recorded by default. To enable the feature, enable the log_slow_extra option:
Here, SET PERSIST is used, so the configuration change is persisted when MySQL is restarted. If you just want to try the feature, you can use SET GLOBAL, then decide later whether you want to keep it enabled.
That is all that is required. You can now execute a “slow” query and take a look at the recorded data. An easy way to execute a slow query is to execute DO SLEEP(...) where you can replace ... with the number of seconds you want to sleep. However that is not a very interesting query. Another option is to lower long_query_time to ensure your query is captured. If you set long_query_time to zero, all queries are recorded:
Here SET SESSION is used so only the current connection is affected.
Be Aware
The slow query log does have overhead. It is for most systems not recommended to set long_query_time to zero for all connections all the time. Doing so can cause performance problems.
Finally, execute a query:
mysql> SELECT * FROM world.city ORDER BY Population DESC LIMIT 10;
+------+------------------+-------------+------------------+------------+
| ID | Name | CountryCode | District | Population |
+------+------------------+-------------+------------------+------------+
| 1024 | Mumbai (Bombay) | IND | Maharashtra | 10500000 |
| 2331 | Seoul | KOR | Seoul | 9981619 |
| 206 | São Paulo | BRA | São Paulo | 9968485 |
| 1890 | Shanghai | CHN | Shanghai | 9696300 |
| 939 | Jakarta | IDN | Jakarta Raya | 9604900 |
| 2822 | Karachi | PAK | Sindh | 9269265 |
| 3357 | Istanbul | TUR | Istanbul | 8787958 |
| 2515 | Ciudad de México | MEX | Distrito Federal | 8591309 |
| 3580 | Moscow | RUS | Moscow (City) | 8389200 |
| 3793 | New York | USA | New York | 8008278 |
+------+------------------+-------------+------------------+------------+
10 rows in set (0.04 sec)
MySQL 8.0 introduced a new feature that allows you to persist configuration changes from inside MySQL. Previously you could execute SET GLOBAL to change the configuration at runtime, but you needed to update your MySQL configuration file in order to persist the change. In MySQL 8.0 you can skip the second step. This blog discuss how this works and how to backup and restore the configuration.
Using SET PERSIST to set a variable and the persisted_variables table in the Performance Schema to get a list of persisted variables.
Persisting Variables
You persist changes with either the SET PERSIST or SET PERSIST_ONLY statement. The different is that SET PERSIST_ONLY only updates the configuration whereas SET PERSIST essentially combines SET GLOBAL and SET PERSIST_ONLY.
Information
Some variables such as innodb_buffer_pool_instances can only use PERSIST_ONLY, i.e. it requires a restart to make the changes take effect. Still others, such as datadir, requires cryptographically signed SET statements which are available in MySQL 8.0.14 and later.
mysqld-auto.cnf and variables_info
The persisted variables are stored in the file mysqld-auto.cnf located in the data directory using the JSON format. It includes more information than just the persisted value. It also includes information such as who made the change and when. An example file is:
Notice that the source for join_buffer_size is DYNAMIC whereas the two other variables have the source set to PERSISTED. Why? After all they all three existed in the mysqld-auto.cnf file. DYNAMIC means that the variable was changes since the last restart either using SET GLOBAL or SET PERSIST. Another thing to be aware of is that variables changed with SET PERSIST_ONLY will not show up in variables_info until after the next restart. I will soon get back to show a way to get the variables that have been persisted in one way or another.
Backup and Restore
As a simple way to back up the configuration is simply copy the mysqld-auto.cnf file to a safe location. Similarly, you can restore the configuration by copying it back.
However, what if you want most of the configuration but not everything or you want to edit some of the values? In that case you need another way of exporting the configuration as you should not manually edit mysqld-auto.cnf.
Warning
Do not edit the mysqld-auto.cnf file manually. It should only be changed with SET PERSIST and SET PERSIST_ONLY. If there are any errors in the file, MySQL will refuse to start.
Fortunately as it turns out, it is easy to export all persisted variables. The table performance_schema.persisted_variables includes all variables that has either been read from mysqld-auto.cnf or has been changed with SET PERSIST or SET PERSIST_ONLY since the last restart. The table include the persisted values. For example:
On Microsoft Windows ensure everything is on one line and the backslashes are removed.
Now the file config.sql contains an export of the persisted variables:
shell$ cat config.sql
SET PERSIST_ONLY sort_buffer_size = 32768;
SET PERSIST_ONLY join_buffer_size = 131072;
SET PERSIST_ONLY slave_parallel_type = 'LOGICAL_CLOCK';
This example creates SET PERSIST_ONLY statement as those will work with all persistable variables. When you replay the SET statements, it will require a restart of MySQL for the changes to take effect. If you want to use SET PERSIST where possible, then you need to take into consideration whether the variable support SET PERSIST. A list of variables that require SET PERSIST_ONLY are included at the end.
As promised, I will conclude with a list of persistable variables that only supports SET PERSIST_ONLY. As of MySQL 8.0.12 without any plugins installed, the variables are:
With MySQL 5.6 just gone GA, I thought it would be good to take a look at the changes in options and variables that comes with the new release.
First of all, several of the existing options have get new default values. As James Day already have written a good post about that in his blog, I will refer to that instead of going through the changes. For a general overview of the new features and improvements, the recent blogs by Rob Young and Peter Saitsev are good starting points together with the What is New in MySQL 5.6 page in the Reference Manual are good places to start.
Instead I will focus a little on the new options that has been introduced. The first thing to note is that a in the current 5.5. release (5.5.30) there are 323 variables whereas 5.6 GA (5.6.10) returns 440 rows.
MySQL 5.5.29> SELECT COUNT(*) FROM information_schema.GLOBAL_VARIABLES;
+----------+
| COUNT(*) |
+----------+
| 323 |
+----------+
1 row in set (0.04 sec)
MySQL 5.6.10> SELECT COUNT(*) FROM information_schema.GLOBAL_VARIABLES;
+----------+
| COUNT(*) |
+----------+
| 440 |
+----------+
1 row in set (0.02 sec)
Note
This post is written using the Enterprise versions with the semi-synchronous replication plugins enabled in both versions plus the memcached and password validation plugins in 5.6.
Actually the number of new variables is not 117 but 129 as 12 variables have been removed in 5.6.
So what are all of these 129 new variables good for? Actually there is a good chance that you will never need to touch many of them as the default value is good enough, they simply have been added to provide the value of options already present in 5.5 but not exposed through SHOW GLOBAL VARIABLES, or that they are for features you are not using. If we try to group the new variables the distribution comes out as:
Feature
New Variables
Global Transaction IDs
5
Other Replication
19
Memcached Plugin
6
Validate Password Plugin
6
Other Security Related
5
InnoDB
54
Optimizer Traces
5
Performance Schema
15
Exposing Previously Existing Variables
2
Other
12
New Variables in MySQL 5.6
The 54 new InnoDB variables span a number of different changes and additions such as:
New adaptive flushing algorithm
Buffer Pool dumps to disk and restore
Support for additional checksum algorithms
Improvements for compression
Full text indexes
New monitoring options (the information_schema.metrics table)
Configurable page size
Persistent statistics
Undo logs improvements
And more …
For reference I have added a list of the new variables with the release they were introduced and the default value (additionally innodb_print_all_deadlocks is also new, but that was also added to 5.5.30):
