性能问题 MySQL 5.7

Posted 2023-02-16

【中文标题】性能问题 MySQL 5.7

【英文标题】：performance issue MySQL 5.7

【发布时间】：2016-02-21 07:18:34

【问题描述】：

我已将我的 mysql 数据库从 5.6 迁移到 5.7 版。现在我遇到了性能问题。

在两个版本中都执行了特定的语句，我注意到 5.7 的速度要慢得多。特别是有大量数据。

即使是简单的选择语句也慢得多

5.6: 2,948 sec
5.7: 6,536 sec

在我的应用程序中，我有更复杂的语句，需要更多时间来执行，因此无法使用。

我还注意到我的表的字段数在 5.7 中很重要，但在 5.6 中不重要。

my.ini 中的配置值没有改变。

要测试此问题，您可以使用以下代码：

#Create Tables
DROP TABLE IF EXISTS aTest;
DROP TABLE IF EXISTS sTest;
CREATE TABLE `aTest` (
    `T_BETREFF` CHAR(40) NOT NULL DEFAULT '',
    `T_DATUM` DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00',
    `T_ZEIT` CHAR(5) NOT NULL DEFAULT '',
    `T_SACHBEARB` CHAR(2) NOT NULL DEFAULT '',
    `SYS_UID` INT NOT NULL AUTO_INCREMENT,
    `SYS_DATE` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    UNIQUE INDEX `TODO01` (`SYS_UID`),
    INDEX `TODO02` (`T_SACHBEARB`)
)
COLLATE='latin1_swedish_ci'
ENGINE=MyISAM;

CREATE TABLE `sTest` (
    `V_NR` CHAR(2) NOT NULL DEFAULT '',
    `V_NAME` CHAR(30) NOT NULL DEFAULT '',
    `SYS_UID` INT NOT NULL AUTO_INCREMENT,
    `SYS_DATE` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    UNIQUE INDEX `S_VERK01` (`SYS_UID`),
    UNIQUE INDEX `S_VERK02` (`V_NR`)
)
COLLATE='latin1_swedish_ci'
ENGINE=MyISAM;

#Fill Tables
DROP PROCEDURE IF EXISTS load_foo_atest_data;
DELIMITER //
CREATE PROCEDURE load_foo_atest_data()
BEGIN
    DECLARE v_max int unsigned default 1000000;
    DECLARE v_counter INT UNSIGNED DEFAULT 0;

    WHILE v_counter < v_max DO
        INSERT INTO aTest (T_BETREFF, T_DATUM, T_ZEIT, T_SACHBEARB) VALUES('TEST 1234,', NOW(), '00:00', v_counter DIV 100000 * 10);
        IF (v_counter MOD 100000 = 0) THEN
            INSERT INTO sTest (V_NR, V_NAME) VALUES(v_counter DIV 100000 * 10, CONCAT('TEST', v_counter));
        END IF;
        SET v_counter=v_counter+1;
    END WHILE;
END //
DELIMITER ;

call load_foo_atest_data();

这是我的声明：

FLUSH TABLES;
SELECT * FROM atest
LEFT JOIN sTEST ON v_nr = t_sachbearb
GROUP BY atest.sys_uid
ORDER BY t_datum DESC, t_zeit DESC

编辑

解释结果：

5.6：

+----+-------------+--------+--------+------------- --+------------+----------+-------------- -----------------+---------+---------- ------------+ |编号 |选择类型 |表|类型 |可能的键 |关键 | key_len |参考 |行 |额外 | +----+-------------+--------+--------+------------- --+------------+----------+-------------- -----------------+---------+---------- ------------+ | 1 |简单 |证明 |全部 |待办事项01 |空 |空 |空 | 1000000 |使用临时的；使用文件排序 | | 1 |简单 |测试 | eq_ref | S_VERK02 | S_VERK02 | 2 | obs_mysql_bergau_57test.test.T_SACHBEARB | 1 |空 | +----+-------------+--------+--------+------------- --+------------+----------+-------------- -----------------+---------+---------- ------------+

5.7

+----+-------------+--------+------------+--------+ ---------------+----------+---------+---------- ------------------+---------+---------- -+----------------------------------+ |编号 |选择类型 |表|隔断 |类型 |可能的键 |关键 | key_len |参考 |行 |过滤 |额外 | +----+-------------+--------+------------+--------+ ---------------+----------+---------+---------- ------------------+---------+---------- -+----------------------------------+ | 1 |简单 |证明 |空 |全部 |待办事项01 |空 |空 |空 | 1000000 | 100.00 |使用临时的；使用文件排序 | | 1 |简单 |测试 |空 | eq_ref | S_VERK02 | S_VERK02 | 2 | obs_mysql_bergau_57test.test.T_SACHBEARB | 1 | 100.00 |空 | +----+-------------+--------+------------+--------+ ---------------+----------+---------+---------- ------------------+---------+---------- -+----------------------------------+

我的.ini：

# Other default tuning values
# MySQL Server Instance Configuration File
# ----------------------------------------------------------------------
# Generated by the MySQL Server Instance Configuration Wizard
#
#
# Installation Instructions
# ----------------------------------------------------------------------
#
# On Linux you can copy this file to /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options
# (@localstatedir@ for this installation) or to
# ~/.my.cnf to set user-specific options.
#
# On Windows you should keep this file in the installation directory 
# of your server (e.g. C:\Program Files\MySQL\MySQL Server X.Y). To
# make sure the server reads the config file use the startup option 
# "--defaults-file". 
#
# To run run the server from the command line, execute this in a 
# command line shell, e.g.
# mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# To install the server as a Windows service manually, execute this in a 
# command line shell, e.g.
# mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# And then execute this in a command line shell to start the server, e.g.
# net start MySQLXY
#
#
# Guildlines for editing this file
# ----------------------------------------------------------------------
#
# In this file, you can use all long options that the program supports.
# If you want to know the options a program supports, start the program
# with the "--help" option.
#
# More detailed information about the individual options can also be
# found in the manual.
#
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
#
#
# CLIENT SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by MySQL client applications.
# Note that only client applications shipped by MySQL are guaranteed
# to read this section. If you want your own MySQL client program to
# honor these values, you need to specify it as an option during the
# MySQL client library initialization.
# [client] no-beep

# pipe
# socket=0.0 port=3306

[mysql]

default-character-set=utf8


# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this 
# file.
#
# server_type=1 [mysqld]

# The next three options are mutually exclusive to SERVER_PORT below.
# skip-networking

# enable-named-pipe

# shared-memory

# shared-memory-base-name=MYSQL

# The Pipe the MySQL Server will use
# socket=MYSQL

# The TCP/IP Port the MySQL Server will listen on port=3306

# Path to installation directory. All paths are usually resolved relative to this.
# basedir="C:/Program Files/MySQL/MySQL Server 5.7/"

# Path to the database root datadir=C:/ProgramData/MySQL/MySQL Server 5.7\Data

# The default character set that will be used when a new schema or table is
# created and no character set is defined character-set-server = latin1

# The default storage engine that will be used when create new tables when default-storage-engine = MYISAM

# Set the SQL mode to strict sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

# Enable Windows Authentication
# plugin-load=authentication_windows.dll

# General and Slow logging. log-output=FILE general-log=1 general_log_file="PC-THIEL.log" slow-query-log=1 slow_query_log_file="PC-THIEL-slow.log" long_query_time=10

# Binary Logging.
# log-bin

# Error Logging. log-error="PC-THIEL.err"

# Server Id. server-id=1

# Secure File Priv. secure-file-priv="C:/ProgramData/MySQL/MySQL Server 5.7/Uploads"

# The maximum amount of concurrent sessions the MySQL server will
# allow. One of these connections will be reserved for a user with
# SUPER privileges to allow the administrator to login even if the
# connection limit has been reached. max_connections=1000

# Query cache is used to cache SELECT results and later return them
# without actual executing the same query once again. Having the query
# cache enabled may result in significant speed improvements, if your
# have a lot of identical queries and rarely changing tables. See the
# "Qcache_lowmem_prunes" status variable to check if the current value
# is high enough for your load.
# Note: In case your tables change very often or if your queries are
# textually different every time, the query cache may result in a
# slowdown instead of a performance improvement. query_cache_size = 2G

# The number of open tables for all threads. Increasing this value
# increases the number of file descriptors that mysqld requires.
# Therefore you have to make sure to set the amount of open files
# allowed to at least 4096 in the variable "open-files-limit" in
# section [mysqld_safe] table_open_cache=2000

# Maximum size for internal (in-memory) temporary tables. If a table
# grows larger than this value, it is automatically converted to disk
# based table This limitation is for a single table. There can be many
# of them. tmp_table_size=3G

# How many threads we should keep in a cache for reuse. When a client
# disconnects, the client's threads are put in the cache if there aren't
# more than thread_cache_size threads from before.  This greatly reduces
# the amount of thread creations needed if you have a lot of new
# connections. (Normally this doesn't give a notable performance
# improvement if you have a good thread implementation.) thread_cache_size=10

#*** MyISAM Specific options
# The maximum size of the temporary file MySQL is allowed to use while
# recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE.
# If the file-size would be bigger than this, the index will be created
# through the key cache (which is slower). myisam_max_sort_file_size=100G

# If the temporary file used for fast index creation would be bigger
# than using the key cache by the amount specified here, then prefer the
# key cache method.  This is mainly used to force long character keys in
# large tables to use the slower key cache method to create the index. myisam_sort_buffer_size=6G

# Size of the Key Buffer, used to cache index blocks for MyISAM tables.
# Do not set it larger than 30% of your available memory, as some memory
# is also required by the OS to cache rows. Even if you're not using
# MyISAM tables, you should still set it to 8-64M as it will also be
# used for internal temporary disk tables. key_buffer_size = 8G

# Size of the buffer used for doing full table scans of MyISAM tables.
# Allocated per thread, if a full scan is needed. read_buffer_size=64K read_rnd_buffer_size=256K

#*** INNODB Specific options ***
# innodb_data_home_dir=0.0

# Use this option if you have a MySQL server with InnoDB support enabled
# but you do not plan to use it. This will save memory and disk space
# and speed up some things.
# skip-innodb

# If set to 1, InnoDB will flush (fsync) the transaction logs to the
# disk at each commit, which offers full ACID behavior. If you are
# willing to compromise this safety, and you are running small
# transactions, you may set this to 0 or 2 to reduce disk I/O to the
# logs. Value 0 means that the log is only written to the log file and
# the log file flushed to disk approximately once per second. Value 2
# means the log is written to the log file at each commit, but the log
# file is only flushed to disk approximately once per second. innodb_flush_log_at_trx_commit=1

# The size of the buffer InnoDB uses for buffering log data. As soon as
# it is full, InnoDB will have to flush it to disk. As it is flushed
# once per second anyway, it does not make sense to have it very large
# (even with long transactions). innodb_log_buffer_size=1M

# InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and
# row data. The bigger you set this the less disk I/O is needed to
# access data in tables. On a dedicated database server you may set this
# parameter up to 80% of the machine physical memory size. Do not set it
# too large, though, because competition of the physical memory may
# cause paging in the operating system.  Note that on 32bit systems you
# might be limited to 2-3.5G of user level memory per process, so do not
# set it too high. innodb_buffer_pool_size=8M

# Size of each log file in a log group. You should set the combined size
# of log files to about 25%-100% of your buffer pool size to avoid
# unneeded buffer pool flush activity on log file overwrite. However,
# note that a larger logfile size will increase the time needed for the
# recovery process. innodb_log_file_size=48M

# Number of threads allowed inside the InnoDB kernel. The optimal value
# depends highly on the application, hardware as well as the OS
# scheduler properties. A too high value may lead to thread thrashing. innodb_thread_concurrency=17

# The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full. innodb_autoextend_increment=64

# The number of regions that the InnoDB buffer pool is divided into.
# For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency,
# by reducing contention as different threads read and write to cached pages. innodb_buffer_pool_instances=8

# Determines the number of threads that can enter InnoDB concurrently. innodb_concurrency_tickets=5000

# Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before
# it can be moved to the new sublist. innodb_old_blocks_time=1000

# It specifies the maximum number of .ibd files that MySQL can keep open at one time. The minimum value is 10. innodb_open_files=300

# When this variable is enabled, InnoDB updates statistics during metadata statements. innodb_stats_on_metadata=0

# When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table
# in a separate .ibd file, rather than in the system tablespace. innodb_file_per_table=1

# Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none. innodb_checksum_algorithm=0

# The number of outstanding connection requests MySQL can have.
# This option is useful when the main MySQL thread gets many connection requests in a very short time.
# It then takes some time (although very little) for the main thread to check the connection and start a new thread.
# The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily
# stops answering new requests.
# You need to increase this only if you expect a large number of connections in a short period of time. back_log=80

# If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and
# synchronize unflushed data to disk.
# This option is best used only on systems with minimal resources. flush_time=0

# The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use
# indexes and thus perform full table scans. join_buffer_size=256K

# The maximum size of one packet or any generated or intermediate string, or any parameter sent by the
# mysql_stmt_send_long_data() C API function. max_allowed_packet=4M

# If more than this many successive connection requests from a host are interrupted without a successful connection,
# the server blocks that host from performing further connections. max_connect_errors=1000

# Changes the number of file descriptors available to mysqld.
# You should try increasing the value of this option if mysqld gives you the error "Too many open files". open_files_limit=4161

# Set the query cache type. 0 for OFF, 1 for ON and 2 for DEMAND. query_cache_type = 1

# If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the
# sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization
# or improved indexing. sort_buffer_size=256K

# The number of table definitions (from .frm files) that can be stored in the definition cache.
# If you use a large number of tables, you can create a large table definition cache to speed up opening of tables.
# The table definition cache takes less space and does not use file descriptors, unlike the normal table cache.
# The minimum and default values are both 400. table_definition_cache=1400

# Specify the maximum size of a row-based binary log event, in bytes.
# Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256. binlog_row_event_max_size=8K

# If the value of this variable is greater than 0, a replication slave synchronizes its master.info file to disk.
# (using fdatasync()) after every sync_master_info events. sync_master_info=10000

# If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk.
# (using fdatasync()) after every sync_relay_log writes to the relay log. sync_relay_log=10000

# If the value of this variable is greater than 0, a replication slave synchronizes its relay-log.info file to disk.
# (using fdatasync()) after every sync_relay_log_info transactions. sync_relay_log_info=10000

query_cache_limit = 30M collation-server = latin1_swedish_ci default_tmp_storage_engine = MYISAM


myisam_use_mmap concurrent_insert = AUTO

【问题讨论】：

SELECT 是什么样子的？它是在生成的测试数据上运行的吗？硬件是一样的吗？

好问题 - 很多有用的信息。你能把这两个版本的解释都贴出来吗？

@Mihai SELECT 语句在我的问题中发布并在生成的测试数据上运行。迁移的数据库在同一台机器上。为了测试我的问题，我在测试机器上并行运行了两个版本。

@Neville K 我已经编辑了我的问题

将CHAR 更改为VARCHAR，除非列的宽度确实是固定的。切换到 InnoDB。如果 5.7 仍然较慢，请在 bugs.mysql.com 提交错误报告。

【参考方案1】：

如果没有 SQLFiddle，我无法确定，但我认为您的问题是由 change to the performance tuning of "group by" in 5.7 引起的。

我不完全确定，但我认为这意味着过去有意义的查询优化器决策不再有效。例如，在您的示例查询中，EXPLAIN 显示查询正在使用较小表上的索引。

在您发布的代码中，我认为“group by”子句的作用不大——您仍在检索每一行，因为 sysdate 应该是唯一的。

在这种特定情况下，您可以通过在表 atest 上创建索引，其中包含 t_sachbearb、atest.sys_uid、t_datum DESC、t_zeit DESC 列，可以获得更好的结果。这应该会导致查询优化器选择较大列上的索引，并将其用于过滤和排序。

我会考虑在没有 group by 的情况下重写，以尽可能支持“不同”的要求。

如果没有，您可能需要查看单独的“EXPLAIN”输出并优化索引。

【讨论】：

这只是一个例子来说明我遇到的问题。在我的应用程序中，我使用更复杂的语句，需要使用“分组依据”。

我有许多没有 GROUP BY 的慢速 5.7-only-sql。请看一下：***.com/questions/38499811/…