从 MySQL 5.6.35 迁移到 MariaDB 10.6.3 - 查询不起作用
Posted
技术标签:
【中文标题】从 MySQL 5.6.35 迁移到 MariaDB 10.6.3 - 查询不起作用【英文标题】：Migration from MySQL 5.6.35 to MariaDB 10.6.3 - Queries not working 【发布时间】：2021-11-14 17:17:27 【问题描述】：我最近将数据库服务器从 mysql 5.6.35 迁移到 MariaDB 10.6.3
我发现过去在 MySQL 上运行很快的查询现在非常慢或在 MariaDB 中无法运行。 它给出了：
错误代码：2013。在查询期间丢失与 MySQL 服务器的连接。
我什至增加了 DBMS 连接读取超时间隔 编辑 -> 首选项。查询仍然给出相同的错误。该查询包含大量外部连接，我认为这可能是个问题。
整体DB大小小于5GB，每张表行数小于10k。
我已经为MariaDB 服务器运行了MySQL 调谐器脚本，这些是该工具报告的建议。
**
>
>
> perl mysqltuner.pl
> >> MySQLTuner 1.8.1 - Major Hayden <major@mhtx.net>
> >> Bug reports, feature requests, and downloads at http://mysqltuner.pl/
> >> Run with '--help' for additional options and output filtering
>
> [--] Skipped version check for MySQLTuner script
> [OK] Currently running supported MySQL version 10.6.3-MariaDB
> [OK] Operating on 64-bit architecture
>
> -------- Log file Recommendations ---------------------------------------------- -------------------- [OK] Log file /var/log/mariadb/mariadb.log exists [--] Log file: /var/log/mariadb/mariadb.log(10M) [OK] Log
> file /var/log/mariadb/mariadb.log is not empty [OK] Log file
> /var/log/mariadb/mariadb.log is smaller than 32 Mb [OK] Log file
> /var/log/mariadb/mariadb.log is readable. [!!]
> /var/log/mariadb/mariadb.log contains 90 warning(s). [!!]
> /var/log/mariadb/mariadb.log contains 12 error(s). [--] 0 start(s)
> detected in /var/log/mariadb/mariadb.log [--] 0 shutdown(s) detected
> in /var/log/mariadb/mariadb.log
>
> -------- Storage Engine Statistics --------------------------------------------- -------------------- [--] Status: +Aria +CSV +InnoDB +MEMORY +MRG_MyISAM +MyISAM+PERFORMANCE_SCHEMA+SEQUENCE [--] Data in Aria tables: 32.0K (Tables: 1) [--] Data in MyISAM tables: 287.0K (Tables:
> 8) [--] Data in InnoDB tables: 4.5G (Tables: 900) [OK] Total
> fragmented tables: 0
>
> -------- Analysis Performance Metrics ------------------------------------------ -------------------- [--] innodb_stats_on_metadata: OFF [OK] No stat updates during querying INFORMATION_SCHEMA.
>
> -------- Security Recommendations ---------------------------------------------- --------------------
> [OK] There are no anonymous accounts for any database users [OK] All database users have passwords assigned [!!] User 'user'@%
> does not specify hostname restrictions. [!!] User 'root'@% does not
> specify hostname restrictions. [!!] User 'support'@% does not specify
> hostname restrictions. [!!] There is no basic password file list!
>
> -------- CVE Security Recommendations ------------------------------------------ --------------------
> [--] Skipped due to --cvefile option undefined
>
> -------- Performance Metrics --------------------------------------------------- --------------------
> [--] Up for: 44d 22h 1m 57s (38M q [9.997 qps], 107K conn, TX: 212G, RX: 36G) [--] Reads / Writes: 99% / 1% [--] Binary logging is
> disabled [--] Physical Memory : 7.6G [--] Max MySQL memory :
> 18.9G [--] Other process memory: 0B [--] Total buffers: 417.0M global + 18.9M per thread (1000 max threads) [--] P_S Max memory
> usage: 0B [--] Galera GCache Max memory usage: 0B [OK] Maximum
> reached memory usage: 4.4G (58.64% of installed RAM) [!!] Maximum
> possible memory usage: 18.9G (249.66% of installed RAM) [!!] Overall
> possible memory usage with other process exceeded memory [OK] Slow
> queries: 0% (841/38M) [OK] Highest usage of available connections:
> 21% (218/1000) [OK] Aborted connections: 0.01% (15/107805) [!!]
> name resolution is active : a reverse name resolution is made for each
> new connection and can reduce performance [OK] Query cache is
> disabled by default due to mutex contention on multiprocessor
> machines. [OK] Sorts requiring temporary tables: 0% (33 temp sorts /
> 1M sorts) [!!] Joins performed without indexes: 383010 [OK]
> Temporary tables created on disk: 9% (1M on disk / 19M total) [OK]
> Thread cache hit rate: 99% (935 created / 107K connections) [OK] Table
> cache hit rate: 99% (181M hits / 181M requests) [!!]
> table_definition_cache(400) is lower than number of tables(1224) [OK]
> Open file limit used: 0% (33/32K) [OK] Table locks acquired
> immediately: 100% (25K immediate / 25K locks)
>
> -------- Performance schema ---------------------------------------------------- --------------------
> [--] Performance schema is disabled. [--] Memory used by P_S: 0B [--] Sys schema is installed.
>
> -------- ThreadPool Metrics ---------------------------------------------------- --------------------
> [--] ThreadPool stat is enabled. [--] Thread Pool Size: 4 thread(s). [--] Using default value is good enough for your version
> (10.6.3-MariaDB)
>
> -------- MyISAM Metrics -------------------------------------------------------- --------------------
> [!!] Key buffer used: 18.2% (24M used / 134M cache) [OK] Key buffer size / total MyISAM indexes: 128.0M/58.0K [OK] Read Key buffer
> hit rate: 100.0% (17K cached / 0 reads) [!!] Write Key buffer hit
> rate: 1.0% (8K cached / 92 writes)
>
> -------- InnoDB Metrics -------------------------------------------------------- --------------------
> [--] InnoDB is enabled. [OK] InnoDB File per table is activated [!!] InnoDB buffer pool / data size: 128.0M/4.5G [!!]
> Ratio InnoDB log file size / InnoDB Buffer pool size (75 %): 96.0M *
> 1/128. 0M should be equal to 25% [--] Number of InnoDB Buffer Pool
> Chunk : 1 for 1 Buffer Pool Instance(s) [OK] Innodb_buffer_pool_size
> aligned with Innodb_buffer_pool_chunk_size &
> Innodb_buffer_pool_instances [OK] InnoDB Read buffer efficiency:
> 99.89% (127064097429 hits/ 127199499037 total) [OK] InnoDB Write log efficiency: 98.65% (6576119 hits/ 6666329 total) [OK] InnoDB log
> waits: 0.00% (0 waits / 90210 writes)
>
> -------- Aria Metrics ---------------------------------------------------------- --------------------
> [--] Aria Storage Engine is enabled. [OK] Aria pagecache size / total Aria indexes: 128.0M/352.0K [OK] Aria pagecache hit rate:
> 97.3% (9M cached / 255K reads)
>
> -------- TokuDB Metrics -------------------------------------------------------- --------------------
> [--] TokuDB is disabled.
>
> -------- XtraDB Metrics -------------------------------------------------------- --------------------
> [--] XtraDB is disabled.
>
> -------- Galera Metrics -------------------------------------------------------- --------------------
> [--] Galera is disabled.
>
> -------- Replication Metrics --------------------------------------------------- --------------------
> [--] Galera Synchronous replication: NO [--] No replication slave(s) for this server. [--] Binlog format: MIXED [--] XA support
> enabled: ON [--] Semi synchronous replication Master: OFF [--] Semi
> synchronous replication Slave: OFF [--] This is a standalone server
>
> -------- Recommendations ------------------------------------------------------- --------------------
> General recommendations:
> Check warning line(s) in /var/log/mariadb/mariadb.log file
> Check error line(s) in /var/log/mariadb/mariadb.log file
> Restrict Host for 'user'@'%' to 'user'@LimitedIPRangeOrLocalhost
> RENAME USER 'user'@'%' TO 'user'@LimitedIPRangeOrLocalhost;
> Restrict Host for 'root'@'%' to 'root'@LimitedIPRangeOrLocalhost
> RENAME USER 'root'@'%' TO 'root'@LimitedIPRangeOrLocalhost;
> Restrict Host for 'support'@'%' to 'support'@LimitedIPRangeOrLocalhost
> RENAME USER 'support'@'%' TO 'support'@LimitedIPRangeOrLocalhost;
> Reduce your overall MySQL memory footprint for system stability
> Dedicate this server to your database for highest performance.
> Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
> We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
> See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
> (specially the conclusions at the bottom of the page).
> Performance schema should be activated for better diagnostics
> Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: Variables to adjust: ***
> MySQL's maximum memory usage is dangerously high ***
> *** Add RAM before increasing MySQL buffer variables ***
> join_buffer_size (> 256.0K, or always use indexes with JOINs)
> table_definition_cache(400) > 1224 or -1 (autosizing if supported)
> performance_schema = ON enable PFS
> innodb_buffer_pool_size (>= 4.5G) if possible.
> innodb_log_file_size should be (=32M) if possible, so InnoDB total log files
**
由于 JOIN 缓冲区大小低，是否有可能导致慢查询？ 线程池也会提高数据库的性能吗？ 谁能帮我解决这个问题？
【问题讨论】：
请提供查询EXPLAIN SELECT...（在两台服务器上）和SHOW CREATE TABLE。
请发布 A) SELECT @@wait_timeout 的结果； B）选择@@interactive_timeout；
【参考方案1】：
这更像是一个建议而不是一个答案。
我认为在您的情况下，您必须检查所有查询，因为MySQL 5.6 和MariaDB 之间存在很大差异，例如引入窗口功能：
https://mariadb.com/kb/en/window-functions/
您必须检查并调整查询。
慢查询是否可能是由于 JOIN 缓冲区大小太小？
它用于不能使用索引的查询，而是执行 全表扫描。添加索引时增加以获得更快的完全连接 是不可能的，虽然要注意内存问题，因为连接会 始终分配最小大小。
join_buffer_size: https://mariadb.com/kb/en/server-system-variables/#join_buffer_size
我的建议是：尽可能添加索引。
根据线程池阅读更多内容，因为我不知道您的情况是哪种情况：https://mariadb.com/kb/en/thread-pool-in-mariadb/。
如果您在特定查询方面遇到困难，请提出另一个问题，据我所知，您的问题没有捷径可走。
错误代码：2013。在查询期间丢失与 MySQL 服务器的连接。
为什么会这样？当您之间的连接出现此错误 MySQL 客户端和数据库服务器超时。本质上，它也花了 查询返回数据的时间很长，因此连接被断开。
通过优化查询来避免问题 在许多情况下，您可以 通过优化 SQL 查询来完全避免这个问题。例如， 而不是加入两个非常大的表的所有内容，尝试 过滤掉不需要的记录。在可能的情况下，尝试减少 单个查询中的连接数。这应该有添加 使您的查询更易于阅读的好处。为了我的目的，我已经 发现将内容非规范化到工作表中可以提高 读取性能。这样可以避免超时。
重写查询并不总是选项，因此您可以尝试 遵循服务器端和客户端的解决方法。
服务器端解决方案如果您是 MySQL 服务器的管理员， 尝试更改一些值。 MySQL 文档建议增加 服务器上的 net_read_timeout 或 connect_timeout 值。
https://dev.mysql.com/doc/refman/8.0/en/error-lost-connection.html
【讨论】：
感谢您的回答。 join 子句中的所有列都被索引。我又检查了一遍。 @SimsSusee 我只是编辑答案，包括Lost connection to MySQL server during query. 的解决方案。希望对你有帮助【参考方案2】：
Sim Susee，欢迎来到 SO。为您的 my.cnf [mysqld] 部分考虑的建议
innodb_buffer_pool_size=4G # from 128M to accomodate more of your 5+G of data in RAM.
max_connections=500 # from 1000 because you have only used 218 concurrent connections in 44 days.
innodb_buffer_pool_instances=4 # from default of 1 to reduce mutex contention
innodb_lru_scan_depth=100 # from 1024 to conserve 90% of CPU cycles used for function, every SECOND.
在 44 天内，您有 383,000 多个未使用索引的连接。需要适当的索引，可能需要多列索引。请查看个人资料以获得更多帮助。
【讨论】：
感谢威尔逊的回答。我会检查索引。 线程池怎么样。需要吗？ 通常线程池不是必需的，但在适当调整后可以提高性能。有许多变量需要考虑。 在某些查询中，列缺少索引。添加了 FK 引用并通过查询获得了出色的性能。感谢您的回答。 @SimsSusee 恭喜您取得进步。周末愉快。以上是关于从 MySQL 5.6.35 迁移到 MariaDB 10.6.3 - 查询不起作用的主要内容，如果未能解决你的问题，请参考以下文章