索引实践和调优

Posted 2020-11-19 ---wunian

Ⅰ、如何使用B+ tree索引

([email protected]) [test]> desc l;
+-------+---------+------+-----+---------+-------+
| Field | Type    | Null | Key | Default | Extra |
+-------+---------+------+-----+---------+-------+
| a     | int(11) | NO   | PRI | NULL    |       |
| b     | int(11) | YES  | MUL | NULL    |       |
| c     | int(11) | YES  | UNI | NULL    |       |
| d     | int(11) | YES  |     | NULL    |       |
+-------+---------+------+-----+---------+-------+
4 rows in set (0.00 sec)

([email protected]) [test]> explain select b from l where c = 10;
+----+-------------+-------+------------+-------+---------------+------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type  | possible_keys | key  | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | l     | NULL       | const | c             | c    | 5       | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+-------+---------------+------+---------+-------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

([email protected]) [test]> explain select b from l where d = 10;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | l     | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    4 |    25.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.01 sec)

看key值,表示这条sql语句的执行计划使用了哪一个索引,没走索引,key值就是NULL,这时候就会扫描全部数据

线上删除索引不需要在线工具,只是将索引所占的空间释放掉,很快,不需要pt-osc

alter table orders drop index xxx

Ⅱ、线上调优

大部分都是看慢查询日志,找到慢sql,复制出来去命令行里explain一把,看下具体情况,缺少索引就加一下,一百五十万数据的一张表加一个索引差不多要4s

线上的slow-log好几个g,怎么看？

mysqldumpslow slow.log |less,这样会把对一些表的操作格式化
-s [option]
at(average query time) 执行时间     默认
-r 逆序
-n 10 只显示多少
-t 3 看执行时间最长的3条

这个工具会解析所有slow.log 量太大，解析依然很慢，这时候就需要用到采样

tail -n 10000 slow.log > analytics.log
mysqldumpslow analytics.log

tips：如何在线清理慢日志？

直接 > slow.log 这样是不行的,因为mysql对应这个文件的句柄依然打开,磁盘空间释放不出来的,

正确做法,先备份,mv slow.log slow.lg.170302,虽然改名,但句柄未变,此时慢日志还是会往里面写,

在数据库中flush slow logs 一把,此时才会关闭之前的慢查询日志的句柄,重新打开一个新慢日志的句柄

Ⅲ、5.7中分析慢sql利器

sys库中一个表
statement_analysis表,这个看起来比slow.log看起来更直观,这个表非常重要,这个表不会很大,有参数来控制它最大多少行,后面再讲
x$statement_analysis 这样查,就不会把表里的时间什么的格式化,全是数字,如果想对这张表进行每秒钟采集,将这些数值做差值,可以得到某个波段的增长量

([email protected]) [sys]> show create table statement_analysisG
*************************** 1. row ***************************
                View: statement_analysis
         Create View: CREATE ALGORITHM=MERGE DEFINER=`mysql.sys`@`localhost` SQL SECURITY INVOKER VIEW `statement_analysis` AS select `sys`.`format_statement`(`performance_schema`.`events_statements_summary_by_digest`.`DIGEST_TEXT`) AS `query`,`performance_schema`.`events_statements_summary_by_digest`.`SCHEMA_NAME` AS `db`,if(((`performance_schema`.`events_statements_summary_by_digest`.`SUM_NO_GOOD_INDEX_USED` > 0) or (`performance_schema`.`events_statements_summary_by_digest`.`SUM_NO_INDEX_USED` > 0)),‘*‘,‘‘) AS `full_scan`,`performance_schema`.`events_statements_summary_by_digest`.`COUNT_STAR` AS `exec_count`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_ERRORS` AS `err_count`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_WARNINGS` AS `warn_count`,`sys`.`format_time`(`performance_schema`.`events_statements_summary_by_digest`.`SUM_TIMER_WAIT`) AS `total_latency`,`sys`.`format_time`(`performance_schema`.`events_statements_summary_by_digest`.`MAX_TIMER_WAIT`) AS `max_latency`,`sys`.`format_time`(`performance_schema`.`events_statements_summary_by_digest`.`AVG_TIMER_WAIT`) AS `avg_latency`,`sys`.`format_time`(`performance_schema`.`events_statements_summary_by_digest`.`SUM_LOCK_TIME`) AS `lock_latency`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_ROWS_SENT` AS `rows_sent`,round(ifnull((`performance_schema`.`events_statements_summary_by_digest`.`SUM_ROWS_SENT` / nullif(`performance_schema`.`events_statements_summary_by_digest`.`COUNT_STAR`,0)),0),0) AS `rows_sent_avg`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_ROWS_EXAMINED` AS `rows_examined`,round(ifnull((`performance_schema`.`events_statements_summary_by_digest`.`SUM_ROWS_EXAMINED` / nullif(`performance_schema`.`events_statements_summary_by_digest`.`COUNT_STAR`,0)),0),0) AS `rows_examined_avg`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_ROWS_AFFECTED` AS `rows_affected`,round(ifnull((`performance_schema`.`events_statements_summary_by_digest`.`SUM_ROWS_AFFECTED` / nullif(`performance_schema`.`events_statements_summary_by_digest`.`COUNT_STAR`,0)),0),0) AS `rows_affected_avg`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_CREATED_TMP_TABLES` AS `tmp_tables`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_CREATED_TMP_DISK_TABLES` AS `tmp_disk_tables`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_SORT_ROWS` AS `rows_sorted`,`performance_schema`.`events_statements_summary_by_digest`.`SUM_SORT_MERGE_PASSES` AS `sort_merge_passes`,`performance_schema`.`events_statements_summary_by_digest`.`DIGEST` AS `digest`,`performance_schema`.`events_statements_summary_by_digest`.`FIRST_SEEN` AS `first_seen`,`performance_schema`.`events_statements_summary_by_digest`.`LAST_SEEN` AS `last_seen` from `performance_schema`.`events_statements_summary_by_digest` order by `performance_schema`.`events_statements_summary_by_digest`.`SUM_TIMER_WAIT` desc
character_set_client: utf8
collation_connection: utf8_general_ci
1 row in set (0.00 sec)

会发现是一张视图,数据是从performance_schema库中的events_statements_summary_by_digest中抽取的,并且这张表本身就根据总的等待时间排序了,这东西方便后续做awr之类的工具

tips：
sys库中,所有的表都是视图,用于方便统计,之前需要去performance_schema中看events_statements_summary_by_digest

statements_with_errors_or_warnings        执行后有错或者报警的
statements_with_full_table_scans          没有走索引也就是全表扫描
statements_with_sorting                   带有排序的
statements_with_temp_tables               带有临时表的

找线上哪些sql平均慢了看sys库,哪个时间点慢了看slow.log

5.6怎么办,没sys库
自己创建sys库

cd /tmp
git clone https://github.com/mysql/mysql-sys
cd mysql-sys/
mysql -u root -p < ./sys_56.sql

貌似表比5.7要少一点

tips：
这些视图可以认为存内存的,不占特别大开销,5.6开始,其实是需要打开performance_schema参数的,不过是默认打开的
performance_schema库太专业,很多东西和内核有关,普通用户不建议看,能看sys库已经不错了

Ⅳ、补充

sys库中还有个表schema_index_statistics可以查看每个索引使用情况,增删查改所有的次数和时间都可以看到,能知道哪张表的哪个索引比较活跃

statement_analysis、schema_index_statistics、慢查询 三个结合起来可以进行一个初步调优了