优化一个巨大的 MySQL 视图

Posted 2023-02-26

技术标签:

【中文标题】优化一个巨大的 MySQL 视图【英文标题】：Optimizing a huge MySQL view 【发布时间】：2015-08-22 20:05:31 【问题描述】：

我为朋友开发了一个用于记录游戏统计数据的小应用程序。一开始还不错，但现在我的 snapshot_details 表包含近 55,000 行，查看 wide_snapshots 视图的前 25 行需要 28 秒。

有人可以帮我优化这个怪物吗？

查看wide_snapshots：

select `s`.`id_snapshot` AS `s_id_snapshot`,
       `s`.`ref_player` AS `s_ref_player`,
       `s`.`ref_world` AS `s_ref_world`,
       `s`.`ref_alliance` AS `s_ref_alliance`,
       `s`.`alliance_role` AS `s_alliance_role`,
       `s`.`classe` AS `s_classe`,
       unix_timestamp(`s`.`timestamp`) AS `s_timestamp`,
       `s`.`rank` AS `s_rank`,
       `s`.`is_substitution` AS `s_is_substitution`,
       `s`.`nb_off_bases` AS `s_nb_off_bases`,
       `s`.`moy_defense` AS `s_moy_defense`,
       `s`.`nb_sup_bases` AS `s_nb_sup_bases`,
       `s`.`moy_sup_lvl` AS `s_moy_sup_lvl`,
       `s`.`moy_def_facility` AS `s_moy_def_facility`,
       `s`.`max_cp` AS `s_max_cp`,
       `s`.`research_points` AS `s_research_points`,
       `s`.`supply_points` AS `s_supply_points`,
       `s`.`command_points` AS `s_command_points`,
       `s`.`credits` AS `s_credits`,
       `s`.`max_repa` AS `s_max_repa`,
       `s`.`nb_bases` AS `s_nb_bases`,
       `s`.`prod_credit` AS `s_prod_credit`,
       `s`.`prod_tiberium` AS `s_prod_tiberium`,
       `s`.`prod_crystal` AS `s_prod_crystal`,
       `s`.`prod_energy` AS `s_prod_energy`,
       `s`.`base1_lvl` AS `s_base1_lvl`,
       `s`.`base1_def` AS `s_base1_def`,
       `s`.`base1_off` AS `s_base1_off`,
       `s`.`base1_prod_energy` AS `s_base1_prod_energy`,
       `s`.`base1_avia_repa` AS `s_base1_avia_repa`,
       `s`.`base1_vehi_repa` AS `s_base1_vehi_repa`,
       `s`.`base1_inf_repa` AS `s_base1_inf_repa`,
       `s`.`base2_lvl` AS `s_base2_lvl`,
       `s`.`base2_def` AS `s_base2_def`,
       `s`.`base2_off` AS `s_base2_off`,
       `s`.`base2_prod_energy` AS `s_base2_prod_energy`,
       `s`.`base2_avia_repa` AS `s_base2_avia_repa`,
       `s`.`base2_vehi_repa` AS `s_base2_vehi_repa`,
       `s`.`base2_inf_repa` AS `s_base2_inf_repa`,
       `sd`.`id_detail` AS `sd_id_detail`,
       `sd`.`ref_snapshot` AS `sd_ref_snapshot`,
       `sd`.`ingame_id` AS `sd_ingame_id`,
       `sd`.`base_id` AS `sd_base_id`,
       `sd`.`base_name` AS `sd_base_name`,
       `sd`.`base_lvl` AS `sd_base_lvl`,
       `sd`.`base_off` AS `sd_base_off`,
       `sd`.`base_def` AS `sd_base_def`,
       `sd`.`construction_yard` AS `sd_construction_yard`,
       `sd`.`defense_facility` AS `sd_defense_facility`,
       `sd`.`defense_hq` AS `sd_defense_hq`,
       `sd`.`command_center` AS `sd_command_center`,
       `sd`.`support_lvl` AS `sd_support_lvl`,
       `sd`.`support_type` AS `sd_support_type`,
       `sd`.`prod_credit` AS `sd_prod_credit`,
       `sd`.`prod_energy` AS `sd_prod_energy`,
       `sd`.`prod_tiberium` AS `sd_prod_tiberium`,
       `sd`.`prod_crystal` AS `sd_prod_crystal`,
       `p`.`ingame_id` AS `p_ingame_id`,
       `p`.`name` AS `p_name`,
       `a`.`ingame_id` AS `a_ingame_id`,
       `a`.`name` AS `a_name`
from (
    (
        (
            `kakawi_basetracker`.`snapshots` `s`
            join `kakawi_basetracker`.`snapshot_details` `sd`
            on `s`.`id_snapshot` = `sd`.`ref_snapshot`
        )
        join `kakawi_basetracker`.`players` `p` 
        on `s`.`ref_player` = `p`.`id_player`
    )
    join `kakawi_basetracker`.`alliances` `a`
    on `s`.`ref_alliance` = `a`.`id_alliance`
)
where (
    `s`.`id_snapshot` = (
        select `s2`.`id_snapshot`
        from `kakawi_basetracker`.`snapshots` `s2`
        where (
            (`s2`.`ref_player` = `s`.`ref_player`) and (`s2`.`ref_world` = `s`.`ref_world`) and (`s2`.`timestamp` > (now() - interval 5 day))
        )
        order by `s2`.`id_snapshot` desc limit 1
    )
)
order by `p`.`name`,`s`.`ref_world`,`sd`.`base_id`

表格快照：

CREATE TABLE `snapshots` (
 `id_snapshot` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
 `ref_player` smallint(5) unsigned zerofill NOT NULL,
 `ref_world` smallint(10) unsigned zerofill NOT NULL,
 `ref_alliance` smallint(10) unsigned zerofill DEFAULT NULL,
 `alliance_role` varchar(32) DEFAULT NULL,
 `classe` enum('GDI','NOD') NOT NULL,
 `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
 `ip_address` varchar(64) NOT NULL DEFAULT '0.0.0.0',
 `rank` smallint(5) unsigned NOT NULL,
 `is_substitution` tinyint(1) NOT NULL,
 `nb_off_bases` tinyint(3) unsigned NOT NULL,
 `moy_defense` decimal(5,2) unsigned NOT NULL,
 `nb_sup_bases` tinyint(3) unsigned NOT NULL,
 `moy_sup_lvl` decimal(5,2) unsigned NOT NULL,
 `moy_def_facility` decimal(5,2) unsigned NOT NULL,
 `max_cp` smallint(5) unsigned NOT NULL,
 `research_points` bigint(20) unsigned NOT NULL,
 `supply_points` smallint(5) unsigned NOT NULL,
 `command_points` mediumint(8) unsigned NOT NULL,
 `credits` bigint(20) unsigned NOT NULL,
 `max_repa` smallint(5) unsigned NOT NULL,
 `nb_bases` tinyint(3) unsigned NOT NULL,
 `prod_credit` bigint(20) unsigned NOT NULL,
 `prod_tiberium` bigint(20) unsigned NOT NULL,
 `prod_crystal` bigint(20) unsigned NOT NULL,
 `prod_energy` bigint(20) unsigned NOT NULL,
 `base1_lvl` decimal(5,2) unsigned NOT NULL,
 `base1_def` decimal(5,2) unsigned NOT NULL,
 `base1_off` decimal(5,2) unsigned NOT NULL,
 `base1_prod_energy` bigint(20) unsigned NOT NULL,
 `base1_avia_repa` mediumint(8) unsigned NOT NULL,
 `base1_vehi_repa` mediumint(8) unsigned NOT NULL,
 `base1_inf_repa` mediumint(8) unsigned NOT NULL,
 `base2_lvl` decimal(5,2) unsigned NOT NULL,
 `base2_def` decimal(5,2) unsigned NOT NULL,
 `base2_off` decimal(5,2) unsigned NOT NULL,
 `base2_prod_energy` bigint(20) unsigned NOT NULL,
 `base2_avia_repa` mediumint(8) unsigned NOT NULL,
 `base2_vehi_repa` mediumint(8) unsigned NOT NULL,
 `base2_inf_repa` mediumint(8) unsigned NOT NULL,
 PRIMARY KEY (`id_snapshot`),
 KEY `snapshots_ref_player_idx` (`ref_player`),
 KEY `snapshots_ref_alliance_idx` (`ref_alliance`),
 KEY `snapshots_ref_world_idx` (`ref_world`),
 KEY `snapshots_ref_player_ref_world_timestamp_idx` (`ref_player`,`ref_world`,`timestamp`),
 CONSTRAINT `snapshots_ref_alliance_fkey` FOREIGN KEY (`ref_alliance`) REFERENCES `alliances` (`id_alliance`) ON DELETE SET NULL ON UPDATE CASCADE,
 CONSTRAINT `snapshots_ref_player_fkey` FOREIGN KEY (`ref_player`) REFERENCES `players` (`id_player`) ON DELETE CASCADE ON UPDATE CASCADE,
 CONSTRAINT `snapshots_ref_world_fkey` FOREIGN KEY (`ref_world`) REFERENCES `worlds` (`id_world`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=12162 DEFAULT CHARSET=utf8

表格快照详细信息：

CREATE TABLE `snapshot_details` (
 `id_detail` bigint(20) unsigned zerofill NOT NULL AUTO_INCREMENT,
 `ref_snapshot` int(10) unsigned zerofill NOT NULL,
 `ingame_id` int(10) unsigned NOT NULL,
 `base_id` tinyint(3) unsigned NOT NULL,
 `base_name` varchar(256) NOT NULL,
 `base_lvl` decimal(5,2) unsigned NOT NULL,
 `base_off` decimal(5,2) unsigned NOT NULL,
 `base_def` decimal(5,2) unsigned NOT NULL,
 `construction_yard` tinyint(3) unsigned NOT NULL,
 `defense_facility` tinyint(3) unsigned NOT NULL,
 `defense_hq` tinyint(3) unsigned NOT NULL,
 `command_center` tinyint(3) unsigned NOT NULL,
 `support_lvl` tinyint(3) unsigned NOT NULL,
 `support_type` enum('Ion','Air','Art') DEFAULT NULL,
 `prod_credit` int(10) unsigned NOT NULL,
 `prod_energy` int(10) unsigned NOT NULL,
 `prod_tiberium` int(10) unsigned NOT NULL,
 `prod_crystal` int(10) unsigned NOT NULL,
 PRIMARY KEY (`id_detail`),
 UNIQUE KEY `snapshot_details_ref_snapshot_base_id_uniq` (`ref_snapshot`,`base_id`) USING BTREE,
 CONSTRAINT `snapshot_details_ref_snapshot_fkey` FOREIGN KEY (`ref_snapshot`) REFERENCES `snapshots` (`id_snapshot`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=61299 DEFAULT CHARSET=utf8

桌面玩家：

CREATE TABLE `players` (
 `id_player` smallint(5) unsigned zerofill NOT NULL AUTO_INCREMENT,
 `ingame_id` int(5) unsigned NOT NULL,
 `name` varchar(64) NOT NULL,
 PRIMARY KEY (`id_player`),
 UNIQUE KEY `players_ingame_id_uniq` (`ingame_id`)
) ENGINE=InnoDB AUTO_INCREMENT=200 DEFAULT CHARSET=utf8

餐桌联盟：

CREATE TABLE `alliances` (
 `id_alliance` smallint(5) unsigned zerofill NOT NULL AUTO_INCREMENT,
 `ingame_id` smallint(5) unsigned NOT NULL,
 `ref_world` smallint(5) unsigned zerofill NOT NULL,
 `name` varchar(64) NOT NULL,
 PRIMARY KEY (`id_alliance`),
 UNIQUE KEY `alliances_ingame_id_ref_world_uniq` (`ingame_id`,`ref_world`) USING BTREE,
 KEY `alliances_ref_world_fkey` (`ref_world`),
 CONSTRAINT `alliances_ref_world_fkey` FOREIGN KEY (`ref_world`) REFERENCES `worlds` (`id_world`)
) ENGINE=InnoDB AUTO_INCREMENT=203 DEFAULT CHARSET=utf8

用原始查询解释：

id  select_type     table   type    possible_keys   key     key_len     ref     rows    Extra   
1   PRIMARY     p   ALL     PRIMARY     NULL    NULL    NULL    197     Using temporary; Using filesort
1   PRIMARY     s   ref     PRIMARY,snapshots_ref_player_idx,snapshots_ref_all...   snapshots_ref_player_idx    2   kakawi_basetracker.p.id_player  27  Using where
1   PRIMARY     a   eq_ref  PRIMARY     PRIMARY     2   kakawi_basetracker.s.ref_alliance   1   
1   PRIMARY     sd  ref     snapshot_details_ref_snapshot_base_id_uniq  snapshot_details_ref_snapshot_base_id_uniq  4   func    2   Using where
2   DEPENDENT SUBQUERY  s2  index   snapshots_ref_player_idx,snapshots_ref_world_idx    PRIMARY     4   NULL    1   Using where

用 user4621032 的建议解释一下：

id  select_type     table   type    possible_keys   key     key_len     ref     rows    Extra   
1   PRIMARY     <derived2>  ALL     NULL    NULL    NULL    NULL    899     Using where; Using temporary; Using filesort
1   PRIMARY     a   eq_ref  PRIMARY     PRIMARY     2   s.ref_alliance  1   
1   PRIMARY     p   eq_ref  PRIMARY     PRIMARY     2   s.ref_player    1   
1   PRIMARY     sd  ref     snapshot_details_ref_snapshot_base_id_uniq  snapshot_details_ref_snapshot_base_id_uniq  4   func    2   Using where
3   DEPENDENT SUBQUERY  s2  index   snapshots_ref_player_idx,snapshots_ref_world_idx    PRIMARY     4   NULL    1   Using where
2   DERIVED     snapshots   ALL     NULL    NULL    NULL    NULL    11898   Using where

【问题讨论】：

索引并通过explain运行它请不要向我们展示构建alliances的步骤；只需向我们展示最终的SHOW CREATE TABLE。 【参考方案1】：

类似这样的：选择最近五天的所需记录记录。然后从中选择需要的记录，然后加入需要的表记录。

select
.....
from (
    (
        (
            (select * from `kakawi_basetracker`.`snapshots` where `timestamp` > (now() - interval 5 day)) `s`
            join `kakawi_basetracker`.`snapshot_details` `sd`
            on `s`.`id_snapshot` = `sd`.`ref_snapshot`
        )
        join `kakawi_basetracker`.`players` `p` 
        on `s`.`ref_player` = `p`.`id_player`
    )
    join `kakawi_basetracker`.`alliances` `a`
    on `s`.`ref_alliance` = `a`.`id_alliance`
)
where (
    `s`.`id_snapshot` = (
        select `s2`.`id_snapshot`
        from `kakawi_basetracker`.`snapshots` `s2`
        where (
            (`s2`.`ref_player` = `s`.`ref_player`) and (`s2`.`ref_world` = `s`.`ref_world`)
        )
        order by `s2`.`id_snapshot` desc limit 1
    )
)
order by `p`.`name`,`s`.`ref_world`,`sd`.`base_id`

【讨论】：

不能在 mysql 视图中使用子查询。所以我创建了一个新视图 (select * from snapshots where timestamp > (now() - interval 5 day)) 从大约 30 秒到 0.12 秒非常感谢。【参考方案2】：

添加此索引：

S2:  INDEX(ref_player, ref_world, timestamp)

VIEW 运行速度是否比单独执行查询慢？（有时这是一个因素。）

请提供EXPLAIN SELECT ...。

【讨论】：

有了这个索引，原始查询需要 1 秒而不是 30 秒。

以上是关于优化一个巨大的 MySQL 视图的主要内容，如果未能解决你的问题，请参考以下文章

「mysql优化专题」视图应用竟然还可以这么优化？不得不收藏

数据库优化专题MySQL视图优化

MYSQL数据太大时怎么优化视图？存储过程？ - 技术问答

《数据库优化》- MySQL视图

优化多个 MySQL 视图的连接

使用聚合函数优化 MySQL 视图