MySQL ORDER BY 子句将查询速度降低 4 倍
Posted
技术标签:
【中文标题】MySQL ORDER BY 子句将查询速度降低 4 倍【英文标题】：MySQL ORDER BY clause slowing query by 4x 【发布时间】：2021-11-17 23:36:58 【问题描述】：更新：我已经大大简化了下面的调用并对问题进行了调整。
我正在想办法让这个查询更快。
我的查询检索特定活动（在本例中为 4）中个人的个人/通信信息。我加入了最近的通信并按通信 date_created 值排序。下面是我们使用的 SQL。
请注意，我之所以使用 LEFT JOIN，是因为只有少数人可能会进行与他们相关的对话；我仍然想根据核心选择返回匹配的人（例如，如果 5 人进行了对话，但有 30 人，我仍然想显示全部 30 人，但按有对话的人以及这些对话的日期排序）。
以下是执行时间仅为 350 毫秒的 FAST 调用；问题是由于 AND 子句，它相当于 INNER JOIN，因此不会 LEFT JOIN 个人主记录。
SELECT *
FROM person_main pm
LEFT JOIN cms_conversation_member ccmem ON ccmem.member_object_record_id = pm.ID
AND ccmem.member_type_c = get_gcs('cms_conversation_member', 'member_type_c', 'internal')
AND ccmem.member_object_type_sc = get_gss('OBJECT_BASE','PERSON')
AND ccmem.member_object_subtype_sc = get_gss('OBJECT_PERSON','MAIN')
LEFT JOIN cms_conversation_main convm ON convm.ID = ccmem.cms_conversation_main_ref_id
LEFT JOIN cms_communication_main ccomm ON ccomm.cms_conversation_main_ref_id = convm.ID
AND ccomm.comm_direction_c = get_gcs('cms_communication_main', 'comm_direction_c', 'incoming')
WHERE 1 = 1
GROUP BY pm.ID
ORDER BY ccomm.date_created
LIMIT 0,30;
这是调用的 SQL 解释
[
"id": "1",
"select_type": "PRIMARY",
"table": "ci",
"partitions": "null",
"type": "ref",
"possible_keys": "UK_campaign_item",
"key": "UK_campaign_item",
"key_len": "6",
"ref": "const,const,const",
"rows": "1000",
"filtered": "100.0",
"Extra": "Using index; Using temporary; Using filesort"
,
"id": "1",
"select_type": "PRIMARY",
"table": "pm",
"partitions": "null",
"type": "eq_ref",
"possible_keys": "PRIMARY,ID_UNIQUE,IDX_person_main_name_first,IDX_person_main_name_middle,IDX_person_main_name_last,IDX_person_main_name_nick",
"key": "PRIMARY",
"key_len": "4",
"ref": "xxx.ci.item_object_ref_id",
"rows": "1",
"filtered": "100.0",
"Extra": "null"
,
"id": "1",
"select_type": "PRIMARY",
"table": "ccomm",
"partitions": "null",
"type": "eq_ref",
"possible_keys": "PRIMARY,UK_cms_communication_main_ID",
"key": "PRIMARY",
"key_len": "4",
"ref": "func",
"rows": "1",
"filtered": "100.0",
"Extra": "Using where"
,
"id": "2",
"select_type": "DEPENDENT SUBQUERY",
"table": "ccomm2",
"partitions": "null",
"type": "index",
"possible_keys": "FK_cms_communication_main_cms_conversation_main_ref_id,IDX_cms_communication_main_comm_direction_c",
"key": "IDX_cms_communication_main_date_created",
"key_len": "5",
"ref": "null",
"rows": "2",
"filtered": "40.0",
"Extra": "Using where; Backward index scan"
,
"id": "2",
"select_type": "DEPENDENT SUBQUERY",
"table": "<subquery3>",
"partitions": "null",
"type": "eq_ref",
"possible_keys": "<auto_distinct_key>",
"key": "<auto_distinct_key>",
"key_len": "6",
"ref": "xxx.ccomm2.cms_conversation_main_ref_id,xxx.pm.ID",
"rows": "1",
"filtered": "100.0",
"Extra": "Using where"
,
"id": "3",
"select_type": "MATERIALIZED",
"table": "ccmem",
"partitions": "null",
"type": "ref",
"possible_keys": "IDX_cms_conversation_member,IDX_cms_conversation_member_member_type_c,FK_cms_conversation_member_cms_conversation_main_ref_id",
"key": "IDX_cms_conversation_member",
"key_len": "6",
"ref": "const,const,xxx.pm.ID",
"rows": "3",
"filtered": "100.0",
"Extra": "Using where"
]
这是返回所有记录的调用（即使是没有通信的人），但需要 2.4 秒（比上述调用慢 6 倍）。 但这是我需要的结果数据。
SELECT *
FROM person_main pm
LEFT JOIN cms_conversation_member ccmem ON ccmem.member_object_record_id = pm.ID
AND ccmem.member_type_c = get_gcs('cms_conversation_member', 'member_type_c', 'internal')
AND ccmem.member_object_type_sc = get_gss('OBJECT_BASE','PERSON')
AND ccmem.member_object_subtype_sc = get_gss('OBJECT_PERSON','MAIN')
LEFT JOIN cms_conversation_main convm ON convm.ID = ccmem.cms_conversation_main_ref_id
LEFT JOIN cms_communication_main ccomm ON ccomm.cms_conversation_main_ref_id = convm.ID
AND ccomm.comm_direction_c = get_gcs('cms_communication_main', 'comm_direction_c', 'incoming')
WHERE 1 = 1
GROUP BY pm.ID
ORDER BY ccomm.date_created
LIMIT 0,30;
这是上述调用的解释：
[
"id": "1",
"select_type": "PRIMARY",
"table": "ci",
"partitions": "null",
"type": "ref",
"possible_keys": "UK_campaign_item",
"key": "UK_campaign_item",
"key_len": "6",
"ref": "const,const,const",
"rows": "1000",
"filtered": "100.0",
"Extra": "Using index; Using temporary; Using filesort"
,
"id": "1",
"select_type": "PRIMARY",
"table": "pm",
"partitions": "null",
"type": "eq_ref",
"possible_keys": "PRIMARY,ID_UNIQUE,IDX_person_main_name_first,IDX_person_main_name_middle,IDX_person_main_name_last,IDX_person_main_name_nick",
"key": "PRIMARY",
"key_len": "4",
"ref": "xxx.ci.item_object_ref_id",
"rows": "1",
"filtered": "100.0",
"Extra": "null"
,
"id": "1",
"select_type": "PRIMARY",
"table": "ccomm",
"partitions": "null",
"type": "eq_ref",
"possible_keys": "PRIMARY,UK_cms_communication_main_ID",
"key": "PRIMARY",
"key_len": "4",
"ref": "func",
"rows": "1",
"filtered": "100.0",
"Extra": "Using where"
,
"id": "2",
"select_type": "DEPENDENT SUBQUERY",
"table": "ccomm2",
"partitions": "null",
"type": "index",
"possible_keys": "FK_cms_communication_main_cms_conversation_main_ref_id,IDX_cms_communication_main_comm_direction_c",
"key": "IDX_cms_communication_main_date_created",
"key_len": "5",
"ref": "null",
"rows": "2",
"filtered": "40.0",
"Extra": "Using where; Backward index scan"
,
"id": "2",
"select_type": "DEPENDENT SUBQUERY",
"table": "<subquery3>",
"partitions": "null",
"type": "eq_ref",
"possible_keys": "<auto_distinct_key>",
"key": "<auto_distinct_key>",
"key_len": "6",
"ref": "xxx.ccomm2.cms_conversation_main_ref_id,xxx.pm.ID",
"rows": "1",
"filtered": "100.0",
"Extra": "Using where"
,
"id": "3",
"select_type": "MATERIALIZED",
"table": "ccmem",
"partitions": "null",
"type": "ref",
"possible_keys": "IDX_cms_conversation_member,IDX_cms_conversation_member_member_type_c,FK_cms_conversation_member_cms_conversation_main_ref_id",
"key": "IDX_cms_conversation_member",
"key_len": "6",
"ref": "const,const,xxx.pm.ID",
"rows": "3",
"filtered": "100.0",
"Extra": "Using where"
]
这里是表创建脚本。
cms_communication_main
SET NAMES 'utf8mb4';
--
-- Create table `cms_communication_main`
--
CREATE TABLE cms_communication_main (
ID int NOT NULL AUTO_INCREMENT,
date_created datetime NOT NULL,
comm_id_external varchar(1024) NOT NULL,
comm_platform_c smallint NOT NULL,
comm_direction_c tinyint NOT NULL,
comm_state_sc smallint NOT NULL,
comm_timestamp datetime DEFAULT NULL,
comm_details json DEFAULT NULL,
cms_conversation_main_ref_id int NOT NULL,
PRIMARY KEY (ID),
UNIQUE INDEX UK_cms_communication_main_ID (ID)
)
ENGINE = INNODB,
AUTO_INCREMENT = 6,
AVG_ROW_LENGTH = 3276,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci;
--
-- Create index `IDX_cms_communication_main_date_created` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD INDEX IDX_cms_communication_main_date_created (date_created);
--
-- Create index `IDX_cms_communication_main_comm_platform_c` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD INDEX IDX_cms_communication_main_comm_platform_c (comm_platform_c);
--
-- Create index `IDX_cms_communication_main_comm_direction_c` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD INDEX IDX_cms_communication_main_comm_direction_c (comm_direction_c);
--
-- Create index `IDX_cms_communication_main_comm_id_external` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD FULLTEXT INDEX IDX_cms_communication_main_comm_id_external (comm_id_external);
--
-- Create foreign key
--
ALTER TABLE cms_communication_main
ADD CONSTRAINT FK_cms_communication_main_cms_conversation_main_ref_id FOREIGN KEY (cms_conversation_main_ref_id)
REFERENCES cms_conversation_main (ID);
cms_conversation_main
SET NAMES 'utf8mb4';
CREATE TABLE cms_conversation_main (
ID int NOT NULL AUTO_INCREMENT,
date_created datetime NOT NULL,
conversation_type_c smallint NOT NULL,
conversation_id_external varchar(995) DEFAULT '',
owner_account_main_ref_id int NOT NULL,
conversation_state_sc smallint NOT NULL,
PRIMARY KEY (ID),
UNIQUE INDEX UK_cms_conversation_main_ID (ID)
)
ENGINE = INNODB,
AUTO_INCREMENT = 6,
AVG_ROW_LENGTH = 47,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci;
--
-- Create index `IDX_cms_conversation_main_date_created` on table `cms_conversation_main`
--
ALTER TABLE cms_conversation_main
ADD INDEX IDX_cms_conversation_main_date_created (date_created);
--
-- Create foreign key
--
ALTER TABLE cms_conversation_main
ADD CONSTRAINT FK_cms_conversation_main_owner_account_main_ref_id FOREIGN KEY (owner_account_main_ref_id)
REFERENCES account_main (ID);
cms_conversation_member
SET NAMES 'utf8mb4';
CREATE TABLE cms_conversation_member (
ID int NOT NULL AUTO_INCREMENT,
date_joined datetime NOT NULL,
member_type_c smallint NOT NULL,
member_originator tinyint(1) NOT NULL DEFAULT 0,
member_object_type_sc smallint NOT NULL,
member_object_subtype_sc smallint NOT NULL,
member_object_record_id smallint NOT NULL,
member_custom_id varchar(995) DEFAULT NULL,
member_communication_key varchar(255) DEFAULT NULL,
cms_conversation_main_ref_id int NOT NULL,
PRIMARY KEY (ID),
UNIQUE INDEX UK_cms_conversation_member_ID (ID)
)
ENGINE = INNODB,
AUTO_INCREMENT = 11,
AVG_ROW_LENGTH = 50,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci;
--
-- Create index `IDX_cms_conversation_member_date_joined` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_date_joined (date_joined);
--
-- Create index `IDX_cms_conversation_member` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member (member_object_type_sc, member_object_subtype_sc, member_object_record_id);
--
-- Create index `IDX_cms_conversation_member_member_type_c` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_member_type_c (member_type_c);
--
-- Create index `IDX_cms_conversation_member_member_originator` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_member_originator (member_originator);
--
-- Create index `IDX_cms_conversation_member_member_communication_key` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_member_communication_key (member_communication_key);
--
-- Create foreign key
--
ALTER TABLE cms_conversation_member
ADD CONSTRAINT FK_cms_conversation_member_cms_conversation_main_ref_id FOREIGN KEY (cms_conversation_main_ref_id)
REFERENCES cms_conversation_main (ID);
知道为什么第二次调用的时间比第一次长 5-6 倍吗？为什么 mysql 在与来自 person_main 表的通信不匹配的左连接上“滞后”这么多？
【问题讨论】：
请编辑您的问题以添加（作为文本，而不是图像）show create table yourtablename 的查询和输出中的所有表的explain SELECT rest-of-your-statement 的输出，无论是否按顺序
你说的没有命令的速度是有限制还是没有限制？如果有，没有限制会有多少行，删除限制也会使其变慢吗？
@ysth 我添加了 SQL 解释。我不想用表格创建来膨胀它；但我可以说所有引用的字段都是索引的。
为了帮助您解决query-optimization 的问题，我们需要更多信息。请read this。我们确实需要您的表定义。
@O.Jones 理解并更新了。
【参考方案1】：
get_gcs() 是什么？
如果是存储函数，是DETERMINISTIC吗？
尝试使用JOIN 而不是IN ( SELECT ... )。 （好像IN (... campaign_item ci ...)已经变成了JOIN（见EXPLAIN的第一行）。
暂定索引：
ccomm: INDEX(ID, date_created)
cms_conversation_member: INDEX(member_type_c, member_object_type_sc,
member_object_subtype_sc, member_object_record_id, cms_conversation_main_ref_id)
cms_communication_main: INDEX(cms_conversation_main_ref_id,
comm_direction_c, date_created, ID)
ci: INDEX(campaign_main_ref_id, item_object_type_sc,
item_object_subtype_sc, item_object_ref_id)
您确定要LEFT JOIN，而不仅仅是INNER JOIN？毕竟，查询似乎期望这个值：ORDER BY ccomm.date_created。
您可能不需要GROUP BY。如果你这样做了，那么你可能仍然可以通过将查询从里到外来摆脱它。
为子查询计时（为pm.ID 填写一些值）。目标是确定其中之一是否是主要的性能瓶颈。
PRIMARY,ID_UNIQUE -- 这似乎暗示你有PRIMARY KEY(ID), UNIQUE(ID)。如果是这样，请注意 PK 是唯一键。因此UNIQUE(ID) 是多余的。
【讨论】：
是的，似乎几乎每个条件都包含在一个函数中，这意味着他将绕过几乎所有这些索引，除非我弄错了。 @ChrisStricland - 有迹象表明get_gss('constant', 'constant') 是不变的。也就是说，它似乎是预先评估的。因此，column = get_gss(...)可以在column上建立索引。
实际上，我想只要您不将该函数应用于列本身，就可以了。我刚刚在函数调用中看到（看起来像）一些列名并误读了它。再看一遍，我想只要你在做WHERE get_gss(col, arg) = condition 就可以了。
@ChrisStricland - 第一部分是正确的。 fcn(col) 不是“sargable”，因此索引没有用处。最后一部分是不可分割的。
感谢 Rick 的评论，但不幸的是它对我没有帮助。注意我已经更新了核心示例代码以优化和简化问题。以上是关于MySQL ORDER BY 子句将查询速度降低 4 倍的主要内容，如果未能解决你的问题，请参考以下文章
在 mysql 中使用 union 和 order by 子句