查询花费了太多时间与恼人的性能[关闭]

Posted

技术标签:

【中文标题】查询花费了太多时间与恼人的性能[关闭]【英文标题】:query taking too much time with irritated performance [closed] 【发布时间】:2013-01-16 19:35:49 【问题描述】:
    我的查询需要很长时间,大约 1 小时 它正在影响数据库性能。 即使是解释计划也需要很长时间。 请重写查询以获得最佳性能。

查询:

SELECT Count(*)
FROM   (SELECT paid.keyword_id,
               paid.keyword_name,
               stat.orgentrances,
               keyword.rank1,
               keyword.rank_check,
               Sum(paid.clicks)                                     AS sumclick,
               Sum(paid.clicks * paid.avg_position)                 AS
               sumclickavgpos,
               Sum(paid.itemrevenue)                                AS sumitem,
               Sum(paid.cost)                                       AS sumcost,
               Sum(paid.transactions)                               AS sumtrans,
               Sum(paid.impressions)                                AS
               sumimpress,
               IF(Sum(paid.impressions) = 0, 0, Sum(
               paid.impressions * paid.avg_position) / Sum
                                                (paid.impressions)) AS
                      sumimpressavgrank,
               con.item_revenue,
               con.transactions,
               keyword.monthly_search_volume
        FROM   `t_keyword_paid_analytics_google_ib` paid
               LEFT JOIN (SELECT outer_t.keyword_id,
                                 Sum(outer_t.item_revenue) AS item_revenue,
                                 Sum(outer_t.transactions) AS transactions
                          FROM   t_keyword_conversion_ga_ib outer_t
                          WHERE  outer_t.own_domain_id = 720
                                 AND outer_t.traffic_date >= '2012-12-01'
                                 AND outer_t.traffic_date <= '2012-12-31'
                          GROUP  BY outer_t.keyword_id) con
                      ON paid.keyword_id = con.keyword_id
               LEFT JOIN (SELECT outer_t.keyword_id,
                                 Sum(outer_t.entrances) AS orgEntrances
                          FROM   t_keyword_stat_ga_ib outer_t
                          WHERE  outer_t.own_domain_id = 720
                                 AND outer_t.traffic_date >= '2012-12-01'
                                 AND outer_t.traffic_date <= '2012-12-31'
                                 AND ( outer_t.medium = 'organic'
                                        OR outer_t.medium IS NULL )
                          GROUP  BY outer_t.keyword_id) stat
                      ON paid.keyword_id = stat.keyword_id
               LEFT JOIN `t_managed_keyword_ib` keyword
                      ON keyword.id = paid.keyword_id
        WHERE  paid.own_domain_id = 720
               AND paid.traffic_date >= '2012-12-01'
               AND paid.traffic_date <= '2012-12-31'
               AND ( paid.channel IS NULL
                      OR paid.channel = 'Google' )
        GROUP  BY paid.keyword_id
    HAVING paid.keyword_id IS NOT NULL) tempt; 

(outer_t.medium = 'organic' or outer_t.medium is null ) after outer_t.own_domain_id = 720 and (paid.channel is null or paid.channel = 'Google') 
after paid.own_domain_id = 720

表结构:

mysql> show create table t_keyword_paid_analytics_google_ib\G
*************************** 1. row ***************************
Table: t_keyword_paid_analytics_google_ib
Create Table: CREATE TABLE `t_keyword_paid_analytics_google_ib` (
`keyword_name` varchar(255) DEFAULT NULL,
`id` int(11) NOT NULL,
`keyword_id` int(11) NOT NULL,
`target_url_id` int(11) DEFAULT NULL,
`own_domain_id` int(11) NOT NULL,
`log_date` date NOT NULL,
`traffic_date` date NOT NULL,
`impressions` int(11) DEFAULT NULL,
`clicks` int(11) DEFAULT NULL,
`entrances` int(11) DEFAULT NULL,
`match_type` int(11) DEFAULT NULL COMMENT '1: Phrase, 2: Exact, 3:Broad 4: etc',
`ad_group_name` varchar(200) DEFAULT NULL,
`ad_distribution_network` varchar(500) CHARACTER SET latin1 DEFAULT NULL,
`match_query` varchar(500) CHARACTER SET latin1 DEFAULT NULL,
`cost` decimal(10,2) DEFAULT NULL,
`cpm` decimal(10,2) DEFAULT NULL,
`ctr` decimal(10,2) DEFAULT NULL COMMENT 'percent',
`cpc` decimal(10,2) DEFAULT NULL,
`campaign` varchar(200) CHARACTER SET latin1 DEFAULT NULL,
`keyword_status` tinyint(4) DEFAULT NULL COMMENT '1: Active,2: Approved, 3: Disapproved, 4: Paused , 5:Pending, 6: Failed, 7:etc',
`ad_group_status` tinyint(4) DEFAULT NULL COMMENT '1:ELIGIBLE; 2=PAUSED;3=LOW_SEARCH_VOLUME;4 =LOW_QUALITY_SCORE; 5=DISAPPROVED; 6=AD_GROUP_PAUSED; 7=etc',
`max_cpc` decimal(10,2) DEFAULT NULL,
`quality_score` tinyint(4) DEFAULT NULL,
`channel` varchar(100) DEFAULT NULL,
`first_page_cpc` decimal(10,2) DEFAULT NULL,
`avg_position` decimal(10,2) DEFAULT NULL,
`itemRevenue` decimal(10,2) DEFAULT NULL,
`goal1value` decimal(10,2) DEFAULT NULL,
`goal2value` decimal(10,2) DEFAULT NULL,
`goal3value` decimal(10,2) DEFAULT NULL,
`goal4value` decimal(10,2) DEFAULT NULL,
`transactions` int(10) DEFAULT NULL,
`goal1completions` int(10) DEFAULT NULL,
`goal2completions` int(10) DEFAULT NULL,
`goal3completions` int(10) DEFAULT NULL,
`goal4completions` int(10) DEFAULT NULL
) ENGINE=BRIGHTHOUSE DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

mysql> show create table t_keyword_conversion_ga_ib\G
*************************** 1. row ***************************
Table: t_keyword_conversion_ga_ib
Create Table: CREATE TABLE `t_keyword_conversion_ga_ib` (
`keyword_name` varchar(255) COLLATE latin1_bin DEFAULT NULL,
`id` int(11) NOT NULL,
`own_domain_id` int(11) DEFAULT NULL,
`keyword_id` int(11) DEFAULT NULL,
`traffic_date` date DEFAULT NULL,
`targeturl_id` int(11) DEFAULT NULL,
`entrance` int(11) DEFAULT NULL,
`transactions` int(11) DEFAULT NULL,
`item_revenue` decimal(9,2) DEFAULT NULL,
`goal1completions` int(11) DEFAULT NULL,
`goal2completions` int(11) DEFAULT NULL,
`goal3completions` int(11) DEFAULT NULL,
`goal4completions` int(11) DEFAULT NULL,
`goal5completions` int(11) DEFAULT NULL,
`goal6completions` int(11) DEFAULT NULL,
`goal7completions` int(11) DEFAULT NULL,
`goal8completions` int(11) DEFAULT NULL,
`goal9completions` int(11) DEFAULT NULL,
`goal10completions` int(11) DEFAULT NULL,
`goal1Value` decimal(9,2) DEFAULT NULL,
`goal2Value` decimal(9,2) DEFAULT NULL,
`goal3Value` decimal(9,2) DEFAULT NULL,
`goal4Value` decimal(9,2) DEFAULT NULL,
`goal5Value` decimal(9,2) DEFAULT NULL,
`goal6Value` decimal(9,2) DEFAULT NULL,
`goal7Value` decimal(9,2) DEFAULT NULL,
`goal8Value` decimal(9,2) DEFAULT NULL,
`goal9Value` decimal(9,2) DEFAULT NULL,
`goal10Value` decimal(9,2) DEFAULT NULL,
`medium` varchar(255) COLLATE latin1_bin DEFAULT NULL,
`source` varchar(255) COLLATE latin1_bin DEFAULT NULL
) ENGINE=BRIGHTHOUSE DEFAULT CHARSET=latin1 COLLATE=latin1_bin
1 row in set (0.00 sec)

请帮帮我。

【问题讨论】:

在这里提出具体问题,参考***.com/faq 我们是否应该猜测您的查询应该做什么?如果您希望有人提供帮助,至少要正确格式化。 对不起,我为我的错误道歉。请在这里帮助我。 您的查询似乎做了很多分组和求和,只是为了返回一个数字 (count(*))。如果您只想要计数,请不要费心进行所有计算。 【参考方案1】:

您没有指明任何索引。您将需要它们来提高性能。

好的,分号后面的部分我不明白,所以我会忽略它。

您的整个查询对子选择进行计数(*),但您在子查询中进行了一堆聚合(调用 SUM),这是数据库不必做的工作,所以消除它。

您在进一步的子查询中进行聚合,所以把它删掉。

在你的主子选择中,你只是在做一个计数(*),你选择不在你的组中的字段,这是在不影响计数的数据库上工作的,所以摆脱它.

这给了我们类似...

select count(*)
from (select paid.keyword_id
      from `t_keyword_paid_analytics_google_ib` paid
        left join
          (select outer_t.keyword_id
           from t_keyword_conversion_ga_ib outer_t
           where ...
           group by outer_t.keyword_id  ) con
          on paid.keyword_id = con.keyword_id
        left join
          (select outer_t.keyword_id
           from t_keyword_stat_ga_ib outer_t
           where ...
           group by outer_t.keyword_id) stat on paid.keyword_id = stat.keyword_id
        left join `t_managed_keyword_ib` keyword on keyword.id = paid.keyword_id
     where ...
     group by paid.keyword_id
     having paid.keyword_id is not null
    ) tempt;

此外,计数可能比选择更快,您似乎正在寻找关键字 ID 并最终计算它们。如果您可以边走边算,那将比选择要好。我不确定您是否可以将其应用于此查询,但请记住这一点。

【讨论】:

【参考方案2】:

你必须为你的表添加一些索引,

ALTER TABLE table_name ADD UNIQUE INDEX cate_id(field_name);

这里是documentation

在您的情况下,我会将索引添加到:

paid.keyword_id
con.keyword_id
stat.keyword_id
keyword.id
paid.keyword_id

因为您正在加入 on 这些字段

【讨论】:

以上是关于查询花费了太多时间与恼人的性能[关闭]的主要内容,如果未能解决你的问题,请参考以下文章

循环渲染的 Django 模板查询花费了太多时间

在 MySQL 中进行排序时查询花费了太多时间

需要帮助来优化 ORACLE SQL 查询 [关闭]

Hibernate 花费了太多时间并执行了一些神秘的操作。

从 Sqlite 游标创建 Pojo 类花费了太多时间

与硬编码值相比,Oracle SQL In Subquery 花费了太多时间