在其他连接的 select 语句中优化联合连接

Posted

技术标签:

【中文标题】在其他连接的 select 语句中优化联合连接【英文标题】:optimizing a union join inside select statement of other joins 【发布时间】:2013-06-17 16:57:54 【问题描述】:

我有一个由 3 -4 部分组成的查询。一旦我添加联合加入和加入,这需要超过 140 秒才能运行。如何更改联合联接以更快地执行它。

SELECT
    testing.CLIENTID,
    testing.COMPANY,
    testing.CONTACT,
    testing.CONTACTID,
    `orders`.`ORDERNO` AS `ORDERNO`,
    `orders`.`BIDNO` AS `BIDNO`,
    `projects`.`PROJID` AS `PROJID`,
    `projects`.`PROJCODE` AS `PROJCODE`,
    `projects`.`StartDate` AS `StartDate`,
    `category`.`type` AS `CATEGORY`,
    `projects`.`country` AS `COUNTRY`,
    `projects`.`VALUE` AS `VALUE`,
    `projects`.`PROCESSOR` AS `PROCESSOR`,
    `projects`.`NES` AS `NES`,
    `projects`.`SPECSALE` AS `SPECSALE`,
    `projects`.`OFFICE` AS `OFFICE`,
    `projects`.`LORM` AS `LORM`,
    `lookupcountry`.`REGION` AS `REGION`
FROM
    (
        (
            (
                (
                    (
                        (
                            SELECT
                                contactmerge.CLIENTID,
                                contactmerge.CONTACT,
                                contactmerge.CONTACTID,
                                accountmerge.COMPANY
                            FROM
                                (
                                    SELECT
                                        `hdb`.`contacts`.`CONTACTID` AS `CONTACTID`,
                                        `hdb`.`contacts`.`CLIENTID` AS `CLIENTID`,
                                        concat(
                                            `hdb`.`contacts`.`FIRSTNAME`,
                                            " ",
                                            `hdb`.`contacts`.`LASTNAME`
                                        ) AS CONTACT,
                                        _utf8 'paradox' AS `SOURCEDATABASE`
                                    FROM
                                        `hdb`.`contacts`
                                    UNION
                                        SELECT
                                            `sugarcrm`.`contacts`.`id` AS `CONTACTID`,
                                            `sugarcrm`.`accounts_contacts`.`account_id` AS `CLIENTID`,
                                            concat(
                                                `sugarcrm`.`contacts`.`first_name`,
                                                " ",
                                                `sugarcrm`.`contacts`.`last_name`
                                            ) AS CONTACT,
                                            _utf8 'sugar' AS `SOURCEDATABASE`
                                        FROM
                                            (
                                                (
                                                    (
                                                        (
                                                            `sugarcrm`.`contacts`
                                                            LEFT JOIN `sugarcrm`.`email_addr_bean_rel` ON (
                                                                (
                                                                    (
                                                                        `sugarcrm`.`contacts`.`id` = `sugarcrm`.`email_addr_bean_rel`.`bean_id`
                                                                    )
                                                                    AND (
                                                                        (
                                                                            `sugarcrm`.`email_addr_bean_rel`.`primary_address` = 1
                                                                        )
                                                                        OR (
                                                                            (
                                                                                `sugarcrm`.`email_addr_bean_rel`.`primary_address` IS NOT NULL
                                                                            )
                                                                            AND (
                                                                                `sugarcrm`.`email_addr_bean_rel`.`primary_address` <> 0
                                                                            )
                                                                        )
                                                                    )
                                                                )
                                                            )
                                                        )
                                                        LEFT JOIN `sugarcrm`.`accounts_contacts` ON (
                                                            (
                                                                `sugarcrm`.`contacts`.`id` = `sugarcrm`.`accounts_contacts`.`contact_id`
                                                            )
                                                        )
                                                    )
                                                    JOIN `sugarcrm`.`email_addresses` ON (
                                                        (
                                                            `sugarcrm`.`email_addr_bean_rel`.`email_address_id` = `sugarcrm`.`email_addresses`.`id`
                                                        )
                                                    )
                                                )
                                                LEFT JOIN `sugarcrm`.`accounts` ON (
                                                    (
                                                        `sugarcrm`.`accounts`.`id` = `sugarcrm`.`accounts_contacts`.`account_id`
                                                    )
                                                )
                                            )
                                ) AS contactmerge
                            LEFT JOIN (
                                SELECT
                                    CLIENTID,
                                    `hdb`.`clients`.`COMPANY` AS `COMPANY`
                                FROM
                                    `hdb`.`clients`
                                UNION
                                    SELECT
                                        id AS CLIENTID,
                                        `sugarcrm`.`accounts`.`name` AS `COMPANY`
                                    FROM
                                        `sugarcrm`.`accounts`
                            ) AS accountmerge ON contactmerge.CLIENTID = accountmerge.CLIENTID
                        ) AS testing
                    )
                    JOIN `orders` ON (
                        (
                            `testing`.`CONTACTID` = `orders`.`CONTACTID`
                        )
                    )
                )
                JOIN `projects` ON (
                    (
                        `orders`.`ORDERNO` = `projects`.`ORDERNO`
                    )
                )
            )
            JOIN `category` ON (
                (
                    `category`.`category_id` = `projects`.`category_id`
                )
            )
        )
        LEFT JOIN `lookupcountry` ON (
            (
                CONVERT (
                    `lookupcountry`.`COUNTRY` USING utf8
                ) = CONVERT (
                    `projects`.`country` USING utf8
                )
            )
        )
    )
ORDER BY
    `testing`.`COMPANY`,
    `projects`.`StartDate`

名为testing 的表别名需要很长时间才能执行。然后我需要把它变成一个视图

原始查询没有加入sugarcrm。

SELECT
    `clients`.`CORPORATE` AS `CORPORATE`,
    `clients`.`COMPANY` AS `COMPANY`,
    `clients`.`CLIENTID` AS `CLIENTID`,
    `contacts`.`CONTACTID` AS `CONTACTID`,
    concat(
            `contacts`.`LASTNAME`,
            `contacts`.`FIRSTNAME`,     
            `contacts`.`INITIALS`
    ) AS `Contact`,
    `orders`.`ORDERNO` AS `ORDERNO`,
    `orders`.`BIDNO` AS `BIDNO`,
    `projects`.`PROJID` AS `PROJID`,
    `projects`.`PROJCODE` AS `PROJCODE`,
    `projects`.`StartDate` AS `StartDate`,
    `category`.`type` AS `CATEGORY`,
    `projects`.`country` AS `COUNTRY`,
    `projects`.`VALUE` AS `VALUE`,
    `projects`.`PROCESSOR` AS `PROCESSOR`,
    `projects`.`NES` AS `NES`,
    `projects`.`SPECSALE` AS `SPECSALE`,
    `projects`.`OFFICE` AS `OFFICE`,
    `projects`.`LORM` AS `LORM`,
    `lookupcountry`.`REGION` AS `REGION`
FROM
    (
        (
            (
                (
                    (
                        `clients`
                        JOIN `contacts` ON (
                            (
                                `clients`.`CLIENTID` = `contacts`.`CLIENTID`
                            )
                        )
                    )
                    JOIN `orders` ON (
                        (
                            `contacts`.`CONTACTID` = `orders`.`CONTACTID`
                        )
                    )
                )
                JOIN `projects` ON (
                    (
                        `orders`.`ORDERNO` = `projects`.`ORDERNO`
                    )
                )
            )
            JOIN `category` ON (
                (
                    `category`.`category_id` = `projects`.`category_id`
                )
            )
        )
        LEFT JOIN `lookupcountry` ON (
            (
                CONVERT (
                    `lookupcountry`.`COUNTRY` USING utf8
                ) = CONVERT (
                    `projects`.`country` USING utf8
                )
            )
        )
    )
ORDER BY
    `clients`.`CORPORATE`,
    `clients`.`COMPANY`,
    `contacts`.`LASTNAME`,
    `projects`.`StartDate`

【问题讨论】:

【参考方案1】:

您从 sugarcrm.contacts 到 sugarcrm.email_addr_bean_rel 的 LEFT JOIN 在 id=bean_id 上是可以的,但是你对 Primary_Address = 1 的测试 OR (primary address IS NOT NULL AND primary_address 0 ) 是浪费的。

不为空意味着它有一个值。 1 的第一个限定符是可以的,但随后 您测试任何不等于 0 的地址(因此 1 是,但 2、3、400、1809 或 任何其他号码。那么为什么不直接看我如何简化它呢。

SELECT
      O.ORDERNO,
      O.BIDNO,

      CASE when c.ContactID IS NULL
         then sc.id
         ELSE c.contactid  END as ContactID,

      CASE when c.ContactID IS NULL
         then sac.account_id
         ELSE c.clientid   END as ClientID,

      CASE when c.ContactID IS NULL
         then concat( sc.first_name, " ", sc.last_name ) 
         ELSE concat( c.FIRSTNAME, " ", c.LASTNAME )  END as Contact,

      CASE when c.ContactID IS NULL
         then sCli.`name`
         ELSE cCli.Company  END as Company,

      CASE when c.ContactID IS NULL
         then _utf8 'sugar' 
         ELSE _utf8 'paradox'  END as SOURCEDATABASE,

      P.PROJID,
      P.PROJCODE,
      P.StartDate,
      Cat.`type` AS CATEGORY,
      P.`country` AS COUNTRY,
      P.`VALUE` AS `VALUE`,
      P.PROCESSOR,
      P.NES,
      P.SPECSALE,
      P.OFFICE,
      P.LORM,
      LC.REGION
   FROM 
      orders O
         JOIN projects P
            ON O.ORDERNO = P.ORDERNO
            JOIN category Cat 
               ON P.category_id = Cat.category_id
            LEFT JOIN lookupcountry LC
               ON CONVERT( P.`country` USING utf8 ) = CONVERT( LC.COUNTRY USING utf8 )

         LEFT JOIN hdb.contacts c
            ON  O.ContactID = c.ClientID
            LEFT JOIN hdb.clients cCli
               ON c.ClientID = cCli.ClientID

         LEFT JOIN sugarcrm.contacts sc
            ON O.ContactID = sc.id
            LEFT JOIN sugarcrm.accounts sCli
               ON sc.id = sCli.id
            LEFT JOIN sugarcrm.accounts_contacts sac
               ON sc.id = sac.contact_id
               LEFT JOIN sugarcrm.accounts Acc 
                  ON sac.account_id = Acc.id

            LEFT JOIN sugarcrm.email_addr_bean_rel EABR
               ON sc.id = EABR.bean_id
              AND EABR.primary_address IS NOT NULL
              LEFT JOIN sugarcrm.email_addresses EA 
                 ON EABR.email_address_id = EA.id
ORDER BY
   CASE when c.ContactID IS NULL
      then sCli.`name`
      ELSE cCli.Company  END,
   P.StartDate

我不介意提供帮助,但是从现在开始,您应该看看我在做什么...建立关系...从您的数据(订单)的基础开始并查看 ONE PATH关于如何连接到您的“联系人”表......写下这些连接(作为左连接)。然后,将您的路径写入 SUGAR 帐户联系人并写入 THOSE 连接(也是左连接)。不要尝试预先查询所有可能的联系人,而是使用 CASE/WHEN 来确定基于空路由而不是像我对联系人、客户、公司等进行的获取。您将从一条路径获取数据与其他...保持一致即可。

【讨论】:

我不完全理解您提出的问题。为什么订单在第 1 节?此查询也不返回准确的结果集。我应该得到 6250 条记录,见上面的原始查询。您的查询给了我 9455 条记录。 我想我已经修复了它,关系是错误的,所以我把它改成了LEFT JOIN hdb.contacts c ON O.ContactID = c.CONTACTID,非常感谢。 @sharif,很高兴你修复了它,我刚醒来,无法早点回复。至于订单,这是您想要的数据的基础,从订单中您获得产品以及购买者。我从获取的主要数据开始沿着链向下工作,并获取所有额外的查找内容(例如联系人、电子邮件等) 我想我现在明白了,我把主表放在 from 部分,然后离开加入。我发布的查询让我感到困惑,因为顶部没有订单。再次感谢

以上是关于在其他连接的 select 语句中优化联合连接的主要内容,如果未能解决你的问题,请参考以下文章

第4讲:多表联合查询

MySQL学习笔记连接子分页联合查询以及sql语句执行顺序总结

MySQL 通过使用连接查询来优化联合查询

提高系统性能——对SQL语句优化的思考

联合查询、表连接查询、子查询三种查询的特点和注意事项各是啥

同一组连接表上的 Mysql 联合