从 Laravel 5.1 升级到 Laravel 5.8 后 whereHas() 变慢

Posted 2023-03-14

【中文标题】从 Laravel 5.1 升级到 Laravel 5.8 后 whereHas() 变慢

【英文标题】：Slow whereHas() after upgrade from Laravel 5.1 to Laravel 5.8

【发布时间】：2019-12-05 10:35:56

【问题描述】：

我通过设置一个新的 5.8 项目并复制文件，在这里和那里进行一些调整，从而将应用从 Laravel 5.1 切换到 Laravel 5.8。

问题在于 whereHas 的查询变得非常缓慢。

这是一个示例代码：

Article::whereHas('categories', function ($category) 
            $category->where('link', 'foto');
        )
        ->active()
        ->recent()
        ->take(3)
        ->get();

此代码在 Laravel 5.1 上生成以下查询，并在 0.05-0.07 秒内完成。

SELECT *
FROM `articles`
WHERE `articles`.`deleted_at` IS NULL
  AND
    (SELECT count(*)
     FROM `categories`
     INNER JOIN `article_category` 
       ON `categories`.`id` = `article_category`.`category_id`
     WHERE `article_category`.`article_id` = `articles`.`id`
       AND `link` = 'foto'
       AND `categories`.`deleted_at` IS NULL) >= 1
ORDER BY IFNULL(published_at, created_at) DESC
LIMIT 3

下面是解释：

+------+--------------------+------------------+------+--------------------------------------------------------------------------+-------------------------------------+---------+-----------------+------+----------+------------------------------------+
| id   | select_type        | table            | type | possible_keys                                                            | key                                 | key_len | ref             | rows | filtered | Extra                              |
+------+--------------------+------------------+------+--------------------------------------------------------------------------+-------------------------------------+---------+-----------------+------+----------+------------------------------------+
|    1 | PRIMARY            | articles         | ALL  | NULL                                                                     | NULL                                | NULL    | NULL            | 4846 |   100.00 | Using where; Using filesort        |
|    2 | DEPENDENT SUBQUERY | categories       | ref  | PRIMARY,categories_link_index                                            | categories_link_index               | 767     | const           |    1 |   100.00 | Using index condition; Using where |
|    2 | DEPENDENT SUBQUERY | article_category | ref  | article_category_category_id_foreign,article_category_article_id_foreign | article_category_article_id_foreign | 4       | lcf.articles.id |    1 |   100.00 | Using where                        |
+------+--------------------+------------------+------+--------------------------------------------------------------------------+-------------------------------------+---------+-----------------+------+----------+------------------------------------+

在 Laravel 5.8 上，它会生成以下运行 10-13 秒的查询。

SELECT *
FROM `articles`
WHERE EXISTS
    (SELECT *
     FROM `categories`
     INNER JOIN `article_category` 
       ON `categories`.`id` = `article_category`.`category_id`
     WHERE `articles`.`id` = `article_category`.`article_id`
       AND `link` = 'foto'
       AND `categories`.`deleted_at` IS NULL)
  AND `articles`.`deleted_at` IS NULL
ORDER BY IFNULL(published_at, created_at) DESC
LIMIT 3

这里是解释

+------+--------------+------------------+------+--------------------------------------------------------------------------+--------------------------------------+---------+-------------------+------+----------+------------------------------------+
| id   | select_type  | table            | type | possible_keys                                                            | key                                  | key_len | ref               | rows | filtered | Extra                              |
+------+--------------+------------------+------+--------------------------------------------------------------------------+--------------------------------------+---------+-------------------+------+----------+------------------------------------+
|    1 | PRIMARY      | <subquery2>      | ALL  | distinct_key                                                             | NULL                                 | NULL    | NULL              |  107 |   100.00 | Using temporary; Using filesort    |
|    1 | PRIMARY      | articles         | ALL  | PRIMARY                                                                  | NULL                                 | NULL    | NULL              | 4846 |    75.01 | Using where                        |
|    2 | MATERIALIZED | categories       | ref  | PRIMARY,categories_link_index                                            | categories_link_index                | 767     | const             |    1 |   100.00 | Using index condition; Using where |
|    2 | MATERIALIZED | article_category | ref  | article_category_category_id_foreign,article_category_article_id_foreign | article_category_category_id_foreign | 4       | lcf.categories.id |  107 |   100.00 |                                    |
+------+--------------+------------------+------+--------------------------------------------------------------------------+--------------------------------------+---------+-------------------+------+----------+------------------------------------+

我在同一台服务器上运行两个代码库，同一 MariaDB 10.2.24 数据库。数据集大小约为 6k 篇文章、80 个类别和 10k 条记录。

我应该在这里做什么？到目前为止，我在代码库中发现了超过 10 个遇到此问题的查询。我可以以某种方式翻转配置中的开关并让它们都使用旧方式检查存在吗？还是我应该以某种方式指导每个查询改进他们的计划？

更新

我刚刚注意到，如果我使用whereHas(..., '>', 0)，我几乎可以获得旧查询（实际上是WHERE (SELECT COUNT...) > 0）和旧性能。但是，whereHas(..., '>=', 1) 确实将自身简化为使用EXISTS 进行查询。我是否可以在不编辑每个查询的情况下在整个应用程序上切换此行为。

对评论的回答

文章索引

+----------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table    | Non_unique | Key_name                   | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| articles |          0 | PRIMARY                    |            1 | id          | A         |        4846 |     NULL | NULL   |      | BTREE      |         |               |
| articles |          1 | articles_author_id_foreign |            1 | author_id   | A         |          18 |     NULL | NULL   | YES  | BTREE      |         |               |
+----------+------------+----------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

article_category上的索引

+------------------+------------+--------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table            | Non_unique | Key_name                             | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------------+------------+--------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| article_category |          0 | PRIMARY                              |            1 | id          | A         |        9676 |     NULL | NULL   |      | BTREE      |         |               |
| article_category |          1 | article_category_category_id_foreign |            1 | category_id | A         |          90 |     NULL | NULL   |      | BTREE      |         |               |
| article_category |          1 | article_category_article_id_foreign  |            1 | article_id  | A         |        9676 |     NULL | NULL   |      | BTREE      |         |               |
+------------------+------------+--------------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

运行示例的数据可以在这里找到：https://gist.github.com/tontonsb/b97bc33066a67e9d8bc3654f2c01c103

这运行得更快，但仍然是 2.8 对 0.07 秒，因此可以清楚地看到问题，至少在 MariaDB 10.2.24 上是这样。可能速度有所提高，因为我删除了其他列及其索引。

【问题讨论】：

通常，Query builder 比 eloquent 稍微快一点，所以你可以尝试执行原始查询。

你能在articles 和article_category 表上显示索引吗？会有帮助的。

这里是关于 whereHas github.com/laravel/framework/issues/18415的问题

@ZeshanKhattak 我将它们添加到答案中

@SagarGautam 这个问题是关于where (select count(*) ...) >= ... 的性能比加入差。就我而言，我对这种表现很满意，但我在 5.8 中得到了where exists (select * ...)。

【参考方案1】：

试试这个：

mpyw/eloquent-has-by-non-dependent-subquery: Convert has() and whereHas() constraints to non-dependent subqueries.

$articles = Article::query()
    ->hasByNonDependentSubquery('categories', function ($category) 
        $category->where('link', 'foto');
    )
    ->active()
    ->recent()
    ->take(3)
    ->get();