Rails MySQL 查询时间混乱

Posted 2023-04-15

【中文标题】Rails MySQL 查询时间混乱

【英文标题】：Rails MySQL query time confusion

【发布时间】：2010-01-29 11:16:59

【问题描述】：

我有一个计数计算查询，我在我的 Rails 应用程序中运行了数千次，数据库中的每个客户一次。

当我在禁用查询缓存的 mysql 客户端中运行查询时，查询持续时间超过 1 毫秒。

但是，当我在启用查询输出的情况下从 Rails 控制台运行任务时，我注意到在前几个非常快的查询之后，其余查询的时间突然从不到 1 毫秒增加到大约 180 毫秒.

我已经减小了 innodb_buffer_pool_size 以查看行为的变化，但没有注意到任何事情。

这是控制台的输出：

  EmailCampaignReport::Open Columns (143.2ms)   SHOW FIELDS FROM `email_campaign_report_opens`
  SQL (0.3ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332330) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 333333) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332661) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332326) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332665) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 336027) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 333001) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 331983) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332668) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332316) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332325) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 331995) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 334007) 
  SQL (0.2ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 333326) 
  SQL (0.1ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332998) 
  SQL (183.9ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 334673) 
  SQL (183.7ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 336751) 
  SQL (183.6ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 333334) 
  SQL (186.3ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332663) 
  SQL (183.7ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332328) 
  SQL (186.3ms)   SELECT count(*) AS count_all FROM `email_campaign_report_opens` WHERE (customer_id = 332659) 

在该表的 customer_id 列上有一个索引。

有人对为什么会发生这种情况有任何建议吗？

谢谢

【问题讨论】：

您可以发布您的表架构以便我们查看索引等吗？

【参考方案1】：

为什么不只运行一个查询？

SELECT customer_id, count(*) AS count_all FROM `email_campaign_report_opens` GROUP BY customer_id;

如果您有太多记录而担心将它们全部退回，那么请分批进行，但我只是不明白您为什么真的要为每个客户运行此查询。

【讨论】：

很高兴为您工作。在 ActiveRecord 中很容易产生使用对象和迭代器做所有事情的心态，但有时最好使用一点 SQL。

【参考方案2】：

这是否也发生在您的 rails 应用程序中，还是仅在您在控制台中运行时发生？另外，您是在使用 Aptana 之类的客户端，还是在 shell 中运行它？

【讨论】：

这是在 shell 中运行的，它也确实发生在 Rails 应用程序中。

【参考方案3】：

这是什么版本的 Rails？ 根据您的版本和您的 Ruby/Rails 代码，您可能会缓存大量数据而不使用它，并且过了一段时间它必须在获取新数据之前进行垃圾收集，这可能解释了延迟。 请注意，这是一个猜测。

【参考方案4】：

将counter cache 添加到关联是否有意义（阅读：将email_campaign_report_opens_count 添加到您的Customer 模型）？当然，您必须在迁移期间初始化计数器，但它应该非常快，您甚至不需要在遍历客户表时触摸关联的表。

【讨论】：

好建议。这几乎就是我正在做的事情，我在客户模型中有一个类似的计数字段，该脚本正在通过获取所有计数进行更新。在创建关联时更新计数器缓存可能会更好。在初始迁移期间仍然会遇到查询速度问题，但至少是一次。