优化地理搜索查询

Posted

技术标签:

【中文标题】优化地理搜索查询【英文标题】:Optimizing Geo Search Query 【发布时间】:2017-06-17 23:30:22 【问题描述】:

我目前将用户位置存储在 user_geo_places 表中。 我存储了他们所有的最新位置,包括他们过去的位置。

我需要的共同任务,

仅按类别查询最新位置 查询指定位置附近的用户 按距离排序结果 根据指定位置获取每个结果的距离

桌子

+------------+---------------+------+-----+---------------------+-------+
| Field      | Type          | Null | Key | Default             | Extra |
+------------+---------------+------+-----+---------------------+-------+
| id         | varchar(36)   | NO   |     | NULL                |       |
| user_id    | varchar(36)   | NO   | MUL | NULL                |       |
| deleted_at | timestamp     | YES  |     | NULL                |       |
| created_at | timestamp     | NO   |     | 0000-00-00 00:00:00 |       |
| updated_at | timestamp     | NO   | MUL | 0000-00-00 00:00:00 |       |
| latitude   | double(25,20) | NO   | MUL | NULL                |       |
| longitude  | double(25,20) | NO   | MUL | NULL                |       |
| category   | varchar(36)   | YES  |     | NULL                |       |
| status     | int(11)       | YES  |     | 1                   |       |
+------------+---------------+------+-----+---------------------+-------+

这是我的索引

+-----------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table           | Non_unique | Key_name   | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| user_geo_places |          1 | latitude   |            1 | latitude    | A         |     1894347 |     NULL | NULL   |      | BTREE      |         |               |
| user_geo_places |          1 | longitude  |            1 | longitude   | A         |     1894347 |     NULL | NULL   |      | BTREE      |         |               |
| user_geo_places |          1 | updated_at |            1 | updated_at  | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| user_geo_places |          1 | user_id    |            1 | user_id     | A         |     1894347 |     NULL | NULL   |      | BTREE      |         |               |
+-----------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

获取每个用户的最新位置

Select  user_geo_places.*
    from  user_geo_places
    left join  user_geo_places b
               ON (user_geo_places.user_id = b.user_id
              and  user_geo_places.created_at < b.created_at )
    where  b.created_at is NULL
      and  user_geo_places.category = 'plcs'

结果:

+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+
| id                                   | user_id                              | deleted_at | created_at          | updated_at          | latitude                | longitude                | category | status  |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+
| 00019a37-e790-11e6-8469-5404a66ff99a | e20e7777-e788-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.48057242434525200000 | -74.44768883329601000000 | plcs     |       1 |
| 0006162a-e790-11e6-8469-5404a66ff99a | e20e7aef-e772-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.79672692417793640000 | -74.64056815173530000000 | plcs     |       1 |
| 000617f7-e790-11e6-8469-5404a66ff99a | e20ec3c5-e775-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.81266001687632900000 | -74.14159565990940000000 | plcs     |       1 |
| 00061914-e790-11e6-8469-5404a66ff99a | e20edec3-e785-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.26999827965162600000 | -74.92520444926359000000 | plcs     |       1 |
| 00061a1d-e790-11e6-8469-5404a66ff99a | e20eefec-e780-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.81602743809672800000 | -74.30452387342650000000 | plcs     |       1 |
| 00061b21-e790-11e6-8469-5404a66ff99a | e20f3219-e766-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.07453708357600200000 | -74.45911382833413000000 | plcs     |       1 |
| 00061c1e-e790-11e6-8469-5404a66ff99a | e20f7922-e786-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.07195792167629800000 | -74.98244815411275000000 | plcs     |       1 |
| 00061d0e-e790-11e6-8469-5404a66ff99a | e20fd68b-e77e-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.69636703626845700000 | -74.53449074973770000000 | plcs     |       1 |
| 00061e01-e790-11e6-8469-5404a66ff99a | e20fe033-e76b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.58335267061679900000 | -74.31329113460453000000 | plcs     |       1 |
| 00061ef4-e790-11e6-8469-5404a66ff99a | e2101c1b-e776-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.81639769190586900000 | -74.14211508644942000000 | plcs     |       1 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+
10 rows in set (0.00 sec)

解释:

+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-------------------------+
| id   | select_type | table           | type | possible_keys | key     | key_len | ref                                 | rows    | Extra                   |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-------------------------+
|    1 | SIMPLE      | user_geo_places | ALL  | NULL          | NULL    | NULL    | NULL                                | 1894347 | Using where             |
|    1 | SIMPLE      | b               | ref  | user_id       | user_id | 110     | user_geo_places.user_id             |       1 | Using where; Not exists |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-------------------------+
2 rows in set (0.00 sec)

现在当我想通过距离获取用户位置时

Select  user_geo_places.*, (6371 * acos(cos(radians(40.3987545691419)) *
              cos(radians(user_geo_places.latitude)) *
              cos(radians(user_geo_places.longitude)-
              radians(-74.70559604904))+sin(radians(40.3987545691419)) *
              sin(radians(user_geo_places.latitude)))) as distance
    from  user_geo_places
    left join  user_geo_places b
               ON (user_geo_places.user_id = b.user_id
              and  user_geo_places.created_at < b.created_at )
    where  b.created_at is NULL
      and  user_geo_places.category = 'plcs'
    having  distance <= 10
    order by  distance
    limit  10;

结果:

+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+---------------------------+----------+---------+---------------------+
| id                                   | user_id                              | deleted_at | created_at          | updated_at          | latitude                | longitude                 | category | status  | distance            |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+---------------------------+----------+---------+---------------------+
| c4d8e37b-e78f-11e6-8469-5404a66ff99a | 52616262-e78d-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39875468090060600000 | -74.70637827898038000000 | plcs     |       1 | 0.08625581062811027 |
| f4457454-e78f-11e6-8469-5404a66ff99a | cff6b247-e76e-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39847620893127900000 | -74.70646391688517000000 | plcs     |       1 | 0.10058008395499662 |
| bc0dbef2-e78f-11e6-8469-5404a66ff99a | 29ea9f29-e76a-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39796618688662200000 | -74.70656721545996000000 | plcs     |       1 | 0.13839511964323015 |
| c5949373-e78f-11e6-8469-5404a66ff99a | 53e7b25c-e778-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39938782472106400000 | -74.70438668849356000000 | plcs     |       1 | 0.15080387665201841 |
| c9771f58-e78f-11e6-8469-5404a66ff99a | 647afd7e-e76b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40025693587982800000 | -74.70498255799058000000 | plcs     |       1 | 0.18023303176011723 |
| d1d0d5fe-e78f-11e6-8469-5404a66ff99a | 83e6d50e-e78c-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39976791050263500000 | -74.70698894253651000000 | plcs     |       1 | 0.19049204594530353 |
| c8592196-e78f-11e6-8469-5404a66ff99a | 5ef475ae-e789-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39691713303040500000 | -74.70603059763650000000 | plcs     |       1 | 0.20985736972660576 |
| b855003d-e78f-11e6-8469-5404a66ff99a | 18f63baa-e766-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40058719124699700000 | -74.70495101688891000000 | plcs     |       1 | 0.21583472659797534 |
| 0247ffb0-e790-11e6-8469-5404a66ff99a | e534712d-e77b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40040703062006000000 | -74.70668734489631000000 | plcs     |       1 | 0.21964310635910547 |
| ef647fab-e78f-11e6-8469-5404a66ff99a | c981f563-e765-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40036358721588200000 | -74.70425795922452000000 | plcs     |       1 |  0.2319081044464142 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+---------------------+
10 rows in set (14.90 sec)

该死的。 15 秒?这对于网络查询来说非常慢。

解释:

+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-----------------------------+
| id   | select_type | table           | type | possible_keys | key     | key_len | ref                                 | rows    | Extra                       |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-----------------------------+
|    1 | SIMPLE      | user_geo_places | ALL  | NULL          | NULL    | NULL    | NULL                                | 1894347 | Using where; Using filesort |
|    1 | SIMPLE      | b               | ref  | user_id       | user_id | 110     | user_geo_places.user_id             |       1 | Using where; Not exists     |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-----------------------------+
2 rows in set (0.00 sec)

我不想扫描整个数据库。我只需要10公里附近的用户。

Select  user_geo_places.*, (6371 * acos(cos(radians(40.3987545691419)) *
         cos(radians(user_geo_places.latitude)) *
         cos(radians(user_geo_places.longitude)-
         radians(-74.70559604904))+sin(radians(40.3987545691419)) *
         sin(radians(user_geo_places.latitude)))) as distance
    from  user_geo_places
    left join  user_geo_places b  ON (user_geo_places.user_id = b.user_id
              and  user_geo_places.created_at < b.created_at
                          )
    where  b.created_at is NULL
      and  user_geo_places.category = 'plcs'
      and  user_geo_places.longitude
         between -74.70559604904 - 10 / abs(cos(radians( 40.3987545691419 )) * 111.045)
             AND -74.70559604904 + 10 / abs(cos(radians( 40.3987545691419 )) * 111.045)
      and  user_geo_places.latitude
         between 40.3987545691419 - ( 10 / 111.045 )
             AND 40.3987545691419 + ( 10 / 111.045 )
    having  distance <= 10
    order by  distance
    limit  10;

结果:

+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| id                                   | user_id                              | deleted_at | created_at          | updated_at          | latitude               | longitude                | category | status  | distance            |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| c4d8e37b-e78f-11e6-8469-5404a66ff99a | 52616262-e78d-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39875468090060600000 | -74.70637827898038000000 | plcs     |       1 | 0.08625581062811027 |
| f4457454-e78f-11e6-8469-5404a66ff99a | cff6b247-e76e-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39847620893127900000 | -74.70646391688517000000 | plcs     |       1 | 0.10058008395499662 |
| bc0dbef2-e78f-11e6-8469-5404a66ff99a | 29ea9f29-e76a-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39796618688662200000 | -74.70656721545996000000 | plcs     |       1 | 0.13839511964323015 |
| c5949373-e78f-11e6-8469-5404a66ff99a | 53e7b25c-e778-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39938782472106400000 | -74.70438668849356000000 | plcs     |       1 | 0.15080387665201841 |
| c9771f58-e78f-11e6-8469-5404a66ff99a | 647afd7e-e76b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40025693587982800000 | -74.70498255799058000000 | plcs     |       1 | 0.18023303176011723 |
| d1d0d5fe-e78f-11e6-8469-5404a66ff99a | 83e6d50e-e78c-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39976791050263500000 | -74.70698894253651000000 | plcs     |       1 | 0.19049204594530353 |
| c8592196-e78f-11e6-8469-5404a66ff99a | 5ef475ae-e789-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39691713303040500000 | -74.70603059763650000000 | plcs     |       1 | 0.20985736972660576 |
| b855003d-e78f-11e6-8469-5404a66ff99a | 18f63baa-e766-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40058719124699700000 | -74.70495101688891000000 | plcs     |       1 | 0.21583472659797534 |
| 0247ffb0-e790-11e6-8469-5404a66ff99a | e534712d-e77b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40040703062006000000 | -74.70668734489631000000 | plcs     |       1 | 0.21964310635910547 |
| ef647fab-e78f-11e6-8469-5404a66ff99a | c981f563-e765-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40036358721588200000 | -74.70425795922452000000 | plcs     |       1 |  0.2319081044464142 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
10 rows in set (2 min 53.63 sec)

哦,来吧。真的吗? 2分53.63秒?太可怕了。

解释:

+------+-------------+-----------------+-------+--------------------+----------+---------+-------------------------------------+--------+----------------------------------------------------+
| id   | select_type | table           | type  | possible_keys      | key      | key_len | ref                                 | rows   | Extra                                              |
+------+-------------+-----------------+-------+--------------------+----------+---------+-------------------------------------+--------+----------------------------------------------------+
|    1 | SIMPLE      | user_geo_places | range | latitude,longitude | latitude | 8       | NULL                                | 683684 | Using index condition; Using where; Using filesort |
|    1 | SIMPLE      | b               | ref   | user_id            | user_id  | 110     | user_geo_places.user_id             |      1 | Using where; Not exists                            |
+------+-------------+-----------------+-------+--------------------+----------+---------+-------------------------------------+--------+----------------------------------------------------+
2 rows in set (0.01 sec)

也许是因为计算。让我们移动其中一些并设置为 变量。

Set @lon1= -74.70559604904 - 10 / abs(cos(radians( 40.3987545691419 )) * 111.045) ;
Set @lon2= -74.70559604904 + 10 / abs(cos(radians( 40.3987545691419 )) * 111.045) ;
Set @lat1= 40.3987545691419 - ( 10 / 111.045 );
Set @lat2= 40.3987545691419 + ( 10 / 111.045 );
Select
user_geo_places.*,
(6371 * acos(cos(radians(40.3987545691419)) * cos(radians(user_geo_places.latitude)) * cos(radians(user_geo_places.longitude)-radians(-74.70559604904))+sin(radians(40.3987545691419)) * sin(radians(user_geo_places.latitude)))) as distance
from user_geo_places
left join user_geo_places b
on (user_geo_places.user_id = b.user_id and user_geo_places.created_at < b.created_at)
    where  b.created_at is NULL
      and  user_geo_places.category = 'plcs'
      and  user_geo_places.longitude between @lon1 AND @lon2
      and  user_geo_places.latitude  between @lat1 AND @lat2
    having  distance <= 10
    order by  distance
    limit  10;

结果:

+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| id                                   | user_id                              | deleted_at | created_at          | updated_at          | latitude               | longitude                | category | status  | distance            |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| c4d8e37b-e78f-11e6-8469-5404a66ff99a | 52616262-e78d-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39875468090060600000 | -74.70637827898038000000 | plcs     |       1 | 0.08625581062811027 |
| f4457454-e78f-11e6-8469-5404a66ff99a | cff6b247-e76e-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39847620893127900000 | -74.70646391688517000000 | plcs     |       1 | 0.10058008395499662 |
| bc0dbef2-e78f-11e6-8469-5404a66ff99a | 29ea9f29-e76a-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39796618688662200000 | -74.70656721545996000000 | plcs     |       1 | 0.13839511964323015 |
| c5949373-e78f-11e6-8469-5404a66ff99a | 53e7b25c-e778-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39938782472106400000 | -74.70438668849356000000 | plcs     |       1 | 0.15080387665201841 |
| c9771f58-e78f-11e6-8469-5404a66ff99a | 647afd7e-e76b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40025693587982800000 | -74.70498255799058000000 | plcs     |       1 | 0.18023303176011723 |
| d1d0d5fe-e78f-11e6-8469-5404a66ff99a | 83e6d50e-e78c-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39976791050263500000 | -74.70698894253651000000 | plcs     |       1 | 0.19049204594530353 |
| c8592196-e78f-11e6-8469-5404a66ff99a | 5ef475ae-e789-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39691713303040500000 | -74.70603059763650000000 | plcs     |       1 | 0.20985736972660576 |
| b855003d-e78f-11e6-8469-5404a66ff99a | 18f63baa-e766-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40058719124699700000 | -74.70495101688891000000 | plcs     |       1 | 0.21583472659797534 |
| 0247ffb0-e790-11e6-8469-5404a66ff99a | e534712d-e77b-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40040703062006000000 | -74.70668734489631000000 | plcs     |       1 | 0.21964310635910547 |
| ef647fab-e78f-11e6-8469-5404a66ff99a | c981f563-e765-11e6-8469-5404a66ff99a | NULL       | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40036358721588200000 | -74.70425795922452000000 | plcs     |       1 |  0.2319081044464142 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
10 rows in set (2 min 56.29 sec)

男人。你真的很慢。我不知道该怎么办了。是时候问问专家了。

目前。我的表中存储了 180 万个虚拟数据。我的查询仍然非常非常慢。此表按类别存储位置,因此该位置还有其他类别。有什么好的方法来解决和改进这个问题吗?

如果我不能解决这个问题。我计划按每个表的类别移动所有数据。如果由于他们过去的位置,事情仍然会变慢。我可能会删除该功能,但我真的很想实现这一点。希望有人可以帮助我

编辑 1:

Show Create Table :

+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table           | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| user_geo_places | CREATE TABLE `user_geo_places` (
  `id` varchar(36) NOT NULL,
  `user_id` varchar(36) NOT NULL,
  `deleted_at` timestamp NULL DEFAULT NULL,
  `created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  `updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  `latitude` double(25,20) NOT NULL,
  `longitude` double(25,20) NOT NULL,
  `category` varchar(36) DEFAULT NULL,
  `status` int(11) DEFAULT '1',
  KEY `latitude` (`latitude`),
  KEY `longitude` (`longitude`),
  KEY `updated_at` (`updated_at`),
  KEY `user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+



+--------------------------------+----------------------+
| Variable_name                  | Value                |
+--------------------------------+----------------------+
| aria_pagecache_age_threshold   | 300                  |
| aria_pagecache_buffer_size     | 134217728            |
| aria_pagecache_division_limit  | 100                  |
| aria_pagecache_file_hash_size  | 512                  |
| binlog_cache_size              | 32768                |
| binlog_stmt_cache_size         | 32768                |
| have_query_cache               | YES                  |
| host_cache_size                | 279                  |
| innodb_disable_sort_file_cache | OFF                  |
| innodb_ft_cache_size           | 8000000              |
| innodb_ft_result_cache_limit   | 2000000000           |
| innodb_ft_total_cache_size     | 640000000            |
| join_cache_level               | 2                    |
| key_cache_age_threshold        | 300                  |
| key_cache_block_size           | 1024                 |
| key_cache_division_limit       | 100                  |
| key_cache_file_hash_size       | 512                  |
| key_cache_segments             | 0                    |
| max_binlog_cache_size          | 18446744073709547520 |
| max_binlog_stmt_cache_size     | 18446744073709547520 |
| metadata_locks_cache_size      | 1024                 |
| query_cache_limit              | 1048576              |
| query_cache_min_res_unit       | 4096                 |
| query_cache_size               | 1048576              |
| query_cache_strip_comments     | OFF                  |
| query_cache_type               | OFF                  |
| query_cache_wlock_invalidate   | OFF                  |
| stored_program_cache           | 256                  |
| table_definition_cache         | 400                  |
| table_open_cache               | 431                  |
| thread_cache_size              | 0                    |
+--------------------------------+----------------------+

服务器规格

4 核

8 GB 内存

MariaDB 10.1.20

使用的引擎:InnoDB

总行数:1,894,326

innodb_buffer_pool_size : 134,217,728

目前 98% 的记录都在纽约

运行的 SQL_NO_CACHE

Select SQL_NO_CACHE
    user_geo_places.*
from user_geo_places
left join user_geo_places b
on (user_geo_places.user_id = b.user_id and user_geo_places.created_at < b.created_at)
where b.created_at is NULL and user_geo_places.category = 'plcs'
limit 10

10 rows in set (0.00 - 0.07 sec)

顺便说一下,列状态没用,会被drop

【问题讨论】:

你的mysql版本是多少? 表格有多少行? UUIDslat/lng 很难优化。阅读这些链接,看看其中的一些提示是否会有所帮助。然后,我会考虑深入挖掘。 (0.00 sec) 可能是假的 -- 查询缓存是否打开?使用 SELECT SQL_NO_CACHE ... 运行计时测试 三分之一的桌子在纽约,难怪20公里不够紧。 【参考方案1】:

缩小表格会有帮助一些

double(25,20) 占用 12 个字节,精度约为分子宽度。 UUIDs 可以打包成 BINARY(16)(16 字节)而不是 VARCHAR(36)(37 字节)。 将category 标准化为2 字节SMALLINT UNSIGNED 可能很有用。 INT 始终为 4 个字节;对 status 等标志使用 1 字节的 TINYINT

没有SHOW CREATE TABLE,我们就看不到索引,尤其是你有单列索引还是复合索引。

最新分类可能受益于INDEX(category, created_at) “附近”和“按距离排序”特别困难;见here

将“当前状态”和“历史”分成两个表通常很有用。前者每个实体只有一行;后者(可能)有很多。这大大简化了涉及“最新”的查询。这也可以让您摆脱status,方法是从“当前状态”中删除一个死条目,同时将其留在“历史”中。过滤status可能(目前)是查询中的另一个性能杀手。

看起来UUIDs 是“类型 1”;除了缩小它们之外,它们的位可以重新排列以使它们按时间顺序排列。他们将通过改进“参考位置”来提高一些查询的性能。见here

该表使用什么引擎?该引擎的缓存有多大?你有多少内存?

LEFT JOIN 需要INDEX(user_id, created_at)。 (当然,如果你把历史分开,这将不再相关。)

表格的百分之几是“当前状态”?

我提出了几个建议;很难说哪个是最有益的。

为什么用边界框跑起来这么慢?这似乎是多种事物的结合:

纽约市 20 公里涵盖了很多领域。也许是桌子的 1/3?? 在决定使用索引还是简单地扫描表时,优化器并不总是“正确”。 这次,后者会更快。 可能是一些缓存问题。

【讨论】:

嗨,里克,感谢您抽出宝贵时间。我已经更新了我的问题。看起来我有很多错误。我一直在存储狗中每个跳蚤的位置,我将其更改为 DECIMAL(6,4) 并将使用自动增量作为 id。 顺便说一句。如何确定是否存在任何缓存问题? Cache...多少RAM,哪个引擎,表有多大,查询缓存在吗?

以上是关于优化地理搜索查询的主要内容,如果未能解决你的问题,请参考以下文章

地理数据搜索的优化

优化表格以搜索地理位置

RethinkDB - 优化执行地理定位、然后加入、然后分组的查询

Elasticsearch 基于地理位置的搜索查询

Elasticsearch 基于地理位置的搜索查询

Elasticsearch 基于地理位置的搜索查询