优化地理搜索查询
Posted
技术标签:
【中文标题】优化地理搜索查询【英文标题】:Optimizing Geo Search Query 【发布时间】:2017-06-17 23:30:22 【问题描述】:我目前将用户位置存储在 user_geo_places 表中。 我存储了他们所有的最新位置,包括他们过去的位置。
我需要的共同任务,
仅按类别查询最新位置 查询指定位置附近的用户 按距离排序结果 根据指定位置获取每个结果的距离桌子
+------------+---------------+------+-----+---------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+---------------+------+-----+---------------------+-------+
| id | varchar(36) | NO | | NULL | |
| user_id | varchar(36) | NO | MUL | NULL | |
| deleted_at | timestamp | YES | | NULL | |
| created_at | timestamp | NO | | 0000-00-00 00:00:00 | |
| updated_at | timestamp | NO | MUL | 0000-00-00 00:00:00 | |
| latitude | double(25,20) | NO | MUL | NULL | |
| longitude | double(25,20) | NO | MUL | NULL | |
| category | varchar(36) | YES | | NULL | |
| status | int(11) | YES | | 1 | |
+------------+---------------+------+-----+---------------------+-------+
这是我的索引
+-----------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| user_geo_places | 1 | latitude | 1 | latitude | A | 1894347 | NULL | NULL | | BTREE | | |
| user_geo_places | 1 | longitude | 1 | longitude | A | 1894347 | NULL | NULL | | BTREE | | |
| user_geo_places | 1 | updated_at | 1 | updated_at | A | 18 | NULL | NULL | | BTREE | | |
| user_geo_places | 1 | user_id | 1 | user_id | A | 1894347 | NULL | NULL | | BTREE | | |
+-----------------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
获取每个用户的最新位置
Select user_geo_places.*
from user_geo_places
left join user_geo_places b
ON (user_geo_places.user_id = b.user_id
and user_geo_places.created_at < b.created_at )
where b.created_at is NULL
and user_geo_places.category = 'plcs'
结果:
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+
| id | user_id | deleted_at | created_at | updated_at | latitude | longitude | category | status |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+
| 00019a37-e790-11e6-8469-5404a66ff99a | e20e7777-e788-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.48057242434525200000 | -74.44768883329601000000 | plcs | 1 |
| 0006162a-e790-11e6-8469-5404a66ff99a | e20e7aef-e772-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.79672692417793640000 | -74.64056815173530000000 | plcs | 1 |
| 000617f7-e790-11e6-8469-5404a66ff99a | e20ec3c5-e775-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.81266001687632900000 | -74.14159565990940000000 | plcs | 1 |
| 00061914-e790-11e6-8469-5404a66ff99a | e20edec3-e785-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.26999827965162600000 | -74.92520444926359000000 | plcs | 1 |
| 00061a1d-e790-11e6-8469-5404a66ff99a | e20eefec-e780-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.81602743809672800000 | -74.30452387342650000000 | plcs | 1 |
| 00061b21-e790-11e6-8469-5404a66ff99a | e20f3219-e766-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.07453708357600200000 | -74.45911382833413000000 | plcs | 1 |
| 00061c1e-e790-11e6-8469-5404a66ff99a | e20f7922-e786-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.07195792167629800000 | -74.98244815411275000000 | plcs | 1 |
| 00061d0e-e790-11e6-8469-5404a66ff99a | e20fd68b-e77e-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.69636703626845700000 | -74.53449074973770000000 | plcs | 1 |
| 00061e01-e790-11e6-8469-5404a66ff99a | e20fe033-e76b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.58335267061679900000 | -74.31329113460453000000 | plcs | 1 |
| 00061ef4-e790-11e6-8469-5404a66ff99a | e2101c1b-e776-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.81639769190586900000 | -74.14211508644942000000 | plcs | 1 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+
10 rows in set (0.00 sec)
解释:
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-------------------------+
| 1 | SIMPLE | user_geo_places | ALL | NULL | NULL | NULL | NULL | 1894347 | Using where |
| 1 | SIMPLE | b | ref | user_id | user_id | 110 | user_geo_places.user_id | 1 | Using where; Not exists |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-------------------------+
2 rows in set (0.00 sec)
现在当我想通过距离获取用户位置时
Select user_geo_places.*, (6371 * acos(cos(radians(40.3987545691419)) *
cos(radians(user_geo_places.latitude)) *
cos(radians(user_geo_places.longitude)-
radians(-74.70559604904))+sin(radians(40.3987545691419)) *
sin(radians(user_geo_places.latitude)))) as distance
from user_geo_places
left join user_geo_places b
ON (user_geo_places.user_id = b.user_id
and user_geo_places.created_at < b.created_at )
where b.created_at is NULL
and user_geo_places.category = 'plcs'
having distance <= 10
order by distance
limit 10;
结果:
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+---------------------------+----------+---------+---------------------+
| id | user_id | deleted_at | created_at | updated_at | latitude | longitude | category | status | distance |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+---------------------------+----------+---------+---------------------+
| c4d8e37b-e78f-11e6-8469-5404a66ff99a | 52616262-e78d-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39875468090060600000 | -74.70637827898038000000 | plcs | 1 | 0.08625581062811027 |
| f4457454-e78f-11e6-8469-5404a66ff99a | cff6b247-e76e-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39847620893127900000 | -74.70646391688517000000 | plcs | 1 | 0.10058008395499662 |
| bc0dbef2-e78f-11e6-8469-5404a66ff99a | 29ea9f29-e76a-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39796618688662200000 | -74.70656721545996000000 | plcs | 1 | 0.13839511964323015 |
| c5949373-e78f-11e6-8469-5404a66ff99a | 53e7b25c-e778-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39938782472106400000 | -74.70438668849356000000 | plcs | 1 | 0.15080387665201841 |
| c9771f58-e78f-11e6-8469-5404a66ff99a | 647afd7e-e76b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40025693587982800000 | -74.70498255799058000000 | plcs | 1 | 0.18023303176011723 |
| d1d0d5fe-e78f-11e6-8469-5404a66ff99a | 83e6d50e-e78c-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39976791050263500000 | -74.70698894253651000000 | plcs | 1 | 0.19049204594530353 |
| c8592196-e78f-11e6-8469-5404a66ff99a | 5ef475ae-e789-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39691713303040500000 | -74.70603059763650000000 | plcs | 1 | 0.20985736972660576 |
| b855003d-e78f-11e6-8469-5404a66ff99a | 18f63baa-e766-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40058719124699700000 | -74.70495101688891000000 | plcs | 1 | 0.21583472659797534 |
| 0247ffb0-e790-11e6-8469-5404a66ff99a | e534712d-e77b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40040703062006000000 | -74.70668734489631000000 | plcs | 1 | 0.21964310635910547 |
| ef647fab-e78f-11e6-8469-5404a66ff99a | c981f563-e765-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40036358721588200000 | -74.70425795922452000000 | plcs | 1 | 0.2319081044464142 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+-------------------------+--------------------------+----------+---------+---------------------+
10 rows in set (14.90 sec)
该死的。 15 秒?这对于网络查询来说非常慢。
解释:
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-----------------------------+
| 1 | SIMPLE | user_geo_places | ALL | NULL | NULL | NULL | NULL | 1894347 | Using where; Using filesort |
| 1 | SIMPLE | b | ref | user_id | user_id | 110 | user_geo_places.user_id | 1 | Using where; Not exists |
+------+-------------+-----------------+------+---------------+---------+---------+-------------------------------------+---------+-----------------------------+
2 rows in set (0.00 sec)
我不想扫描整个数据库。我只需要10公里附近的用户。
Select user_geo_places.*, (6371 * acos(cos(radians(40.3987545691419)) *
cos(radians(user_geo_places.latitude)) *
cos(radians(user_geo_places.longitude)-
radians(-74.70559604904))+sin(radians(40.3987545691419)) *
sin(radians(user_geo_places.latitude)))) as distance
from user_geo_places
left join user_geo_places b ON (user_geo_places.user_id = b.user_id
and user_geo_places.created_at < b.created_at
)
where b.created_at is NULL
and user_geo_places.category = 'plcs'
and user_geo_places.longitude
between -74.70559604904 - 10 / abs(cos(radians( 40.3987545691419 )) * 111.045)
AND -74.70559604904 + 10 / abs(cos(radians( 40.3987545691419 )) * 111.045)
and user_geo_places.latitude
between 40.3987545691419 - ( 10 / 111.045 )
AND 40.3987545691419 + ( 10 / 111.045 )
having distance <= 10
order by distance
limit 10;
结果:
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| id | user_id | deleted_at | created_at | updated_at | latitude | longitude | category | status | distance |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| c4d8e37b-e78f-11e6-8469-5404a66ff99a | 52616262-e78d-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39875468090060600000 | -74.70637827898038000000 | plcs | 1 | 0.08625581062811027 |
| f4457454-e78f-11e6-8469-5404a66ff99a | cff6b247-e76e-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39847620893127900000 | -74.70646391688517000000 | plcs | 1 | 0.10058008395499662 |
| bc0dbef2-e78f-11e6-8469-5404a66ff99a | 29ea9f29-e76a-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39796618688662200000 | -74.70656721545996000000 | plcs | 1 | 0.13839511964323015 |
| c5949373-e78f-11e6-8469-5404a66ff99a | 53e7b25c-e778-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39938782472106400000 | -74.70438668849356000000 | plcs | 1 | 0.15080387665201841 |
| c9771f58-e78f-11e6-8469-5404a66ff99a | 647afd7e-e76b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40025693587982800000 | -74.70498255799058000000 | plcs | 1 | 0.18023303176011723 |
| d1d0d5fe-e78f-11e6-8469-5404a66ff99a | 83e6d50e-e78c-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39976791050263500000 | -74.70698894253651000000 | plcs | 1 | 0.19049204594530353 |
| c8592196-e78f-11e6-8469-5404a66ff99a | 5ef475ae-e789-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39691713303040500000 | -74.70603059763650000000 | plcs | 1 | 0.20985736972660576 |
| b855003d-e78f-11e6-8469-5404a66ff99a | 18f63baa-e766-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40058719124699700000 | -74.70495101688891000000 | plcs | 1 | 0.21583472659797534 |
| 0247ffb0-e790-11e6-8469-5404a66ff99a | e534712d-e77b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40040703062006000000 | -74.70668734489631000000 | plcs | 1 | 0.21964310635910547 |
| ef647fab-e78f-11e6-8469-5404a66ff99a | c981f563-e765-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40036358721588200000 | -74.70425795922452000000 | plcs | 1 | 0.2319081044464142 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
10 rows in set (2 min 53.63 sec)
哦,来吧。真的吗? 2分53.63秒?太可怕了。
解释:
+------+-------------+-----------------+-------+--------------------+----------+---------+-------------------------------------+--------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------+-------+--------------------+----------+---------+-------------------------------------+--------+----------------------------------------------------+
| 1 | SIMPLE | user_geo_places | range | latitude,longitude | latitude | 8 | NULL | 683684 | Using index condition; Using where; Using filesort |
| 1 | SIMPLE | b | ref | user_id | user_id | 110 | user_geo_places.user_id | 1 | Using where; Not exists |
+------+-------------+-----------------+-------+--------------------+----------+---------+-------------------------------------+--------+----------------------------------------------------+
2 rows in set (0.01 sec)
也许是因为计算。让我们移动其中一些并设置为 变量。
Set @lon1= -74.70559604904 - 10 / abs(cos(radians( 40.3987545691419 )) * 111.045) ;
Set @lon2= -74.70559604904 + 10 / abs(cos(radians( 40.3987545691419 )) * 111.045) ;
Set @lat1= 40.3987545691419 - ( 10 / 111.045 );
Set @lat2= 40.3987545691419 + ( 10 / 111.045 );
Select
user_geo_places.*,
(6371 * acos(cos(radians(40.3987545691419)) * cos(radians(user_geo_places.latitude)) * cos(radians(user_geo_places.longitude)-radians(-74.70559604904))+sin(radians(40.3987545691419)) * sin(radians(user_geo_places.latitude)))) as distance
from user_geo_places
left join user_geo_places b
on (user_geo_places.user_id = b.user_id and user_geo_places.created_at < b.created_at)
where b.created_at is NULL
and user_geo_places.category = 'plcs'
and user_geo_places.longitude between @lon1 AND @lon2
and user_geo_places.latitude between @lat1 AND @lat2
having distance <= 10
order by distance
limit 10;
结果:
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| id | user_id | deleted_at | created_at | updated_at | latitude | longitude | category | status | distance |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
| c4d8e37b-e78f-11e6-8469-5404a66ff99a | 52616262-e78d-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39875468090060600000 | -74.70637827898038000000 | plcs | 1 | 0.08625581062811027 |
| f4457454-e78f-11e6-8469-5404a66ff99a | cff6b247-e76e-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39847620893127900000 | -74.70646391688517000000 | plcs | 1 | 0.10058008395499662 |
| bc0dbef2-e78f-11e6-8469-5404a66ff99a | 29ea9f29-e76a-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39796618688662200000 | -74.70656721545996000000 | plcs | 1 | 0.13839511964323015 |
| c5949373-e78f-11e6-8469-5404a66ff99a | 53e7b25c-e778-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39938782472106400000 | -74.70438668849356000000 | plcs | 1 | 0.15080387665201841 |
| c9771f58-e78f-11e6-8469-5404a66ff99a | 647afd7e-e76b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40025693587982800000 | -74.70498255799058000000 | plcs | 1 | 0.18023303176011723 |
| d1d0d5fe-e78f-11e6-8469-5404a66ff99a | 83e6d50e-e78c-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39976791050263500000 | -74.70698894253651000000 | plcs | 1 | 0.19049204594530353 |
| c8592196-e78f-11e6-8469-5404a66ff99a | 5ef475ae-e789-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.39691713303040500000 | -74.70603059763650000000 | plcs | 1 | 0.20985736972660576 |
| b855003d-e78f-11e6-8469-5404a66ff99a | 18f63baa-e766-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40058719124699700000 | -74.70495101688891000000 | plcs | 1 | 0.21583472659797534 |
| 0247ffb0-e790-11e6-8469-5404a66ff99a | e534712d-e77b-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40040703062006000000 | -74.70668734489631000000 | plcs | 1 | 0.21964310635910547 |
| ef647fab-e78f-11e6-8469-5404a66ff99a | c981f563-e765-11e6-8469-5404a66ff99a | NULL | 2017-01-31 16:31:48 | 2017-01-31 16:31:48 | 40.40036358721588200000 | -74.70425795922452000000 | plcs | 1 | 0.2319081044464142 |
+--------------------------------------+--------------------------------------+------------+---------------------+---------------------+------------------------+--------------------------+----------+---------+---------------------+
10 rows in set (2 min 56.29 sec)
男人。你真的很慢。我不知道该怎么办了。是时候问问专家了。
目前。我的表中存储了 180 万个虚拟数据。我的查询仍然非常非常慢。此表按类别存储位置,因此该位置还有其他类别。有什么好的方法来解决和改进这个问题吗?
如果我不能解决这个问题。我计划按每个表的类别移动所有数据。如果由于他们过去的位置,事情仍然会变慢。我可能会删除该功能,但我真的很想实现这一点。希望有人可以帮助我
编辑 1:
Show Create Table :
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| user_geo_places | CREATE TABLE `user_geo_places` (
`id` varchar(36) NOT NULL,
`user_id` varchar(36) NOT NULL,
`deleted_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`latitude` double(25,20) NOT NULL,
`longitude` double(25,20) NOT NULL,
`category` varchar(36) DEFAULT NULL,
`status` int(11) DEFAULT '1',
KEY `latitude` (`latitude`),
KEY `longitude` (`longitude`),
KEY `updated_at` (`updated_at`),
KEY `user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+--------------------------------+----------------------+
| Variable_name | Value |
+--------------------------------+----------------------+
| aria_pagecache_age_threshold | 300 |
| aria_pagecache_buffer_size | 134217728 |
| aria_pagecache_division_limit | 100 |
| aria_pagecache_file_hash_size | 512 |
| binlog_cache_size | 32768 |
| binlog_stmt_cache_size | 32768 |
| have_query_cache | YES |
| host_cache_size | 279 |
| innodb_disable_sort_file_cache | OFF |
| innodb_ft_cache_size | 8000000 |
| innodb_ft_result_cache_limit | 2000000000 |
| innodb_ft_total_cache_size | 640000000 |
| join_cache_level | 2 |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| key_cache_file_hash_size | 512 |
| key_cache_segments | 0 |
| max_binlog_cache_size | 18446744073709547520 |
| max_binlog_stmt_cache_size | 18446744073709547520 |
| metadata_locks_cache_size | 1024 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 1048576 |
| query_cache_strip_comments | OFF |
| query_cache_type | OFF |
| query_cache_wlock_invalidate | OFF |
| stored_program_cache | 256 |
| table_definition_cache | 400 |
| table_open_cache | 431 |
| thread_cache_size | 0 |
+--------------------------------+----------------------+
服务器规格
4 核
8 GB 内存
MariaDB 10.1.20
使用的引擎:InnoDB
总行数:1,894,326
innodb_buffer_pool_size : 134,217,728
目前 98% 的记录都在纽约
运行的 SQL_NO_CACHE
Select SQL_NO_CACHE
user_geo_places.*
from user_geo_places
left join user_geo_places b
on (user_geo_places.user_id = b.user_id and user_geo_places.created_at < b.created_at)
where b.created_at is NULL and user_geo_places.category = 'plcs'
limit 10
10 rows in set (0.00 - 0.07 sec)
顺便说一下,列状态没用,会被drop
【问题讨论】:
你的mysql版本是多少? 表格有多少行? UUIDs 和 lat/lng 很难优化。阅读这些链接,看看其中的一些提示是否会有所帮助。然后,我会考虑深入挖掘。(0.00 sec)
可能是假的 -- 查询缓存是否打开?使用 SELECT SQL_NO_CACHE ...
运行计时测试
三分之一的桌子在纽约,难怪20公里不够紧。
【参考方案1】:
缩小表格会有帮助一些。
double(25,20)
占用 12 个字节,精度约为分子宽度。
UUIDs
可以打包成 BINARY(16)
(16 字节)而不是 VARCHAR(36)
(37 字节)。
将category
标准化为2 字节SMALLINT UNSIGNED
可能很有用。
INT
始终为 4 个字节;对 status
等标志使用 1 字节的 TINYINT
。
没有SHOW CREATE TABLE
,我们就看不到索引,尤其是你有单列索引还是复合索引。
INDEX(category, created_at)
“附近”和“按距离排序”特别困难;见here。
将“当前状态”和“历史”分成两个表通常很有用。前者每个实体只有一行;后者(可能)有很多。这大大简化了涉及“最新”的查询。这也可以让您摆脱status
,方法是从“当前状态”中删除一个死条目,同时将其留在“历史”中。过滤status
可能(目前)是查询中的另一个性能杀手。
看起来UUIDs
是“类型 1”;除了缩小它们之外,它们的位可以重新排列以使它们按时间顺序排列。他们将通过改进“参考位置”来提高一些查询的性能。见here。
该表使用什么引擎?该引擎的缓存有多大?你有多少内存?
LEFT JOIN
需要INDEX(user_id, created_at)
。 (当然,如果你把历史分开,这将不再相关。)
表格的百分之几是“当前状态”?
我提出了几个建议;很难说哪个是最有益的。
为什么用边界框跑起来这么慢?这似乎是多种事物的结合:
纽约市 20 公里涵盖了很多领域。也许是桌子的 1/3?? 在决定使用索引还是简单地扫描表时,优化器并不总是“正确”。 这次,后者会更快。 可能是一些缓存问题。【讨论】:
嗨,里克,感谢您抽出宝贵时间。我已经更新了我的问题。看起来我有很多错误。我一直在存储狗中每个跳蚤的位置,我将其更改为 DECIMAL(6,4) 并将使用自动增量作为 id。 顺便说一句。如何确定是否存在任何缓存问题? Cache...多少RAM,哪个引擎,表有多大,查询缓存在吗?以上是关于优化地理搜索查询的主要内容,如果未能解决你的问题,请参考以下文章