带你完成数据库的clickbench性能测试(小白都能看懂)

Posted 2023-05-09 Syw

clickbench官网链接：https://benchmark.clickhouse.com/

如下采用的数据库为分析型数据库支持MySQL协议，其他所有的数据库操作方法均类似

测试背景

利用clickhouse的clickbench测试数据与查询SQL对AtomData进行了性能测试，所有的测试方法均与其他clickbench上的其他产品一致。预期将AtomData的测试结果与其他数据库产品进行对比，寻找出性能差异，便于后期产品进行性能优化，并清楚当前AtomData若打榜存在的优劣势。

测试环境

	类型	资源规格（CPU/MEM/DISK）
	RC	16C/24G/100G
AtomData	worker1	16C/24G/300G
	worker2	16C/24G/300G

测试方法

创建库与创建表，请自行连接所测数据库，并进行创建操作。

创建库

create database clickbench;

创建表

CREATE TABLE hits
(
    WatchID BIGINT NOT NULL,
    JavaEnable SMALLINT NOT NULL,
    Title TEXT NOT NULL,
    GoodEvent SMALLINT NOT NULL,
    EventTime TIMESTAMP NOT NULL,
    EventDate Date NOT NULL,
    CounterID INTEGER NOT NULL,
    ClientIP INTEGER NOT NULL,
    RegionID INTEGER NOT NULL,
    UserID BIGINT NOT NULL,
    CounterClass SMALLINT NOT NULL,
    OS SMALLINT NOT NULL,
    UserAgent SMALLINT NOT NULL,
    URL TEXT NOT NULL,
    Referer TEXT NOT NULL,
    IsRefresh SMALLINT NOT NULL,
    RefererCategoryID SMALLINT NOT NULL,
    RefererRegionID INTEGER NOT NULL,
    URLCategoryID SMALLINT NOT NULL,
    URLRegionID INTEGER NOT NULL,
    ResolutionWidth SMALLINT NOT NULL,
    ResolutionHeight SMALLINT NOT NULL,
    ResolutionDepth SMALLINT NOT NULL,
    FlashMajor SMALLINT NOT NULL,
    FlashMinor SMALLINT NOT NULL,
    FlashMinor2 TEXT NOT NULL,
    NetMajor SMALLINT NOT NULL,
    NetMinor SMALLINT NOT NULL,
    UserAgentMajor SMALLINT NOT NULL,
    UserAgentMinor VARCHAR(255) NOT NULL,
    CookieEnable SMALLINT NOT NULL,
    JavascriptEnable SMALLINT NOT NULL,
    IsMobile SMALLINT NOT NULL,
    MobilePhone SMALLINT NOT NULL,
    MobilePhoneModel TEXT NOT NULL,
    Params TEXT NOT NULL,
    IPNetworkID INTEGER NOT NULL,
    TraficSourceID SMALLINT NOT NULL,
    SearchEngineID SMALLINT NOT NULL,
    SearchPhrase TEXT NOT NULL,
    AdvEngineID SMALLINT NOT NULL,
    IsArtifical SMALLINT NOT NULL,
    WindowClientWidth SMALLINT NOT NULL,
    WindowClientHeight SMALLINT NOT NULL,
    ClientTimeZone SMALLINT NOT NULL,
    ClientEventTime TIMESTAMP NOT NULL,
    SilverlightVersion1 SMALLINT NOT NULL,
    SilverlightVersion2 SMALLINT NOT NULL,
    SilverlightVersion3 INTEGER NOT NULL,
    SilverlightVersion4 SMALLINT NOT NULL,
    PageCharset TEXT NOT NULL,
    CodeVersion INTEGER NOT NULL,
    IsLink SMALLINT NOT NULL,
    IsDownload SMALLINT NOT NULL,
    IsNotBounce SMALLINT NOT NULL,
    FUniqID BIGINT NOT NULL,
    OriginalURL TEXT NOT NULL,
    HID INTEGER NOT NULL,
    IsOldCounter SMALLINT NOT NULL,
    IsEvent SMALLINT NOT NULL,
    IsParameter SMALLINT NOT NULL,
    DontCountHits SMALLINT NOT NULL,
    WithHash SMALLINT NOT NULL,
    HitColor CHAR NOT NULL,
    LocalEventTime TIMESTAMP NOT NULL,
    Age SMALLINT NOT NULL,
    Sex SMALLINT NOT NULL,
    Income SMALLINT NOT NULL,
    Interests SMALLINT NOT NULL,
    Robotness SMALLINT NOT NULL,
    RemoteIP INTEGER NOT NULL,
    WindowName INTEGER NOT NULL,
    OpenerName INTEGER NOT NULL,
    HistoryLength SMALLINT NOT NULL,
    BrowserLanguage TEXT NOT NULL,
    BrowserCountry TEXT NOT NULL,
    SocialNetwork TEXT NOT NULL,
    SocialAction TEXT NOT NULL,
    HTTPError SMALLINT NOT NULL,
    SendTiming INTEGER NOT NULL,
    DNSTiming INTEGER NOT NULL,
    ConnectTiming INTEGER NOT NULL,
    ResponseStartTiming INTEGER NOT NULL,
    ResponseEndTiming INTEGER NOT NULL,
    FetchTiming INTEGER NOT NULL,
    SocialSourceNetworkID SMALLINT NOT NULL,
    SocialSourcePage TEXT NOT NULL,
    ParamPrice BIGINT NOT NULL,
    ParamOrderID TEXT NOT NULL,
    ParamCurrency TEXT NOT NULL,
    ParamCurrencyID SMALLINT NOT NULL,
    OpenstatServiceName TEXT NOT NULL,
    OpenstatCampaignID TEXT NOT NULL,
    OpenstatAdID TEXT NOT NULL,
    OpenstatSourceID TEXT NOT NULL,
    UTMSource TEXT NOT NULL,
    UTMMedium TEXT NOT NULL,
    UTMCampaign TEXT NOT NULL,
    UTMContent TEXT NOT NULL,
    UTMTerm TEXT NOT NULL,
    FromTag TEXT NOT NULL,
    HasGCLID SMALLINT NOT NULL,
    RefererHash BIGINT NOT NULL,
    URLHash BIGINT NOT NULL,
    CLID INTEGER NOT NULL,
    PRIMARY KEY (CounterID, EventDate, UserID, EventTime, WatchID)
);

查询SQL

创建文件queries2.sql，并将如下的所有SQL复制粘贴到queries2.sql文件中

SELECT COUNT(*) FROM hits;
SELECT COUNT(*) FROM hits WHERE AdvEngineID <> 0;
SELECT SUM(AdvEngineID), COUNT(*), AVG(ResolutionWidth) FROM hits;
SELECT AVG(UserID) FROM hits;
SELECT COUNT(DISTINCT UserID) FROM hits;
SELECT COUNT(DISTINCT SearchPhrase) FROM hits;
SELECT MIN(EventDate), MAX(EventDate) FROM hits;
SELECT AdvEngineID, COUNT(*) FROM hits WHERE AdvEngineID <> 0 GROUP BY AdvEngineID ORDER BY COUNT(*) DESC;
SELECT RegionID, COUNT(DISTINCT UserID) AS u FROM hits GROUP BY RegionID ORDER BY u DESC LIMIT 10;
SELECT RegionID, SUM(AdvEngineID), COUNT(*) AS c, AVG(ResolutionWidth), COUNT(DISTINCT UserID) FROM hits GROUP BY RegionID ORDER BY c DESC LIMIT 10;
SELECT MobilePhoneModel, COUNT(DISTINCT UserID) AS u FROM hits WHERE MobilePhoneModel <> \'\' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
SELECT MobilePhone, MobilePhoneModel, COUNT(DISTINCT UserID) AS u FROM hits WHERE MobilePhoneModel <> \'\' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
SELECT SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase <> \'\' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> \'\' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
SELECT SearchEngineID, SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase <> \'\' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT UserID, COUNT(*) FROM hits GROUP BY UserID ORDER BY COUNT(*) DESC LIMIT 10;
SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10;
SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase LIMIT 10;
SELECT UserID, extract(minute FROM EventTime) AS m, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, m, SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10;
SELECT UserID FROM hits WHERE UserID = 435090932899640449;
SELECT COUNT(*) FROM hits WHERE URL LIKE \'%google%\';
SELECT SearchPhrase, MIN(URL), COUNT(*) AS c FROM hits WHERE URL LIKE \'%google%\' AND SearchPhrase <> \'\' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT SearchPhrase, MIN(URL), MIN(Title), COUNT(*) AS c, COUNT(DISTINCT UserID) FROM hits WHERE Title LIKE \'%Google%\' AND URL NOT LIKE \'%.google.%\' AND SearchPhrase <> \'\' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT * FROM hits WHERE URL LIKE \'%google%\' ORDER BY EventTime LIMIT 10;
SELECT SearchPhrase FROM hits WHERE SearchPhrase <> \'\' ORDER BY EventTime LIMIT 10;
SELECT SearchPhrase FROM hits WHERE SearchPhrase <> \'\' ORDER BY SearchPhrase LIMIT 10;
SELECT SearchPhrase FROM hits WHERE SearchPhrase <> \'\' ORDER BY EventTime, SearchPhrase LIMIT 10;
SELECT CounterID, AVG(length(URL)) AS l, COUNT(*) AS c FROM hits WHERE URL <> \'\' GROUP BY CounterID HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25;
SELECT REGEXP_REPLACE(Referer, \'^https?://(?:www\\.)?([^/]+)/.*$\', \'\\1\') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> \'\' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25;
SELECT SUM(ResolutionWidth), SUM(ResolutionWidth + 1), SUM(ResolutionWidth + 2), SUM(ResolutionWidth + 3), SUM(ResolutionWidth + 4), SUM(ResolutionWidth + 5), SUM(ResolutionWidth + 6), SUM(ResolutionWidth + 7), SUM(ResolutionWidth + 8), SUM(ResolutionWidth + 9), SUM(ResolutionWidth + 10), SUM(ResolutionWidth + 11), SUM(ResolutionWidth + 12), SUM(ResolutionWidth + 13), SUM(ResolutionWidth + 14), SUM(ResolutionWidth + 15), SUM(ResolutionWidth + 16), SUM(ResolutionWidth + 17), SUM(ResolutionWidth + 18), SUM(ResolutionWidth + 19), SUM(ResolutionWidth + 20), SUM(ResolutionWidth + 21), SUM(ResolutionWidth + 22), SUM(ResolutionWidth + 23), SUM(ResolutionWidth + 24), SUM(ResolutionWidth + 25), SUM(ResolutionWidth + 26), SUM(ResolutionWidth + 27), SUM(ResolutionWidth + 28), SUM(ResolutionWidth + 29), SUM(ResolutionWidth + 30), SUM(ResolutionWidth + 31), SUM(ResolutionWidth + 32), SUM(ResolutionWidth + 33), SUM(ResolutionWidth + 34), SUM(ResolutionWidth + 35), SUM(ResolutionWidth + 36), SUM(ResolutionWidth + 37), SUM(ResolutionWidth + 38), SUM(ResolutionWidth + 39), SUM(ResolutionWidth + 40), SUM(ResolutionWidth + 41), SUM(ResolutionWidth + 42), SUM(ResolutionWidth + 43), SUM(ResolutionWidth + 44), SUM(ResolutionWidth + 45), SUM(ResolutionWidth + 46), SUM(ResolutionWidth + 47), SUM(ResolutionWidth + 48), SUM(ResolutionWidth + 49), SUM(ResolutionWidth + 50), SUM(ResolutionWidth + 51), SUM(ResolutionWidth + 52), SUM(ResolutionWidth + 53), SUM(ResolutionWidth + 54), SUM(ResolutionWidth + 55), SUM(ResolutionWidth + 56), SUM(ResolutionWidth + 57), SUM(ResolutionWidth + 58), SUM(ResolutionWidth + 59), SUM(ResolutionWidth + 60), SUM(ResolutionWidth + 61), SUM(ResolutionWidth + 62), SUM(ResolutionWidth + 63), SUM(ResolutionWidth + 64), SUM(ResolutionWidth + 65), SUM(ResolutionWidth + 66), SUM(ResolutionWidth + 67), SUM(ResolutionWidth + 68), SUM(ResolutionWidth + 69), SUM(ResolutionWidth + 70), SUM(ResolutionWidth + 71), SUM(ResolutionWidth + 72), SUM(ResolutionWidth + 73), SUM(ResolutionWidth + 74), SUM(ResolutionWidth + 75), SUM(ResolutionWidth + 76), SUM(ResolutionWidth + 77), SUM(ResolutionWidth + 78), SUM(ResolutionWidth + 79), SUM(ResolutionWidth + 80), SUM(ResolutionWidth + 81), SUM(ResolutionWidth + 82), SUM(ResolutionWidth + 83), SUM(ResolutionWidth + 84), SUM(ResolutionWidth + 85), SUM(ResolutionWidth + 86), SUM(ResolutionWidth + 87), SUM(ResolutionWidth + 88), SUM(ResolutionWidth + 89) FROM hits;
SELECT SearchEngineID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> \'\' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> \'\' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT URL, COUNT(*) AS c FROM hits GROUP BY URL ORDER BY c DESC LIMIT 10;
SELECT 1, URL, COUNT(*) AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, COUNT(*) AS c FROM hits GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;
SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-01\' AND EventDate <= \'2013-07-31\' AND DontCountHits = 0 AND IsRefresh = 0 AND URL <> \'\' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
SELECT Title, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-01\' AND EventDate <= \'2013-07-31\' AND DontCountHits = 0 AND IsRefresh = 0 AND Title <> \'\' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-01\' AND EventDate <= \'2013-07-31\' AND IsRefresh = 0 AND IsLink <> 0 AND IsDownload = 0 GROUP BY URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000;
SELECT TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE \'\' END AS Src, URL AS Dst, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-01\' AND EventDate <= \'2013-07-31\' AND IsRefresh = 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, Src, Dst ORDER BY PageViews DESC LIMIT 10 OFFSET 1000;
SELECT URLHash, EventDate, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-01\' AND EventDate <= \'2013-07-31\' AND IsRefresh = 0 AND TraficSourceID IN (-1, 6) AND RefererHash = 3594120000172545465 GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 10 OFFSET 100;
SELECT WindowClientWidth, WindowClientHeight, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-01\' AND EventDate <= \'2013-07-31\' AND IsRefresh = 0 AND DontCountHits = 0 AND URLHash = 2868770270353813622 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10 OFFSET 10000;
SELECT DATE_FORMAT(EventTime, \'%Y-%m-%d %H:00:00\') AS M, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= \'2013-07-14\' AND EventDate <= \'2013-07-15\' AND IsRefresh = 0 AND DontCountHits = 0 GROUP BY DATE_FORMAT(EventTime, \'%Y-%m-%d %H:00:00\') ORDER BY DATE_FORMAT(EventTime, \'%Y-%m-%d %H:00:00\') LIMIT 10 OFFSET 1000;

运行脚本

在Linux的随意目录下，创建一个run.sh直接复制如下脚本
● 注意修改数据库的连接IP与密码等
● queries2.sql 存放查询的SQL，请将queries2.sql 文件与run.sh放在同一个目录下（不强求）

#!/bin/bash
  
TRIES=3    #运行的次数
QUERY_NUM=1    #表示excel中从query1开始显示
touch result.csv    #结果将写入到result.csv文件中
truncate -s0 result.csv

# Record start time
echo "Start time: $(date +%F\\ %T)" | tee result.csv

first_query=true
while read -r query; do
    sync

    if ! $first_query; then
        echo "" | tee -a result.csv
    fi
    first_query=false

    echo -n "query$QUERY_NUM," | tee -a result.csv
    for i in $(seq 1 $TRIES); do
         time mysql -h192.168.32.40 -P3001 -uroot -pAa123456 clickbench -e "$query" ;   2>&1 | grep real > a.txt
        grep -oP \'real.*\\K\\d+\\.\\d+\' a.txt | awk \'print $1\' | tr \'\\n\' \',\' | tee -a result.csv
    done

    QUERY_NUM=$((QUERY_NUM + 1))

done <queries2.sql     #所有运行的查询SQL都在queries2.sql文件中

# Record end time
echo "" | tee -a result.csv
echo "End time: $(date +%F\\ %T)" | tee -a result.csv

脚本运行后，会显示每个query的耗时，如上脚本将每个SQL执行3次，因此结果如下：

 

1.作者：Syw 2.出处：http://www.cnblogs.com/syw20170419/ 3.本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。 4.如果文中有什么错误，欢迎指出。以免更多的人被误导。

性能测试那么笼统，测试小白到底该如何认知性能？

性能概念

软件的性能是软件的一种非功能特性，它关注的不是软件是否能够完成特定的功能，而是在完成该功能时展示出来的及时性。

　由定义可知性能关注的是软件的非功能特性，所以一般来说性能测试介入的时机是在功能测试完成之后。另外，由及时性可知性能也是一种指标，可以用时间或其它指标来衡量，通常我们会使用某些工具或手段来检测软件的某些指标是否达到了要求，这就是性能测试。 

性能测试定义：指通过自动化的测试工具模拟多种正常、峰值以及异常负载条件来对系统的各项性能指标进行测试。

1、响应时间

Response time = (N1+N2+N3+N4)+ (A1+A2+a3)，即：（网络时间 + 应用程序处理时间）

响应时间-负载对应关系：

图中拐点说明：

响应时间突然增加

意味着系统的一种或多种资源利用达到的极限

通常可以利用拐点来进行性能测试分析与定位 

2、并发数

并发用户数：某一物理时刻同时向系统提交请求的用户数，提交的请求可能是同一个场景或功能，也可以是不同场景或功能。

在线用户数：某段时间内访问系统的用户数，这些用户并不一定同时向系统提交请求

系统用户数：系统注册的总用户数据

三者之间的关系：系统用户数 >= 在线用户数 >= 并发用户数

3、TPS（吞吐量）

定义：单位时间内系统处理的客户端请求的数量

计算单位：一般使用 请求数／秒做为吞吐量的单位，出可以使用 页面数／秒表表示。

另外，从业务角度来说也可以使用 访问人数 /天 或 页面访问量／天 ，看实际情况。

计算方法：Throughput = (number of requests) / (total time).

吞吐量-负载对应关系：

图中拐点说明：

吞吐量逐渐达到饱和

意味着系统的一种或多种资源利用达到的极限

通常可以利用拐点来进行性能测试分析与定位

4、资源利用率

定义：指的是对不同系统资源的使用程度，通常以占用最大值的百分比来衡量

通常需要关注的服务器资源如下：

CPU：就像人的大脑，主要负责相关事情的判断以及实际处理的机制

内存：大脑中的记忆块区，将眼睛，皮肤等收集到的信息记录起来的地方，以供cpu进行判断，但是是临时的，访问速度快，如果关机或断电这里的数据会消失。

磁盘IO：大脑中的记忆区块，将重要的数据保存起来（永久保存，关机或断电不会丢失，速度慢），以便将来再次使用这些数据。

网络：数据传输通道，一般局域网环境内，

带宽问题的可能性比较小。

资源利用-负载对应关系：

图中拐点说明：

服务器某荐资源使用逐渐达到饱和

通常可以利用拐点来进行性能测试分析与定位

5、事务&思考时间

事务：用户自定义的一个标识，用来衡量不同的操作所花费的时间，事务时间反映的是一个操作过程的响应时间。

思考时间：为了更好的模拟用户的行为，需要模拟用户在不同操作之间等待的时间，例如，当用户收到来自服务器的数据时，可能要等待几秒查看数据，然后再做出响应，这种延迟，就称为思考时间

集合点：模拟系统上较重用户负载时，有时需要模拟多个用户恰好在同一时刻执行任务，这时候就需要创建集合点，配置多个用户同时执行操作，当某个用户到达集合点时，将进行等待，直到指定数据量的用户到达后才释放这些用户。

备注：思考时间内是不对服务器产生压力的，加上思考时间会让你的压测的数据更好看！

6、其它常用概念

点击数：每秒钟用户向WEB服务器提交的HTTP请求数。

这个指标是WEB应用特有的一个指标:WEB应用是"请求-响应"模式,用户发出一次申请,服务器就要处理一次，所以点击是WEB应用能够处理的交易的最小单位。

如果把每次点击定义为一个交易，点击率和TPS就是一个概念。容易看出，点击率越大，对服务器的压力越大。

点击率只是一个性能参考指标，重要的是分析点击时产生的影响。

需要注意的是，这里的点击并非指鼠标的一次单击操作，因为在一次单击操作中，客户端可能向服务器发出多个HTTP请求.

PV：访问一个URL，产生一个PV（Page View，页面访问量），每日每个网站的总PV量是形容一个 网站规模的重要指标。

UV：作为一个独立的用户，访问站点的所有页面均算作一个UV（Unique Visitor，用户访问）

扫描二维码

往期回顾

01 ｜

02 ｜

03 ｜

04 ｜