ceph pool 性能测试
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了ceph pool 性能测试相关的知识，希望对你有一定的参考价值。
Ceph包含在rados bench RADOS存储集群上执行性能基准测试的命令。该命令将执行写测试和两种类型的读测试。--no-cleanup在测试读写性能时，该选项很重要。默认情况下，该rados bench命令将删除已写入存储池的对象。留在这些对象后面允许两个读取测试来测量顺序和随机读取性能。（1）准备工作，清除所有的文件系统缓存
[root@node141 mnt]# echo 3 > /proc/sys/vm/drop_caches
[root@node141 mnt]# sync
（2）创建一个新的存储池
[root@node141 mnt]# ceph osd pool create rbd 128 128
（3）测试一下rbd的写性能，添加 --no-cleanup参数
[root@node141 mnt]# rados bench -p rbd 10 write --no-cleanup
hints = 1
Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects
Object prefix: benchmark_data_node141_28363
sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s)
0 0 0 0 0 0 - 0
1 16 81 65 259.999 260 0.107901 0.200157
2 16 161 145 289.976 320 0.190659 0.215092
3 16 252 236 314.632 364 0.0807696 0.193549
4 16 351 335 334.958 396 0.191582 0.185652
5 16 445 429 343.157 376 0.158618 0.178393
6 16 548 532 354.621 412 0.162274 0.178178
7 16 652 636 363.378 416 0.190742 0.173784
8 16 753 737 368.448 404 0.15392 0.170866
9 16 863 847 376.391 440 0.103038 0.169129
10 16 964 948 379.144 404 0.138741 0.167306
Total time run: 10.1023
Total writes made: 965
Write size: 4194304
Object size: 4194304
Bandwidth (MB/sec): 382.09
Stddev Bandwidth: 53.3267
Max bandwidth (MB/sec): 440
Min bandwidth (MB/sec): 260
Average IOPS: 95
Stddev IOPS: 13.3317
Max IOPS: 110
Min IOPS: 65
Average Latency(s): 0.167422
Stddev Latency(s): 0.0765119
Max latency(s): 0.668499
Min latency(s): 0.0569634（4）测试一下，10秒的顺序读取：
[root@node141 mnt]# rados bench -p rbd 10 seq
hints = 1
sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s)
0 0 0 0 0 0 - 0
1 16 257 241 963.679 964 0.0607106 0.0623843
2 16 549 533 1065.73 1168 0.0450988 0.0583824
3 16 835 819 1091.76 1144 0.044447 0.057111
Total time run: 3.47417
Total reads made: 965
Read size: 4194304
Object size: 4194304
Bandwidth (MB/sec): 1111.06
Average IOPS: 277
Stddev IOPS: 27.8747
Max IOPS: 292
Min IOPS: 241
Average Latency(s): 0.056581
Max latency(s): 0.388605
Min latency(s): 0.015351（5）测试10秒随机读写性能
[root@node141 mnt]# rados bench -p rbd 10 rand
hints = 1
sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s)
0 0 0 0 0 0 - 0
1 16 408 392 1566.56 1568 0.0715088 0.038812
2 16 794 778 1554.08 1544 0.0219942 0.0396005
3 16 1211 1195 1591.93 1668 0.125987 0.0389289
4 15 1604 1589 1587.9 1576 0.0497586 0.0392412
5 16 2013 1997 1596.67 1632 0.00512184 0.0388821
6 16 2415 2399 1598.52 1608 0.146983 0.0389842
7 16 2806 2790 1593.56 1564 0.110503 0.0391283
8 16 3186 3170 1584.34 1520 0.0226104 0.0393014
9 15 3598 3583 1591.83 1652 0.0117805 0.0392624
10 16 3965 3949 1579.03 1464 0.0752829 0.0395739
Total time run: 10.0536
Total reads made: 3965
Read size: 4194304
Object size: 4194304
Bandwidth (MB/sec): 1577.54
Average IOPS: 394
Stddev IOPS: 15.6308
Max IOPS: 417
Min IOPS: 366
Average Latency(s): 0.0397174
Max latency(s): 0.353027
Min latency(s): 0.00414538（6）要增加并发读取和写入的数量，请使用-t 默认为16个现场的选项，-b参数可以调整正在写入的对象的大小，默认对象大小为4MB。安全的最大对象为16MB.
添加 --run-name <label> 选项以控制在基准测试期间写入的对象的名称。多个rados bench命令可以同时通过改变跑出--run-name标签每个运行命令的实例。这可以防止在多个客户端尝试访问同一对象并允许不同客户端访问不同对象时可能发生的潜在I / O错误。--run-name在尝试模拟真实世界的工作负载时，该选项也很有用。例如：
[root@node141 mnt]# rados bench -p rbd 10 write -t 4 --run-name zabbix71
hints = 1
Maintaining 4 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects
Object prefix: benchmark_data_node141_28450
sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s)
0 0 0 0 0 0 - 0
1 4 40 36 143.992 144 0.065704 0.103059
2 4 86 82 163.981 184 0.0763531 0.0955013
3 4 126 122 162.641 160 0.106746 0.0953542
4 4 173 169 168.972 188 0.0872597 0.0934176
5 4 218 214 171.171 180 0.170851 0.0927783
6 4 255 251 167.305 148 0.0943133 0.093864
7 4 294 290 165.686 156 0.0976048 0.0959294
8 4 340 336 167.971 184 0.0882485 0.0946923
9 4 386 382 169.748 184 0.0703278 0.0933611
10 4 430 426 170.37 176 0.0623777 0.0933391
Total time run: 10.1109
Total writes made: 431
Write size: 4194304
Object size: 4194304
Bandwidth (MB/sec): 170.51
Stddev Bandwidth: 16.6747
Max bandwidth (MB/sec): 188
Min bandwidth (MB/sec): 144
Average IOPS: 42
Stddev IOPS: 4.16867
Max IOPS: 47
Min IOPS: 36
Average Latency(s): 0.0935851
Stddev Latency(s): 0.0433054
Max latency(s): 0.579434
Min latency(s): 0.0533891
Cleaning up (deleting benchmark objects)
Removed 431 objects
Clean up completed and total clean up time :1.62203(7) 删除rados bench 命令创建的数据
[root@node141 mnt]# rados -p rbd cleanup
Removed 965 objects
以上是关于ceph pool 性能测试的主要内容，如果未能解决你的问题，请参考以下文章