swift分布式存储性能测试以及破坏性测试
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了swift分布式存储性能测试以及破坏性测试相关的知识，希望对你有一定的参考价值。
延续上一个文章的做接下来的测试，环境是一个proxy和三个storage。
1、性能测试
测试一：五个线程，每个线程上传一个大小差不多一个G的文件
/data/swift/upload> du -sh test*.zip
987M test1.zip
987M test2.zip
987M test3.zip
987M test4.zip
987M test.zip
上传
swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container1 test.zip
swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container2 test1.zip
swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container3 test2.zip
swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container4 test3.zip
swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container5 test4.zip
共用时8:30:45
存储节点查看文件
/srv/node/sdb1> du -sh *
16K accounts
356M async_pending
129M containers
4.9G objects
0 tmp
查看上传的文件
/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container1
test.zip
/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container2
test1.zip
/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container3
test2.zip
/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container4
test3.zip
/data/swift/upload> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container5
test4.zip
测试二，十个线程：
/data/swift/upload/test100M> du -sh *
/data/swift/upload/test100M> du -sh *
505M test10.zip
505M test1.zip
505M test2.zip
505M test3.zip
505M test4.zip
505M test5.zip
505M test6.zip
505M test7.zip
505M test8.zip
505M test9.zip
单个线程测试上传2.6M/s,下载5.1M/s
五线程测试：
上传
第一个用时03:20s,2.5M/s
第二个用时04:30s,1.85M/s
第三个用时05:50s,1.31M/s
第四个用时06:55s,1.2M/s
第五个用时07:00s,1.19M/s
下载
第一个用时30.263s, 17.461 MB/s
第二个用时49.415s, 10.691 MB/s
第三个用时54.724s, 9.653 MB/s
第四个用时86.473s, 6.108 MB/s
第五个用时97.919s, 5.417 MB/s
十线程测试：
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container1 test1.zip
test1.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container2 test2.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container2/test2.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container3 test3.zip
test3.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container4 test4.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container4/test4.zip 503 Internal Server Error [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container5 test5.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container5/test5.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container6 test6.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container6/test6.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container7 test7.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container7/test7.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container8 test8.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container8/test8.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container9 test9.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container9/test9.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container10 test10.zip
Object PUT failed: http://192.168.1.211:8090:8090/v1/AUTH_system/container10/test10.zip 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
但是只有七个上传成功
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container1
test1.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container2
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container3
test3.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container4
test4.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container5
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container6
test6.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container7
test7.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container8
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container9
test9.zip
/data/swift/upload/test100M> swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass list container10
test10.zip
上传
第一个用时00:39s,12M/s
第二个用时01:01s,8.19M/s
第三个用时03:37s,2.3M/s
第四个用时06:16s,1.32M/s
第五个用时06:16s,1.32M/s
第六个用时07:06s,1.17M/s
第七个用时07:06s,1.17M/s
下载
第一个用时73.207s, 7.216 MB/s
第二个用时107.437s, 4.916 MB/s
第三个用时107.581s, 4.910 MB/s
第四个用时108.457s, 4.870 MB/s
第五个用时117.852s, 4.482 MB/s
第六个用时121.565s, 4.345 MB/s
第七个用时122.600s, 4.308 MB/s
第八个用时122.413s, 4.315 MB/s
第九个用时145.383s, 3.633 MB/s
第十个用时174.677s, 3.024 MB/s
2、破坏性测试
测试一：测试关闭一个存储节点的服务之后上传文件会不会在开启该存储服务后同步
查看一个存储节点的服务
/srv/node/sdb1> ps -ef | grep swift
swift 5651 10735 5 17:16 ? 00:01:10 /usr/bin/python /usr/bin/swift-object-updater /etc/swift/object-server.conf
swift 8409 10727 3 17:38 ? 00:00:01 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf
was_wcm 8477 12319 0 17:38 pts/0 00:00:00 grep swift
swift 10723 1 0 Mar28 ? 00:00:50 /usr/bin/python /usr/bin/swift-container-updater /etc/swift/container-server.conf
swift 10724 1 0 Mar28 ? 00:00:05 /usr/bin/python /usr/bin/swift-account-auditor /etc/swift/account-server.conf
swift 10725 1 0 Mar28 ? 03:18:56 /usr/bin/python /usr/bin/swift-object-replicator /etc/swift/object-server.conf
swift 10726 1 1 Mar28 ? 04:23:53 /usr/bin/python /usr/bin/swift-container-replicator /etc/swift/container-server.conf
swift 10727 1 0 Mar28 ? 00:08:35 /usr/bin/python /usr/bin/swift-object-auditor /etc/swift/object-server.conf
swift 10728 1 0 Mar28 ? 00:01:47 /usr/bin/python /usr/bin/swift-object-expirer /etc/swift/object-expirer.conf
swift 10729 1 0 Mar28 ? 00:00:04 /usr/bin/python /usr/bin/swift-container-auditor /etc/swift/container-server.conf
swift 10730 1 0 Mar28 ? 00:00:00 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift 10731 1 0 Mar28 ? 00:00:00 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
swift 10732 1 0 Mar28 ? 00:00:02 /usr/bin/python /usr/bin/swift-account-reaper /etc/swift/account-server.conf
swift 10733 1 0 Mar28 ? 00:00:24 /usr/bin/python /usr/bin/swift-container-sync /etc/swift/container-server.conf
swift 10734 1 0 Mar28 ? 00:06:13 /usr/bin/python /usr/bin/swift-account-replicator /etc/swift/account-server.conf
swift 10735 1 0 Mar28 ? 00:00:06 /usr/bin/python /usr/bin/swift-object-updater /etc/swift/object-server.conf
swift 10736 1 0 Mar28 ? 00:00:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift 10795 10730 0 Mar28 ? 00:18:29 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift 10796 10730 0 Mar28 ? 00:17:56 /usr/bin/python /usr/bin/swift-container-server /etc/swift/container-server.conf
swift 10797 10736 0 Mar28 ? 02:58:11 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift 10798 10736 0 Mar28 ? 03:16:19 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
swift 10799 10731 0 Mar28 ? 00:04:15 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
swift 10800 10731 0 Mar28 ? 00:04:30 /usr/bin/python /usr/bin/swift-account-server /etc/swift/account-server.conf
杀死服务
# kill -9 `ps -ef | grep -v grep | grep object-server | awk ‘{print $2}‘`
上传文件
swift -Ahttp://192.168.1.211:8090/auth/v1.0 -U system:root -K testpass upload container1 apache_install
启动服务之前的三个存储文件对比
[email protected]:/srv/node/sdb1/objects> ll | wc -l
21
[email protected]:/srv/node/sdb1/objects> ll | wc -l
44
[email protected]:/srv/node/sdb1/objects> ll | wc -l
44
启动服务
[email protected]:/srv/node/sdb1/objects> sudo swift-init all restart
[email protected]:/srv/node/sdb1/objects> ll | wc -l
44
可以看到文件已经同步
测试二：测试删除一个存储节点物理服务器上的文件会不会在之后同步其他存储节点上的文件
删除一个存储节点的文件
[email protected]:/srv/node/sdb1/objects> rm -rf *
查看文件的同步变化
[email protected]:/srv/node/sdb1> du -sh *
16K accounts
356M async_pending
129M containers
0 objects
0 tmp
[email protected]:/srv/node/sdb1> du -sh *
16K accounts
356M async_pending
129M containers
260K objects
0 tmp
[email protected]:/srv/node/sdb1> du -sh *
16K accounts
356M async_pending
129M containers
33M objects
0 tmp
[email protected]:/srv/node/sdb1> du -sh *
16K accounts
356M async_pending
129M containers
129M objects
0 tmp
[email protected]:/srv/node/sdb1> du -sh *
16K accounts
356M async_pending
129M containers
5.0G objects
0 tmp
可以看到文件渐渐同步完成
本文出自 “xiaoxiaozhou” 博客，请务必保留此出处http://xiaoxiaozhou.blog.51cto.com/4681537/1918628
以上是关于swift分布式存储性能测试以及破坏性测试的主要内容，如果未能解决你的问题，请参考以下文章