SeaweedFS —— 可存储数十亿文件的分布式文件系统

Posted 2021-04-24 OSC开源社区

SeaweedFS 是简单，高伸缩性的分布式文件系统，包含两部分：存储数十亿的文件；快速为文件服务。SeaweedFS 作为支持全 POSIX 文件系统语义替代，Seaweed-FS 选择仅实现 key-file 的映射，类似 "NoSQL"，也可以说是 "NoFS"。

SeaweedFS 仅花费 40 字节的硬盘来存储每个文件的元数据。

基准测试基于 Mac Book，固态硬盘，CPU：1 Intel Core i7 2.6GHz。

编写 1 百万的 1KB 文件：

Concurrency Level: 16
Time taken for tests: 88.796 seconds
Complete requests: 1048576
Failed requests: 0
Total transferred: 1106764659 bytes
Requests per second: 11808.87 [#/sec]
Transfer rate: 12172.05 [Kbytes/sec]

Connection Times (ms)
min avg max std
Total: 0.2 1.3 44.8 0.9

Percentage of the requests served within a certain time (ms)
50% 1.1 ms
66% 1.3 ms
75% 1.5 ms
80% 1.7 ms
90% 2.1 ms
95% 2.6 ms
98% 3.7 ms
99% 4.6 ms
100% 44.8 ms

自由读取 1 百万文件：

Concurrency Level: 16
Time taken for tests: 34.263 seconds
Complete requests: 1048576
Failed requests: 0
Total transferred: 1106762945 bytes
Requests per second: 30603.34 [#/sec]
Transfer rate: 31544.49 [Kbytes/sec]

Connection Times (ms)
min avg max std
Total: 0.0 0.5 20.7 0.7

Percentage of the requests served within a certain time (ms)
50% 0.4 ms
75% 0.5 ms
95% 0.6 ms
98% 0.8 ms
99% 1.2 ms
100% 20.7 ms