calico性能测试

Posted 2020-08-30 puroc

硬件环境：

三台虚拟机：

192.168.99.129 master（kube-apiserver、kube-controller-manager、kube-proxy、kube-scheduler、kubelet、etcd、calico、docker）

192.168.99.130 slave1（kube-proxy、kubelet、etcd proxy、calico、docker、dns）

192.168.99.131 slave2（kube-proxy、kubelet、etcd proxy、calico、docker）

软件环境：

kubernetes 1.5.2

etcd 3.1.0

calico 0.23.1

qperf （网络性能测试工具，安装方法：yum install -y qperf）

测试结果：

【物理机到物理机】

[[email protected] kubernetes]# qperf 192.168.99.130 -oo msg_size:1:64K:*2  tcp_bw  tcp_lat
tcp_bw:
    bw  =  2.49 MB/sec
tcp_bw:
    bw  =  4.69 MB/sec
tcp_bw:
    bw  =  8.18 MB/sec
tcp_bw:
    bw  =  14.3 MB/sec
tcp_bw:
    bw  =  22.2 MB/sec
tcp_bw:
    bw  =  29.5 MB/sec
tcp_bw:
    bw  =  35.2 MB/sec
tcp_bw:
    bw  =  40.6 MB/sec
tcp_bw:
    bw  =  64.9 MB/sec
tcp_bw:
    bw  =  47.5 MB/sec
tcp_bw:
    bw  =  49.6 MB/sec
tcp_bw:
    bw  =  59.1 MB/sec
tcp_bw:
    bw  =  88.9 MB/sec
tcp_bw:
    bw  =  151 MB/sec
tcp_bw:
    bw  =  178 MB/sec
tcp_bw:
    bw  =  203 MB/sec
tcp_bw:
    bw  =  209 MB/sec
tcp_lat:
    latency  =  1.08 ms
tcp_lat:
    latency  =  1.21 ms
tcp_lat:
    latency  =  180 us
tcp_lat:
    latency  =  68.4 us
tcp_lat:
    latency  =  69 us
tcp_lat:
    latency  =  70.9 us
tcp_lat:
    latency  =  70.5 us
tcp_lat:
    latency  =  67.9 us
tcp_lat:
    latency  =  75.5 us
tcp_lat:
    latency  =  69.9 us
tcp_lat:
    latency  =  81.1 us
tcp_lat:
    latency  =  87.1 us
tcp_lat:
    latency  =  88.8 us
tcp_lat:
    latency  =  112 us
tcp_lat:
    latency  =  145 us
tcp_lat:
    latency  =  218 us
tcp_lat:
    latency  =  502 us

【容器到容器】

[[email protected] kubernetes]# qperf 192.168.140.199 -oo msg_size:1:64K:*2 tcp_bw  tcp_lat  conf
tcp_bw:
    bw  =  2.67 MB/sec
tcp_bw:
    bw  =  4.86 MB/sec
tcp_bw:
    bw  =  8.44 MB/sec
tcp_bw:
    bw  =  15.6 MB/sec
tcp_bw:
    bw  =  23 MB/sec
tcp_bw:
    bw  =  32.6 MB/sec
tcp_bw:
    bw  =  43.8 MB/sec
tcp_bw:
    bw  =  45 MB/sec
tcp_bw:
    bw  =  48.6 MB/sec
tcp_bw:
    bw  =  53.1 MB/sec
tcp_bw:
    bw  =  59.4 MB/sec
tcp_bw:
    bw  =  69.1 MB/sec
tcp_bw:
    bw  =  103 MB/sec
tcp_bw:
    bw  =  156 MB/sec
tcp_bw:
    bw  =  178 MB/sec
tcp_bw:
    bw  =  203 MB/sec
tcp_bw:
    bw  =  209 MB/sec
tcp_lat:
    latency  =  88.7 us
tcp_lat:
    latency  =  82.3 us
tcp_lat:
    latency  =  88.4 us
tcp_lat:
    latency  =  93 us
tcp_lat:
    latency  =  84.1 us
tcp_lat:
    latency  =  89.8 us
tcp_lat:
    latency  =  84.8 us
tcp_lat:
    latency  =  83.8 us
tcp_lat:
    latency  =  82.7 us
tcp_lat:
    latency  =  83.1 us
tcp_lat:
    latency  =  88.6 us
tcp_lat:
    latency  =  101 us
tcp_lat:
    latency  =  105 us
tcp_lat:
    latency  =  127 us
tcp_lat:
    latency  =  170 us
tcp_lat:
    latency  =  283 us
tcp_lat:
    latency  =  618 us

【容器到物理机】

[[email protected] kubernetes]# qperf 192.168.99.129 -oo msg_size:1:64K:*2 tcp_bw  tcp_lat  conf
tcp_bw:
    bw  =  2.65 MB/sec
tcp_bw:
    bw  =  4.92 MB/sec
tcp_bw:
    bw  =  8.66 MB/sec
tcp_bw:
    bw  =  15.2 MB/sec
tcp_bw:
    bw  =  23.3 MB/sec
tcp_bw:
    bw  =  30.4 MB/sec
tcp_bw:
    bw  =  37.5 MB/sec
tcp_bw:
    bw  =  43 MB/sec
tcp_bw:
    bw  =  48.9 MB/sec
tcp_bw:
    bw  =  51.5 MB/sec
tcp_bw:
    bw  =  54.9 MB/sec
tcp_bw:
    bw  =  67.6 MB/sec
tcp_bw:
    bw  =  103 MB/sec
tcp_bw:
    bw  =  147 MB/sec
tcp_bw:
    bw  =  175 MB/sec
tcp_bw:
    bw  =  206 MB/sec
tcp_bw:
    bw  =  213 MB/sec
tcp_lat:
    latency  =  76.5 us
tcp_lat:
    latency  =  77.6 us
tcp_lat:
    latency  =  76.5 us
tcp_lat:
    latency  =  77.7 us
tcp_lat:
    latency  =  74.7 us
tcp_lat:
    latency  =  79.6 us
tcp_lat:
    latency  =  79.3 us
tcp_lat:
    latency  =  81.4 us
tcp_lat:
    latency  =  81.5 us
tcp_lat:
    latency  =  82.1 us
tcp_lat:
    latency  =  80.3 us
tcp_lat:
    latency  =  92.3 us
tcp_lat:
    latency  =  93.7 us
tcp_lat:
    latency  =  112 us
tcp_lat:
    latency  =  155 us
tcp_lat:
    latency  =  237 us
tcp_lat:
    latency  =  609 us

结论：calico的性能几乎与物理网卡的性能一样。