跨主机容器之间通信实现方式:etcd+flanned

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了跨主机容器之间通信实现方式:etcd+flanned相关的知识,希望对你有一定的参考价值。

一、环境说明

node-1: 10.0.0.168
node-2: 10.0.0.169
node-3: 10.0.0.170

二、初始化环境

1、hostnamectl --static set-hostname hostname

2、/etc/hosts

10.0.0.168 node-1
10.0.0.169 node-2
10.0.0.170 node-3

三、安装etcd集群

1、在每台机器上

yum install -y etcd

2、配置 /etc/etcd/etcd.conf

(a)、node1

ETCD_NAME=etcd1
ETCD_DATA_DIR="/var/lib/etcd/etcd1.etcd"
ETCD_LISTEN_PEER_URLS="http://10.0.0.168:2380"
ETCD_LISTEN_CLIENT_URLS="http://10.0.0.168:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.0.0.168:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://10.0.0.168:2380,etcd2=http://10.0.0.169:2380,etcd3=http://10.0.0.170:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.168:2379"

(2)、node2

ETCD_NAME=etcd2
ETCD_DATA_DIR="/var/lib/etcd/etcd2.etcd"
ETCD_LISTEN_PEER_URLS="http://10.0.0.169:2380"
ETCD_LISTEN_CLIENT_URLS="http://10.0.0.169:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.0.0.169:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://10.0.0.168:2380,etcd2=http://10.0.0.169:2380,etcd3=http://10.0.0.170:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.169:2379"

(3)、node3

ETCD_NAME=etcd3
ETCD_DATA_DIR="/var/lib/etcd/etcd3.etcd"
ETCD_LISTEN_PEER_URLS="http://10.0.0.170:2380"
ETCD_LISTEN_CLIENT_URLS="http://10.0.0.170:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.0.0.170:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://10.0.0.168:2380,etcd2=http://10.0.0.169:2380,etcd3=http://10.0.0.170:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.170:2379"

3、修改 etcd 启动文件 /usr/lib/systemd/system/etcd.service

sed -i s/\\\"${ETCD_LISTEN_CLIENT_URLS}\\\"/\\\"${ETCD_LISTEN_CLIENT_URLS}\\\" --listen-client-urls=\\\"${ETCD_LISTEN_CLIENT_URLS}\\\" --advertise-client-urls=\\\"${ETCD_ADVERTISE_CLIENT_URLS}\\\" --initial-cluster-token=\\\"${ETCD_INITIAL_CLUSTER_TOKEN}\\\" --initial-cluster=\\\"${ETCD_INITIAL_CLUSTER}\\\" --initial-cluster-state=\\\"${ETCD_INITIAL_CLUSTER_STATE}\\\"/g /usr/lib/systemd/system/etcd.service

4、分别启动 所有节点的 etcd 服务

systemctl enable etcd
systemctl start etcd
systemctl status etcd

5、查看etcd集群状态,以及集群成员

etcdctl cluster-health
etcdctl member list

四、flanneld网络

1、安装flannel

yum install -y flannel

2、清除网络中遗留的docker网络(如 docker0 flannel0等)

ip link delete docker0

3、设置flannel所用到的网络

etcdctl --endpoint http://10.0.0.168:2379 set /flannel/network/config ‘{"Network":"172.16.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan","VNI":1}}‘
etcdctl --endpoint http://10.0.0.169:2379 set /flannel/network/config ‘{"Network":"172.16.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan","VNI":1}}‘
etcdctl --endpoint http://10.0.0.170:2379 set /flannel/network/config ‘{"Network":"172.16.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan","VNI":1}}‘

4、修改flannel配置文件(/etc/sysconfig/flanneld)

FLANNEL_ETCD="http://10.0.0.168:2379,http://10.0.0.169:2379,http://10.0.0.170:2379"      # 修改为 集群地址
FLANNEL_ETCD_PREFIX="/flannel/network"                                                   # 修改为 上面导入配置中的  /flannel/network
FLANNEL_OPTIONS="--iface=eth0"  

5、启动flannel

systemctl enable flanneld
systemctl start flanneld
systemctl status flanneld

五、修改docker配置(/usr/lib/systemd/system/docker.service)

1、增加如下配置

ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS

2、重新读取配置,并启动docker

systemctl daemon-reload
systemctl start docker

3、查看网络接口

ifconfig

如果看到 docker0 与 flannel1 已经在我们设置的IP段内了,表示已经成功

4、验证

可以在各个服务器上新建docker容器,然后通过(docker exec test1 ping 172.16.5.*)来验证

 




以上是关于跨主机容器之间通信实现方式:etcd+flanned的主要内容,如果未能解决你的问题,请参考以下文章

Docker容器跨主机通信之:直接路由方式

如何实现跨主机之间docker容器通信问题

Docker容器跨主机通信之:直接路由方式

如何实现docker跨主机之间容器通信问题(方法二)

部署Flannel,实现跨主机Docker容器通信

用flannel实现跨主机container通信