Docker成神之路——浅谈DockerOverlay网络
Posted 守夜人爱吃兔子
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Docker成神之路——浅谈DockerOverlay网络相关的知识，希望对你有一定的参考价值。
DockerOverlay网络
Overlay是一种覆盖在主机上的虚拟网络，我理解的overlay他需要借助插件或工具完成对数据的保存和转发。我们这里使用etcd来完成overlay的跨主机容器通信
1. 安装etcd（这里我只启动两个节点的etcd,也可单机模式）
我这里选择用安装包方式，你也可以选择yum
wget https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz
scp etcd-v3.5.0-linux-amd64.tar.gz root@172.16.3.142:/root以下操作两个主机均需要进行
tar -zxvf etcd-v3.5.0-linux-amd64.tar.gz
cd etcd-v3.5.0-linux-amd64
cp etcdctl /usr/local/bin/
我这里遇到一个坑，一定要加上--enable-v2否则会报错，无法创建overlay网络
nohup ./etcd --name docker-node1 --initial-advertise-peer-urls http://172.16.3.141:2380 --listen-peer-urls http://172.16.3.141:2380 --listen-client-urls http://172.16.3.141:2379,http://127.0.0.1:2379 --advertise-client-urls http://172.16.3.141:2379 --initial-cluster-token etcd-cluster --initial-cluster docker-node1=http://172.16.3.141:2380,docker-node2=http://172.16.3.142:2380 --enable-v2 --initial-cluster-state new 2>&1 &
nohup ./etcd --name docker-node2 --initial-advertise-peer-urls http://172.16.3.142:2380 --listen-peer-urls http://172.16.3.142:2380 --listen-client-urls http://172.16.3.142:2379,http://127.0.0.1:2379 --advertise-client-urls http://172.16.3.142:2379 --initial-cluster-token etcd-cluster --initial-cluster docker-node1=http://172.16.3.141:2380,docker-node2=http://172.16.3.142:2380 --enable-v2 --initial-cluster-state new 2>&1 &
etcdctl member list 查看一下节点信息分别添加配置，指定外部存储使用etcd
vi /etc/docker/daemon.json
{
"cluster-store":"etcd://172.16.3.141:2379",
"cluster-advertise":"172.16.3.141:2375"
}一定要重启下
systemctl daemon-reload
service docker restart2. 创建overlay network
# 也可指定subnet
docker network create -d overlay etcdnet在两天主机分别执行,可以看到另一台也出现了etcdnet
docker network ls
NETWORK ID NAME DRIVER SCOPE
810ca8a6e002 bridge bridge local
19234a37ec11 etcdnet overlay global我们来看一下etcd中存储的数据格式,可以看到节点信息和子网的配置都有了
[root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls
/docker
[root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker
/docker/nodes
/docker/network
[root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker/nodes
/docker/nodes/172.16.3.141:2375
/docker/nodes/172.16.3.142:2375
[root@localhost etcd-v3.5.0-linux-amd64]# etcdctl ls /docker/network/v1.0
/docker/network/v1.0/endpoint
/docker/network/v1.0/network
/docker/network/v1.0/endpoint_count
/docker/network/v1.0/ipam
/docker/network/v1.0/idm
/docker/network/v1.0/overlay
[root@localhost etcd-v3.5.0-linux-amd64]# etcdctl get /docker/network/v1.0/overlay/network/19234a37ec11b48a6d1491d99264825ee6452fc3007d0841429ec5ca85c94d64
{"mtu":0,"secure":false,"subnets":[{"SubnetIP":"10.0.9.0/24","GwIP":"10.0.9.1/24","Vni":0}]}3. 使用overlay network创建容器
在141上启动busybox0 在142上启动busybox1
docker run -it -d --name busybox0 --network etcdnet busybox
docker run -it -d --name busybox1 --network etcdnet busybox查看下两个节点的ip分配情况
docker network inspect etcdnet
"Containers": {
"67845aca4909ec68518ee09a5de72c12472bdd5ab044c3df180df85012b381e8": {
"Name": "busybox0",
"EndpointID": "e90dff8605bbe7c7f7db508a6b576d738ddb5faf1e7564191c068134c8c79312",
"MacAddress": "02:42:0a:00:09:02",
"IPv4Address": "10.0.9.2/24",
"IPv6Address": ""
},
"ep-fc9be25e3f8d99df596d59e283fd0237834bb6ace2b72f3182181f132fd51bbd": {
"Name": "busybox1",
"EndpointID": "fc9be25e3f8d99df596d59e283fd0237834bb6ace2b72f3182181f132fd51bbd",
"MacAddress": "02:42:0a:00:09:03",
"IPv4Address": "10.0.9.3/24",
"IPv6Address": ""
}
}我们进入容器看一下网卡信息，发现有两个网卡，这里eth0是overlay的eth1是bridge，从上篇文章的docker0可知默认的bridgeIp不应该是172.17.0.1网段上分配的吗？172.18.0.2又是哪里来的？
[root@localhost etcd-v3.5.0-linux-amd64]# docker exec -it busybox0 sh -c 'ifconfig'
eth0 Link encap:Ethernet HWaddr 02:42:0A:00:09:02
inet addr:10.0.9.2 Bcast:10.0.9.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth1 Link encap:Ethernet HWaddr 02:42:AC:12:00:02
inet addr:172.18.0.2 Bcast:172.18.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:836 (836.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)我们在运行该容器的主机上查看一下network,发现突然多了一个网络配置叫docker_gwbridge是一个bridge类型的
[root@localhost etcd-v3.5.0-linux-amd64]# docker network ls
NETWORK ID NAME DRIVER SCOPE
cd915be3ae23 bridge bridge local
84d1aff11b90 docker_gwbridge bridge local
19234a37ec11 etcdnet overlay global看一下docker_gwbridge对应的详细信息,和容器内的eth1是可以对应上的
docker network inspect 84d1aff11b90
"Containers": {
"67845aca4909ec68518ee09a5de72c12472bdd5ab044c3df180df85012b381e8": {
"Name": "gateway_a6711fa937d4",
"EndpointID": "4ec1a016078d849ae46012c54a712caa7cc64312ebbd78facaecb15e387bda21",
"MacAddress": "02:42:ac:12:00:02",
"IPv4Address": "172.18.0.2/16",
"IPv6Address": ""
}
},4. overlay network测试
在141机器上运行的busbox0去ping142机器上运行的busbox1,可以看到无论是ping容器名还是ip都可以
[root@docker01 etcd-v3.5.0-linux-amd64]# docker exec -it busybox0 sh -c 'ping busybox1'
PING busybox1 (10.0.9.3): 56 data bytes
64 bytes from 10.0.9.3: seq=0 ttl=64 time=0.499 ms
64 bytes from 10.0.9.3: seq=1 ttl=64 time=0.902 ms
64 bytes from 10.0.9.3: seq=2 ttl=64 time=1.757 ms
[root@docker01 etcd-v3.5.0-linux-amd64]# docker exec -it busybox0 sh -c 'ping 10.0.9.3'
PING 10.0.9.3 (10.0.9.3): 56 data bytes
64 bytes from 10.0.9.3: seq=0 ttl=64 time=0.684 ms
64 bytes from 10.0.9.3: seq=1 ttl=64 time=1.390 ms反过来试试也是没问题的
[root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it busybox1 sh -c 'ping busybox0'
PING busybox0 (10.0.9.2): 56 data bytes
64 bytes from 10.0.9.2: seq=0 ttl=64 time=0.563 ms
64 bytes from 10.0.9.2: seq=1 ttl=64 time=0.959 ms
[root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it busybox1 sh -c 'ping 10.0.9.2'
PING 10.0.9.2 (10.0.9.2): 56 data bytes
64 bytes from 10.0.9.2: seq=0 ttl=64 time=0.638 ms至于为什么能解析容器名为ip呢，这也很容易想到，这些信息etcd中都有，创建的时候添加一条规则即可
[root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it busybox1 sh -c 'cat /etc/hosts'
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
10.0.9.3 8139c23b0e575. overly总结
最后测试阶段发现容器互相ping是ping不通的，我很奇怪，去查了一些资料似乎没什么结果，最后升级了一下内核版本从3.10升级到了5.12.13-1.el7.elrepo.x86_64就可以了
6. 没想到吧总结之后还有内容,哎我就是玩
从上一篇docker网络浅析中我们知道网桥模式的玩法，我们这里尝试不使用任何插件来实现跨主机容器互联
首先我们先改一下两台机器的docker0的ip地址,默认是172.17.0.1,不改的话就导致都路由到一个网段了
vi /etc/docker/daemon.json
"bip":"172.17.3.100/24"vi /etc/docker/daemon.json
"bip":"172.17.4.100/24"两天机器分别都改一下然后刷新重启
//主机141
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.3.100 netmask 255.255.255.0 broadcast 172.25.0.255
inet6 fe80::42:5aff:fe07:e6d6 prefixlen 64 scopeid 0x20<link>
//主机142
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.4.100 netmask 255.255.255.0 broadcast 172.25.0.255
inet6 fe80::42:5aff:fe07:e6d6 prefixlen 64 scopeid 0x20<link>然后我们添加两条路由规则
//主机141
route add -net 172.17.4.0 netmask 255.255.255.0 gw 172.16.3.142
//主机142
route add -net 172.17.3.0 netmask 255.255.255.0 gw 172.16.3.141
//防止丢包两边分别加上
iptables -I DOCKER --dst 172.17.0.0/16 -j ACCEPT我们尝试从141机器的172.17.3.0网段的容器，请求142机器的172.17.4.0网段的容器，看看能否成功
//141
docker run -it -d --name t1 busybox -> ip = 172.17.3.1
//142
docker run -it -d --name t2 busybox -> ip = 172.17.4.1[root@docker01 etcd-v3.5.0-linux-amd64]# docker exec -it t1 sh -c 'ping 172.17.4.1'
PING 172.17.4.1 (172.17.4.1): 56 data bytes
64 bytes from 172.17.4.1: seq=0 ttl=62 time=2.386 ms
64 bytes from 172.17.4.1: seq=1 ttl=62 time=2.061 ms
[root@docker02 etcd-v3.5.0-linux-amd64]# docker exec -it t2 sh -c 'ping 172.17.3.1'
PING 172.17.3.1 (172.17.3.1): 56 data bytes
64 bytes from 172.17.3.1: seq=0 ttl=62 time=1.144 ms
64 bytes from 172.17.3.1: seq=1 ttl=62 time=1.606 ms最后
最近我整理了整套《JAVA核心知识点总结》,说实话 ,作为一名Java程序员,不论你需不需要面试都应该好好看下这份资料。拿到手总是不亏的~我的不少粉丝也因此拿到腾讯字节快手等公司的Offer
进【Java进阶之路群】，找管理员获取哦-！
以上是关于Docker成神之路——浅谈DockerOverlay网络的主要内容，如果未能解决你的问题，请参考以下文章