DOCKER学习_005:Flannel网络配置
Posted zyxnhr
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了DOCKER学习_005:Flannel网络配置相关的知识，希望对你有一定的参考价值。
一 简介
- Flannel是一种基于overlay网络的跨主机容器网络解决方案，也就是将TCP数据包封装在另一种网络包里面进行路由转发和通信，
- Flannel是CoreOS开发,专门用于docker多机互联的一个工具,让集群中的不同节点主机创建的容器都具有全集群唯一的虚拟ip地址
- Flannel使用go语言编写
二 Flannel实现原理
2.1原理说明
- Flannel为每个host分配一个subnet，容器从这个subnet中分配IP，这些IP可以在host间路由，容器间无需使用nat和端口映射即可实现跨主机通信
- 每个subnet都是从一个更大的IP池中划分的，flannel会在每个主机上运行一个叫flanneld的agent，其职责就是从池子中分配subnet
- Flannel使用etcd存放网络配置、已分配 的subnet、host的IP等信息
- Flannel数据包在主机间转发是由backend实现的，目前已经支持UDP、VxLAN、host-gw、AWS VPC和GCE路由等多种backend
2.2 数据转发流程
- 容器直接使用目标容器的ip访问，默认通过容器内部的eth0发送出去。
- 报文通过veth pair被发送到vethXXX。
- vethXXX是直接连接到虚拟交换机docker0的，报文通过虚拟bridge docker0发送出去。
- 查找路由表，外部容器ip的报文都会转发到flannel0虚拟网卡，这是一个P2P的虚拟网卡，然后报文就被转发到监听在另一端的flanneld。
- flanneld通过etcd维护了各个节点之间的路由表，把原来的报文UDP封装一层，通过配置的iface发送出去。
- 报文通过主机之间的网络找到目标主机。
- 报文继续往上，到传输层，交给监听在8285端口的flanneld程序处理。
- 数据被解包，然后发送给flannel0虚拟网卡。
- 查找路由表，发现对应容器的报文要交给docker0。
- docker0找到连到自己的容器，把报文发送过去。
三 Flannel安装配置
3.1 环境准备
| 节点名称 | IP地址 | 安装软件 |
| docker-server1 | 192.168.132.131 | etcd、flannel、docker |
| docker-server2 | 192.168.132.132 | flannel、docker |
删掉节点的所有容器
[root@docker-server1 ~]# docker ps -aq|xargs docker rm
[root@docker-server2 ~]# docker ps -aq|xargs docker rm
3.2 安装etcd
etcd下载地址：https://github.com/coreos/etcd/releases
下载
[root@docker-server1 ~]# wget https://github.com/coreos/etcd/releases/download/v3.3.9/etcd-v3.3.9-linux-amd64.tar.gz
[root@docker-server2 ~]# wget https://github.com/coreos/etcd/releases/download/v3.3.9/etcd-v3.3.9-linux-amd64.tar.gz
在131上安装etcd和flannel
[root@docker-server1 ~]# tar -xf etcd-v3.3.9-linux-amd64.tar.gz
[root@docker-server1 ~]# cd etcd-v3.3.9-linux-amd64
[root@docker-server1 etcd-v3.3.9-linux-amd64]# ll
total 33992 drwxr-xr-x 11 joy joy 4096 Jul 24 2018 Documentation -rwxr-xr-x 1 joy joy 18934016 Jul 24 2018 etcd -rwxr-xr-x 1 joy joy 15809280 Jul 24 2018 etcdctl -rw-r--r-- 1 joy joy 38864 Jul 24 2018 README-etcdctl.md -rw-r--r-- 1 joy joy 7262 Jul 24 2018 README.md -rw-r--r-- 1 joy joy 7855 Jul 24 2018 READMEv2-etcdctl.md
[root@docker-server1 etcd-v3.3.9-linux-amd64]# cp etcd* /usr/bin/
启动命令：
[root@docker-server1 ~]# etcd -name etcd-131 -data-dir /var/lib/etcd --advertise-client-urls http://192.168.132.131:2379,http://127.0.0.1:2379 --listen-client-urls http://192.168.132.131:2379,http://127.0.0.1:2379
-name:etc取名
-data-dir:定义数据路径
2019-11-09 14:12:36.719599 I | etcdmain: Git SHA: fca8add78 2019-11-09 14:12:36.719603 I | etcdmain: Go Version: go1.10.3 2019-11-09 14:12:36.719606 I | etcdmain: Go OS/Arch: linux/amd64 2019-11-09 14:12:36.719613 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4 2019-11-09 14:12:36.720048 I | embed: listening for peers on http://localhost:2380 2019-11-09 14:12:36.720124 I | embed: listening for client requests on 127.0.0.1:2379 2019-11-09 14:12:36.720146 I | embed: listening for client requests on 192.168.132.131:2379 2019-11-09 14:12:36.722745 I | etcdserver: name = etcd-131 2019-11-09 14:12:36.722778 I | etcdserver: data dir = /var/lib/etcd 2019-11-09 14:12:36.722783 I | etcdserver: member dir = /var/lib/etcd/member 2019-11-09 14:12:36.722787 I | etcdserver: heartbeat = 100ms 2019-11-09 14:12:36.722791 I | etcdserver: election = 1000ms 2019-11-09 14:12:36.722794 I | etcdserver: snapshot count = 100000 2019-11-09 14:12:36.722811 I | etcdserver: advertise client URLs = http://127.0.0.1:2379,http://192.168.132.131:2379 2019-11-09 14:12:36.722816 I | etcdserver: initial advertise peer URLs = http://localhost:2380 2019-11-09 14:12:36.722823 I | etcdserver: initial cluster = etcd-131=http://localhost:2380 2019-11-09 14:12:36.725597 I | etcdserver: starting member 8e9e05c52164694d in cluster cdf818194e3a8c32 2019-11-09 14:12:36.725645 I | raft: 8e9e05c52164694d became follower at term 0 2019-11-09 14:12:36.725658 I | raft: newRaft 8e9e05c52164694d [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0] 2019-11-09 14:12:36.725663 I | raft: 8e9e05c52164694d became follower at term 1 2019-11-09 14:12:36.731392 W | auth: simple token is not cryptographically signed 2019-11-09 14:12:36.732944 I | etcdserver: starting server... [version: 3.3.9, cluster version: to_be_decided] 2019-11-09 14:12:36.733497 I | etcdserver: 8e9e05c52164694d as single-node; fast-forwarding 9 ticks (election ticks 10) 2019-11-09 14:12:36.734281 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32 2019-11-09 14:12:37.635489 I | raft: 8e9e05c52164694d is starting a new election at term 1 2019-11-09 14:12:37.635568 I | raft: 8e9e05c52164694d became candidate at term 2 2019-11-09 14:12:37.635621 I | raft: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 2 2019-11-09 14:12:37.635656 I | raft: 8e9e05c52164694d became leader at term 2 2019-11-09 14:12:37.635676 I | raft: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 2 2019-11-09 14:12:37.636216 I | etcdserver: setting up the initial cluster version to 3.3 2019-11-09 14:12:37.637689 N | etcdserver/membership: set the initial cluster version to 3.3 2019-11-09 14:12:37.637846 I | etcdserver/api: enabled capabilities for version 3.3 2019-11-09 14:12:37.637990 I | etcdserver: published {Name:etcd-131 ClientURLs:[http://127.0.0.1:2379 http://192.168.132.131:2379]} to cluster cdf818194e3a8c32 2019-11-09 14:12:37.638099 I | embed: ready to serve client requests 2019-11-09 14:12:37.638861 I | embed: ready to serve client requests 2019-11-09 14:12:37.639056 E | etcdmain: forgot to set Type=notify in systemd service file? 2019-11-09 14:12:37.640375 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged! 2019-11-09 14:12:37.640424 N | embed: serving insecure client requests on 192.168.132.131:2379, this is strongly discouraged!
[root@docker-server1 ~]# ps -ef|grep etcd
root 85130 84636 2 14:12 pts/4 00:00:01 etcd -name etcd-131 -data-dir /var/lib/etcd --advertise-client-urls http://192.168.132.131:2379,http://127.0.0.1:2379 --listen-client-urls http://192.168.132.131:2379,http://127.0.0.1:2379
3.3 安装Flannel
flannel下载地址：https://github.com/coreos/flannel/releases
下载
[root@docker-server1 ~]# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
[root@docker-server1 ~]# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
root@docker-server1 ~]# tar -xf flannel-v0.11.0-linux-amd64.tar.gz
[root@docker-server1 ~]# cp flanneld /usr/bin/
[root@docker-server1 ~]# cp mk-docker-opts.sh /usr/bin/
添加flannel网络配置信息到etcd：
[root@docker-server1 ~]# etcdctl set /coreos.com/network/config ‘{"Network": "10.0.0.0/16", "SubnetLen": 24, "SubnetMin": "10.0.1.0","SubnetMax": "10.0.20.0", "Backend": {"Type": "vxlan"}}‘
如果不是本机可以加参数：--endpoints http://IP:2379
{"Network": "10.0.0.0/16", "SubnetLen": 24, "SubnetMin": "10.0.1.0","SubnetMax": "10.0.20.0", "Backend": {"Type": "vxlan"}}
Network：用于指定Flannel地址池
SubnetLen：用于指定分配给单个宿主机的docker0的ip段的子网掩码的长度
SubnetMin：用于指定最小能够分配的ip段
SudbnetMax：用于指定最大能够分配的ip段，在上面的示例中，表示每个宿主机可以分配一个24位掩码长度的子网，可以分配的子网从10.0.1.0/24到10.0.20.0/24，也就意味着在这个网段中，最多只能有20台宿主机
Backend：用于指定数据包以什么方式转发，默认为udp模式，host-gw模式性能最好，但不能跨宿主机网络
[root@docker-server1 ~]# etcdctl get /coreos.com/network/config
{"Network": "10.0.0.0/16", "SubnetLen": 24, "SubnetMin": "10.0.1.0","SubnetMax": "10.0.20.0", "Backend": {"Type": "vxlan"}}
3.4 启动Flannel
[root@docker-server1 ~]# /usr/bin/flanneld --etcd-endpoints="http://192.168.132.131:2379" --iface=192.168.132.131 --etcd-prefix=/coreos.com/network &
[root@docker-server1 ~]# I1109 15:02:01.696813 86107 main.go:450] Searching for interface using 192.168.132.131 I1109 15:02:01.698311 86107 main.go:527] Using interface with name ens33 and address 192.168.132.131 I1109 15:02:01.698335 86107 main.go:544] Defaulting external address to interface address (192.168.132.131) I1109 15:02:01.698556 86107 main.go:244] Created subnet manager: Etcd Local Manager with Previous Subnet: 10.0.4.0/24 I1109 15:02:01.698571 86107 main.go:247] Installing signal handlers I1109 15:02:01.700808 86107 main.go:386] Found network config - Backend type: vxlan I1109 15:02:01.700880 86107 vxlan.go:120] VXLAN config: VNI=1 Port=0 GBP=false DirectRouting=false I1109 15:02:01.703889 86107 local_manager.go:147] Found lease (10.0.4.0/24) for current IP (192.168.132.131), reusing I1109 15:02:01.704954 86107 main.go:317] Wrote subnet file to /run/flannel/subnet.env I1109 15:02:01.704967 86107 main.go:321] Running backend. I1109 15:02:01.705791 86107 vxlan_network.go:60] watching for new subnet leases I1109 15:02:01.709710 86107 main.go:429] Waiting for 22h59m59.994619649s to renew lease
[root@docker-server1 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:91:dd:19 brd ff:ff:ff:ff:ff:ff inet 192.168.132.131/24 brd 192.168.132.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::bcf9:af19:a325:e2c7/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:3e:dd:55:81 brd ff:ff:ff:ff:ff:ff inet 192.168.0.1/24 brd 192.168.0.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:3eff:fedd:5581/64 scope link valid_lft forever preferred_lft forever 94: br-b1c2d9c1e522: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:cd:4a:25:4f brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global br-b1c2d9c1e522 valid_lft forever preferred_lft forever inet6 fe80::42:cdff:fe4a:254f/64 scope link valid_lft forever preferred_lft forever 95: br-ec4a8380b2d3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:c9:ba:23:ae brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 brd 172.18.255.255 scope global br-ec4a8380b2d3 valid_lft forever preferred_lft forever 104: br-f42e46889a2a: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:b4:b6:35:7f brd ff:ff:ff:ff:ff:ff inet 172.22.16.1/24 brd 172.22.16.255 scope global br-f42e46889a2a valid_lft forever preferred_lft forever inet6 fe80::42:b4ff:feb6:357f/64 scope link valid_lft forever preferred_lft forever 123: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 8e:b1:37:26:b0:59 brd ff:ff:ff:ff:ff:ff inet 10.0.4.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::8cb1:37ff:fe26:b059/64 scope link
[root@docker-server1 ~]# ps -ef |grep flan
root 86107 84636 0 15:02 pts/4 00:00:00 /usr/bin/flanneld --etcd-endpoints=http://192.168.132.131:2379 --iface=192.168.132.131 --etcd-prefix=/coreos.com/network
可以使用flannel提供的脚本将subnet.env转写成Docker启动参数，创建好的启动参数默认生成在/run/docker_opts.env文件中：
[root@docker-server1 ~]# mk-docker-opts.sh
[root@docker-server1 ~]# cat /run/docker_opts.env
DOCKER_OPT_BIP="--bip=10.0.4.1/24" DOCKER_OPT_IPMASQ="--ip-masq=true" DOCKER_OPT_MTU="--mtu=1450" DOCKER_OPTS=" --bip=10.0.4.1/24 --ip-masq=true --mtu=1450"
需要把这个文件加到docker的启动项
EnvironmentFile=/run/docker_opts.env ExecStart=/usr/bin/dockerd $DOCKER_OPTS -H fd:// --containerd=/run/containerd/containerd.sock
启动docker
[root@docker-server1 ~]# systemctl daemon-reload
[root@docker-server1 ~]# systemctl restart docker
报错
[root@docker-server1 ~]# journalctl -xe -u docker
Nov 09 15:13:52 docker-server1 dockerd[86540]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following Nov 09 15:13:52 docker-server1 systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
删除/etc/docker/daemon.json里的bip配置
[root@docker-server1 ~]# systemctl restart docker
To force a start use "systemctl reset-failed docker.service" followed by "systemctl start docker.service" again.
[root@docker-server1 ~]# systemctl restart docker
[root@docker-server1 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:91:dd:19 brd ff:ff:ff:ff:ff:ff inet 192.168.132.131/24 brd 192.168.132.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::bcf9:af19:a325:e2c7/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:3e:dd:55:81 brd ff:ff:ff:ff:ff:ff inet 10.0.4.1/24 brd 10.0.4.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:3eff:fedd:5581/64 scope link valid_lft forever preferred_lft forever 94: br-b1c2d9c1e522: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:cd:4a:25:4f brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global br-b1c2d9c1e522 valid_lft forever preferred_lft forever inet6 fe80::42:cdff:fe4a:254f/64 scope link valid_lft forever preferred_lft forever 95: br-ec4a8380b2d3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:c9:ba:23:ae brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 brd 172.18.255.255 scope global br-ec4a8380b2d3 valid_lft forever preferred_lft forever 104: br-f42e46889a2a: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:b4:b6:35:7f brd ff:ff:ff:ff:ff:ff inet 172.22.16.1/24 brd 172.22.16.255 scope global br-f42e46889a2a valid_lft forever preferred_lft forever inet6 fe80::42:b4ff:feb6:357f/64 scope link valid_lft forever preferred_lft forever 123: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 8e:b1:37:26:b0:59 brd ff:ff:ff:ff:ff:ff inet 10.0.4.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::8cb1:37ff:fe26:b059/64 scope link valid_lft forever preferred_lft forever
docker0的IP是10.0.4.1
可以看到flannel0网卡的地址和etcd存储的地址一样，这样flannel网络配置完成
Flannel启动过程解析：
- 从etcd中获取network的配置信息
- 划分subnet，并在etcd中进行注册
- 将子网信息记录到/run/flannel/subnet.env中
- Flannel必须先于Docker启动
3.5 验证Flannel网络
查看etcd中的数据：
[root@docker-server1 ~]# etcdctl ls /coreos.com/network/subnets
/coreos.com/network/subnets/10.0.4.0-24
3.6 配置Docker
Docker安装完成以后，需要修改其启动参数以使其能够使用flannel进行IP分配，以及网络通讯
在Flannel运行之后，会生成一个环境变量文件，包含了当前主机要使用flannel通讯的相关参数，如下：
[root@docker-server1 ~]# cat /run/flannel/subnet.env
FLANNEL_NETWORK=10.0.0.0/16 FLANNEL_SUBNET=10.0.4.1/24 FLANNEL_MTU=1450 FLANNEL_IPMASQ=false
docker-server1相同操作配置flannel，不用安装etcd
[root@docker-server2 ~]# tar -xf flannel-v0.11.0-linux-amd64.tar.gz
[root@docker-server2 ~]# mv flanneld /usr/bin/
[root@docker-server2 ~]# mv mk-docker-opts.sh /usr/bin/
[root@docker-server2 ~]# /usr/bin/flanneld --etcd-endpoints="http://192.168.132.131:2379" --iface=192.168.132.132 --etcd-prefix=/coreos.com/network &
[root@docker-server2 ~]# mk-docker-opts.sh -c
[root@docker-server2 ~]# cat /run/docker_opts.env
修改docker启动文件，删除daemon.json的bip配置
[root@docker-server2 ~]# systemctl daemon-reload
[root@docker-server2 ~]# systemctl restart docker
[root@docker-server2 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:63:fd:11 brd ff:ff:ff:ff:ff:ff inet 192.168.132.132/24 brd 192.168.132.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::6a92:62ba:1b33:c93d/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:50:67:ff:90 brd ff:ff:ff:ff:ff:ff inet 10.0.18.1/24 brd 10.0.18.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:50ff:fe67:ff90/64 scope link valid_lft forever preferred_lft forever 14: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 62:b9:76:44:bf:32 brd ff:ff:ff:ff:ff:ff inet 10.0.18.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::60b9:76ff:fe44:bf32/64 scope link valid_lft forever preferred_lft forever
3.7 验证容器互通
[root@docker-server1 ~]# docker run -it busybox
/ # ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 124: eth0@if125: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue link/ether 02:42:0a:00:04:02 brd ff:ff:ff:ff:ff:ff inet 10.0.4.2/24 brd 10.0.4.255 scope global eth0 valid_lft forever preferred_lft forever
[root@docker-server2 ~]# docker run -it busybox
/ # ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 15: eth0@if16: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue link/ether 02:42:0a:00:12:02 brd ff:ff:ff:ff:ff:ff inet 10.0.18.2/24 brd 10.0.18.255 scope global eth0 valid_lft forever preferred_lft forever
互ping
192.168.132.131容器 / # ping 10.0.18.2 PING 10.0.18.2 (10.0.18.2): 56 data bytes 64 bytes from 10.0.18.2: seq=0 ttl=62 time=2.517 ms 64 bytes from 10.0.18.2: seq=1 ttl=62 time=1.058 ms 64 bytes from 10.0.18.2: seq=2 ttl=62 time=1.676 ms 192.168.132.132容器 PING 10.0.4.2 (10.0.4.2): 56 data bytes 64 bytes from 10.0.4.2: seq=0 ttl=62 time=2.606 ms 64 bytes from 10.0.4.2: seq=1 ttl=62 time=1.727 ms
两主机的容器可以互通
此时的网络数据包流向如图：
3.8 配置backend为host-gwhost-gw bakcend是flannel的另一个backend。与vxlan不同，host-gw不会封装数据包，而是在主机的路由表中创建到其他主机的subnet的路由条目，从而实现容器网络跨主机通信。需要说明的是，host-gw不能跨宿主机网络通信，或者说跨宿主机网络通信需要物理路由支持。
修改etcd如下：
[root@docker-server1 ~]# etcdctl --endpoints http://127.0.0.1:2379 set /coreos.com/network/config ‘{"Network": "10.0.0.0/16", "SubnetLen": 24, "SubnetMin": "10.0.1.0","SubnetMax": "10.0.20.0", "Backend": {"Type": "host-gw"}}‘
{"Network": "10.0.0.0/16", "SubnetLen": 24, "SubnetMin": "10.0.1.0","SubnetMax": "10.0.20.0", "Backend": {"Type": "host-gw"}
重启flanneld与docker：
root@docker-server1 ~]# kill -9 86780
[root@docker-server1 ~]# /usr/bin/flanneld --etcd-endpoints="http://192.168.132.131:2379" --iface=192.168.132.131 --etcd-prefix=/coreos.com/network &
[root@docker-server1 ~]# systemctl restart docker
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:3e:dd:55:81 brd ff:ff:ff:ff:ff:ff inet 10.0.4.1/24 brd 10.0.4.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:3eff:fedd:5581/64 scope link valid_lft forever preferred_lft forever 123: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 8e:b1:37:26:b0:59 brd ff:ff:ff:ff:ff:ff inet 10.0.4.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::8cb1:37ff:fe26:b059/64 scope link valid_lft forever preferred_lft forever
[root@docker-server2 ~]# kill -9 74050
[root@docker-server2 ~]# /usr/bin/flanneld --etcd-endpoints="http://192.168.132.131:2379" --iface=192.168.132.132 --etcd-prefix=/coreos.com/network &
[root@docker-server2 ~]# systemctl restart docker
[root@docker-server2 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:63:fd:11 brd ff:ff:ff:ff:ff:ff inet 192.168.132.132/24 brd 192.168.132.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::6a92:62ba:1b33:c93d/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:50:67:ff:90 brd ff:ff:ff:ff:ff:ff inet 10.0.18.1/24 brd 10.0.18.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:50ff:fe67:ff90/64 scope link valid_lft forever preferred_lft forever 14: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 62:b9:76:44:bf:32 brd ff:ff:ff:ff:ff:ff inet 10.0.18.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::60b9:76ff:fe44:bf32/64 scope link valid_lft forever preferred_lft forever
可以在宿主机上查看到路由条目：
[root@docker-server1 ~]# ip route
default via 192.168.132.2 dev ens33 proto static metric 100 10.0.0.0/16 dev flannel.1 10.0.4.0/24 dev docker0 proto kernel scope link src 10.0.4.1 10.0.18.0/24 via 192.168.132.132 dev ens33 172.17.0.0/16 dev br-b1c2d9c1e522 proto kernel scope link src 172.17.0.1 172.18.0.0/16 dev br-ec4a8380b2d3 proto kernel scope link src 172.18.0.1 172.22.16.0/24 dev br-f42e46889a2a proto kernel scope link src 172.22.16.1 192.168.132.0/24 dev ens33 proto kernel scope link src 192.168.132.131 metric 100
博主声明：本文的内容来源主要来自誉天教育晏威老师，由本人实验完成操作验证，需要的博友请联系誉天教育（http://www.yutianedu.com/），获得官方同意或者晏老师（https://www.cnblogs.com/breezey/）本人同意即可转载，谢谢！
以上是关于DOCKER学习_005:Flannel网络配置的主要内容，如果未能解决你的问题，请参考以下文章
Docker 学习笔记 Docker 仓库数据卷数据卷容器，网络基础实操。高级网络配置学习
大数据技术之_16_Scala学习_11_客户信息管理系统+并发编程模型 Akka+Akka 网络编程-小黄鸡客服案例+Akka 网络编程-Spark Master Worker 进程通讯项目(示例代
大数据技术之_03_Hadoop学习_02_入门_Hadoop运行模式+本地运行模式+伪分布式运行模式+完全分布式运行模式(开发重点)+Hadoop编译源码(面试重点)+常见错误及解决方案(示例代(代