用 Flannel 配置 Kubernetes 网络
Posted CSDN云计算
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了用 Flannel 配置 Kubernetes 网络相关的知识,希望对你有一定的参考价值。
在Container Cloud环境中,跨机器的容器互连一直是讨论的热点;Kubernetes的官方网站也提供了几种备选方案,如 L2网络, Flannel, OpenVSwitch等。本文主要描述Flannel在Kubernetes网络中的应用。
Flannel
编译Flannel:
Flannel的编译比较简单,从github上clone后直接build即可;但目前Flannel 仅支持Linux,在其它平台编译时会报错。
git clone https://github.com/coreos/flannel.gitcd flannel
./build
安装并启动 Etcd
git clone https://github.com/coreos/etcd.git./build
setsid ./etcd --listen-client-urls http://0.0.0.0:4001 --advertise-client-urls http://k82demo1:4001 2>&1 > etcd.log &
配置Flannel
etcdctl set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }'
启动 Flannel 和 Docker:
配置完成后即可启动Flannel和Docker。Flannel会与Etcd通信确定当前机器的子网,并将子网信息写入 /run/flannel/subnet.env;为了让Docker使用Flannel配置的子网,需要重启Docker并配置 –bip [4]。
sudo setsid flanneld -etcd-endpoints http://k82demo1:4001 >flannel.log 2>&1 &source /run/flannel/subnet.envsudo ifconfig docker0 ${FLANNEL_SUBNET}sudo setsid docker daemon --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} >docker.log 2>&1 &
验证
首先查看Flannel是否启动正确;Flannel启动后增加如下路由规则。然后可以启动两个ubuntu容器来验证跨主机的通信。
#route -nKernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface0.0.0.0 9.21.58.2 0.0.0.0 UG 0 0 0 eth09.21.58.0 0.0.0.0 255.255.255.0 U 0 0 0 eth010.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 cbr010.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 flannel010.1.13.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
# hostname -i9.21.58.21#docker run -i -t ubuntu /bin/bashroot@5b45228a7686:/# ifconfigeth0 Link encap:Ethernet HWaddr 02:42:0a:01:0d:02
inet addr:10.1.13.2 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::42:aff:fe01:d02/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1472 Metric:1
RX packets:5 errors:0 dropped:0 overruns:0 frame:0
TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:418 (418.0 B) TX bytes:418 (418.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
# ping 10.1.42.6PING 10.1.42.6 (10.1.42.6) 56(84) bytes of data.64 bytes from 10.1.42.6: icmp_seq=1 ttl=60 time=0.594 ms64 bytes from 10.1.42.6: icmp_seq=2 ttl=60 time=0.483 ms64 bytes from 10.1.42.6: icmp_seq=3 ttl=60 time=0.576 ms64 bytes from 10.1.42.6: icmp_seq=4 ttl=60 time=0.490 ms64 bytes from 10.1.42.6: icmp_seq=5 ttl=60 time=0.524 ms# hostname -i9.21.58.85root@e9b8c093d4cc:/# ifconfigeth0 Link encap:Ethernet HWaddr 02:42:0a:01:2a:06
inet addr:10.1.42.6 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::42:aff:fe01:2a06/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1472 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:648 (648.0 B) TX bytes:648 (648.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Kubernetes
Kubernetes的配置在网上的很多文章都有介绍,这里就不重复;仅列出启动命令,并解释与Flannel相关的参数。
启动Master
setsid kube-apiserver --allow-privileged=true --address=k82demo1 --etcd-servers=http://k82demo1:4001 --service-cluster-ip-range=10.1.0.0/16 --port=8888 2>&1 >api.log &setsid kube-controller-manager --master=http://k82demo1:8888 --allocate-node-cidrs=true --cluster-cidr=10.1.0.0/16 2>&1 >ctrl.log &setsid kube-scheduler --address=k82demo1 --master=http://k82demo1:8888 2>&1 >sched.log &
在启动kube-apiserver时,–service-cluster-ip-range指定了Service的子网信息,并不一定与Flannel的配置一至。在启动kube-controller-manager时,–allocate-node-cidrs=true需要配置,因为Flannel是通过 cidr 进行配置。
启动Agent
setsid kubelet --api-servers=http://k82demo1:8888 >let.log 2>&1 &setsid kube-proxy --master=http://k82demo1:8888 >proxy.log 2>&1 &
虽然Flannel已经将各个主机上的容器连接起来,但是kube-proxy对Service的配置还是需要的;在后面的测试环节会进行相应的介绍。
测试
搭建好环境后,主要针对Kubernetes中的Service进行相应的测试及原理的介绍;在后续后文章会加入 DNS及反向代理的介绍。
ReplicaController
为了创建Service,先通过ReplicaController创建两个Pods;命令如下:
kubectl run my-nginx --image=nginx --replicas=2 --port=80
# kubectl describe po my-nginx-jw91n-ponfaName: my-nginx-jw91n-ponfaNamespace: default
Image(s): nginxNode: ma1demo5/9.21.58.85Start Time: Fri, 01 Apr 2016 05:22:08 -0400Labels: pod-template-hash=3800858182,run=my-nginxStatus: RunningReason:Message:IP: 10.1.42.2Replication Controllers: <none>Containers:
my-nginx:
Container ID: docker://47e3f70cc3614b0f1728fe70fe46d14e648a5df4ee783149ec1589bdeee3561d
Image: nginx
Image ID: docker://6f62f48c4e55d700cf3eb1b5e33fa051802986b77b874cc351cce539e5163707
State: Running
Started: Fri, 01 Apr 2016 05:22:33 -0400
Ready: True
Restart Count: 0
Environment Variables:Conditions:
Type Status
Ready True
No volumes.
No events.# etcdctl ls -r | grep subnets/coreos.com/network/subnets
/coreos.com/network/subnets/10.1.42.0-24/coreos.com/network/subnets/10.1.70.0-24# etcdctl get /coreos.com/network/subnets/10.1.70.0-24{"PublicIP":"9.21.58.21"}
Service
Kubernetes提供Service功能,为其它组件提供了一个相对稳定的服务;在一个Service中可以有多个Pods同时工作,当访问Service时,请求会被重定向到其中的一个Pod中从而提高Service的稳定性。
基于上面的Nginx Pods,通过以下配置创建一个Nginx的Service。
# cat nginx_svc.json{ "kind": "Service", "apiVersion": "v1", "metadata": { "name": "my-nginx-serv"
}, "spec": { "selector": { "run": "my-nginx"
}, "ports": [
{ "protocol": "TCP", "port": 8080, "targetPort": 80
}
]
}
}# kubectl create -f nginx_svc.json
Service创建后,可以通过kubectl describe查看Service的基本信息。
# kubectl describe svc my-nginx-servName: my-nginx-servNamespace: defaultLabels: <none>Selector: run=my-nginxType: ClusterIPIP: 10.1.59.126Port: <unnamed> 8080/TCPEndpoints: 10.1.42.2:80,10.1.42.3:80Session Affinity: None
No events.
# curl 10.1.59.126:8080<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p><p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p></body></html>
虚拟地址
# iptables-save | grep default/my-nginx-serv
-A KUBE-SEP-3KJD4NXGHJJSNEUV -s 10.1.42.3/32 -m comment --comment "default/my-nginx-serv:" -j KUBE-MARK-MASQ-A KUBE-SEP-3KJD4NXGHJJSNEUV -p tcp -m comment --comment "default/my-nginx-serv:" -m tcp -j DNAT --to-destination 10.1.42.3:80-A KUBE-SEP-XMTMJD5DZAOI47WT -s 10.1.42.2/32 -m comment --comment "default/my-nginx-serv:" -j KUBE-MARK-MASQ-A KUBE-SEP-XMTMJD5DZAOI47WT -p tcp -m comment --comment "default/my-nginx-serv:" -m tcp -j DNAT --to-destination 10.1.42.2:80-A KUBE-SERVICES -d 10.1.59.126/32 -p tcp -m comment --comment "default/my-nginx-serv: cluster IP" -m tcp --dport 8080 -j KUBE-SVC-SIE4CNHSX37MXAT2-A KUBE-SVC-SIE4CNHSX37MXAT2 -m comment --comment "default/my-nginx-serv:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-XMTMJD5DZAOI47WT-A KUBE-SVC-SIE4CNHSX37MXAT2 -m comment --comment "default/my-nginx-serv:" -j KUBE-SEP-3KJD4NXGHJJSNEUV
结语
本文主要介绍了Flannel在Kubernetes网络中的使用;虽然Flannel为Kubernetes提供了Overlay网络,但是为了使Docker可以使用该网络,Docker需要在Flannel之后启动并进行相应的配置。需要借助主机上Systemd进行这种依赖关系的管理。在后续的文章中会对Kubernetes DNS,及Flannel/Docker在生产环境中的部署进行介绍。
引用
[1]
[2]
[3]
[4]
以上是关于用 Flannel 配置 Kubernetes 网络的主要内容,如果未能解决你的问题,请参考以下文章