ETCD：在容器中运行etcd集群

Posted 2020-11-22

原文地址：Docker container
以下指南显示了如何使用静态引导过程在rkt和Docker上运行etcd。

rkt

---

运行单节点的etcd
以下rkt run命令将在端口2379上公开etcd客户端API，并在端口2380上公开对等API。

配置etcd时使用主机IP地址。

export NODE1=192.168.1.21

信任CoreOS App签名密钥。

sudo rkt trust --prefix quay.io/coreos/etcd
# gpg key fingerprint is: 18AD 5014 C99E F7E3 BA5F  6CE9 50BD D3E0 FC8A 365E

运行etcd v3.2版本或指定其他发行版本。

sudo rkt run --net=default:IP=${NODE1} quay.io/coreos/etcd:v3.2 -- -name=node1 -advertise-client-urls=http://${NODE1}:2379 -initial-advertise-peer-urls=http://${NODE1}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE1}:2380 -initial-cluster=node1=http://${NODE1}:2380

列出集群成员：

etcdctl --endpoints=http://192.168.1.21:2379 member list

运行3个节点的etcd
使用-initial-cluster参数在本地使用rkt设置3节点集群。

export NODE1=172.16.28.21
export NODE2=172.16.28.22
export NODE3=172.16.28.23

# node 1
sudo rkt run --net=default:IP=${NODE1} quay.io/coreos/etcd:v3.2 -- -name=node1 -advertise-client-urls=http://${NODE1}:2379 -initial-advertise-peer-urls=http://${NODE1}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE1}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380

# node 2
sudo rkt run --net=default:IP=${NODE2} quay.io/coreos/etcd:v3.2 -- -name=node2 -advertise-client-urls=http://${NODE2}:2379 -initial-advertise-peer-urls=http://${NODE2}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE2}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380

# node 3
sudo rkt run --net=default:IP=${NODE3} quay.io/coreos/etcd:v3.2 -- -name=node3 -advertise-client-urls=http://${NODE3}:2379 -initial-advertise-peer-urls=http://${NODE3}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE3}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380

验证集群是否健康并且可以访问。

ETCDCTL_API=3 etcdctl --endpoints=http://172.16.28.21:2379,http://172.16.28.22:2379,http://172.16.28.23:2379 endpoint health

DNS

通过本地解析器已知的DNS名称引用对等方的生产群集必须安装主机的DNS配置。

Docker

为了向Docker主机外部的客户端公开etcd API，请使用容器的主机IP地址。 请参阅docker inspect了解有关如何获取IP地址的更多详细信息。 或者，为docker run命令指定--net = host标志，以跳过将容器放置在单独的网络堆栈内的操作。
运行单节点的etcd
适用主机Ip地址配置etcd：

export NODE1=192.168.1.21

配置Docker卷存储etcd数据:

docker volume create --name etcd-data
export DATA_DIR="etcd-data"

运行最新版本的etcd：

REGISTRY=quay.io/coreos/etcd
# available from v3.2.5
REGISTRY=gcr.io/etcd-development/etcd

docker run   -p 2379:2379   -p 2380:2380   --volume=${DATA_DIR}:/etcd-data   --name etcd ${REGISTRY}:latest   /usr/local/bin/etcd   --data-dir=/etcd-data --name node1   --initial-advertise-peer-urls http://${NODE1}:2380 --listen-peer-urls http://0.0.0.0:2380   --advertise-client-urls http://${NODE1}:2379 --listen-client-urls http://0.0.0.0:2379   --initial-cluster node1=http://${NODE1}:2380

列出集群成员：

etcdctl --endpoints=http://${NODE1}:2379 member list

运行3个节点的etcd

REGISTRY=quay.io/coreos/etcd
# available from v3.2.5
REGISTRY=gcr.io/etcd-development/etcd

# For each machine
ETCD_VERSION=latest
TOKEN=my-etcd-token
CLUSTER_STATE=new
NAME_1=etcd-node-0
NAME_2=etcd-node-1
NAME_3=etcd-node-2
HOST_1=10.20.30.1
HOST_2=10.20.30.2
HOST_3=10.20.30.3
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
DATA_DIR=/var/lib/etcd

# For node 1
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
docker run   -p 2379:2379   -p 2380:2380   --volume=${DATA_DIR}:/etcd-data   --name etcd ${REGISTRY}:${ETCD_VERSION}   /usr/local/bin/etcd   --data-dir=/etcd-data --name ${THIS_NAME}   --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380   --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379   --initial-cluster ${CLUSTER}   --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

# For node 2
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
docker run   -p 2379:2379   -p 2380:2380   --volume=${DATA_DIR}:/etcd-data   --name etcd ${REGISTRY}:${ETCD_VERSION}   /usr/local/bin/etcd   --data-dir=/etcd-data --name ${THIS_NAME}   --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380   --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379   --initial-cluster ${CLUSTER}   --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

# For node 3
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
docker run   -p 2379:2379   -p 2380:2380   --volume=${DATA_DIR}:/etcd-data   --name etcd ${REGISTRY}:${ETCD_VERSION}   /usr/local/bin/etcd   --data-dir=/etcd-data --name ${THIS_NAME}   --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380   --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379   --initial-cluster ${CLUSTER}   --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

适用版本v3的etcdctl：

docker exec etcd /bin/sh -c "export ETCDCTL_API=3 && /usr/local/bin/etcdctl put foo bar"

Bare Metal

---

要在裸机上配置3节点etcd集群，裸机存储库中的示例可能会有用。

挂载一个证书卷：

etcd发布容器不包含默认的根证书。 要将HTTPS与受根权限信任的证书一起使用（例如，用于发现），请将证书目录安装到etcd容器中：

REGISTRY=quay.io/coreos/etcd
# available from v3.2.5
REGISTRY=docker://gcr.io/etcd-development/etcd

rkt run   --insecure-options=image   --volume etcd-ssl-certs-bundle,kind=host,source=/etc/ssl/certs/ca-certificates.crt   --mount volume=etcd-ssl-certs-bundle,target=/etc/ssl/certs/ca-certificates.crt   ${REGISTRY}:latest -- --name my-name   --initial-advertise-peer-urls http://localhost:2380 --listen-peer-urls http://localhost:2380   --advertise-client-urls http://localhost:2379 --listen-client-urls http://localhost:2379   --discovery https://discovery.etcd.io/c11fbcdc16972e45253491a24fcf45e1

REGISTRY=quay.io/coreos/etcd
# available from v3.2.5
REGISTRY=gcr.io/etcd-development/etcd

docker run   -p 2379:2379   -p 2380:2380   --volume=/etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt   ${REGISTRY}:latest   /usr/local/bin/etcd --name my-name   --initial-advertise-peer-urls http://localhost:2380 --listen-peer-urls http://localhost:2380   --advertise-client-urls http://localhost:2379 --listen-client-urls http://localhost:2379   --discovery https://discovery.etcd.io/86a9ff6c8cb8b4c4544c1a2f88f8b801