centos搭建k8s集群,安装Docker,, Harbor, etcd
Posted 民工黑猫
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了centos搭建k8s集群,安装Docker,, Harbor, etcd相关的知识,希望对你有一定的参考价值。
【前期准备】
下载 etcd 二进制安装包:https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz
下载 kubernetes 1.18.3 二进制安装包:https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1183
注意:打开链接有很多下载包,包含kubernetes-client 、kubernetes-server 、kubernetes-node ,下载其中一个64位的就行。
安装证书签发工具cfssl |
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
cp cfssl_linux-amd64 /usr/local/bin/cfssl
cp cfssljson_linux-amd64 /usr/local/bin/cfssljson
cp cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
下载etcd安装包 |
下载etcd-v3.4.9二进制包
下载etcd-v.3.4.9二进制安装包,
其它版本下载地址: https://github.com/etcd-io/etcd/tags
wget https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz
下载完成后,解压etcd安装包,复制解压目录里的etcd与etcdctl 两个文件到 /usr/local/bin 目录下面,并且赋予两个文件可执行权限
tar xfv etcd-v3.4.9-linux-amd64.tar.gz cd etcd-v3.4.9-linux-amd64 cp etcd /usr/local/bin cp etcdctl /usr/local/bin #赋予执行权限 chmod +x /usr/local/bin/etcd chmod +x /usr/local/bin/etcdctl
签发etcd证书 |
创建证书存放目录。
mkdir -p /opt/certs
【创建证书】
首先安装cfssl 证书制作工具,安装方法参考:https://www.cnblogs.com/yyee/p/13189331.html
在etcd01 (192.168.0.102)节点上创建证书。
(1) 创建根证书配置文件
vi /opt/certs/ca-config.json
{
"signing": {
"default": {
"expiry": "175200h"
},
"profiles":{
"k8s-server": {
"expiry": "175200h",
"usages": [
"signing",
"key encipherment",
"server auth"
]
},
"k8s-client": {
"expiry": "175200h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
},
"k8s-server-client": {
"expiry": "175200h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
signing:表示该证书可用于签名其它证书(生成的 ca.pem 证书中 CA=TRUE);
{
"CN": "k8s",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing" ,
"O": "k8s",
"OU": "system"
}
]
}
CN:Common Name:kube-apiserver 从证书中提取该字段作为请求的用户名 (User Name),
(3) 创建etcd证书请求文件etcd-peer-csr.json
vi /opt/certs/etcd-peer-csr.json
{
"CN": "k8s-etcd",
"hosts": [
"192.168.0.101",
"192.168.0.102",
"192.168.0.103",
"192.168.0.104",
"192.168.0.105",
"192.168.0.106"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Beijing",
"L": "Beijing",
"O": "k8s",
"OU": "system"
}
]
}
三个json文件编辑完在之后,/opt/certs 目录有三个json文件。
cd /opt/certs
#生成 ca 证书和私钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
生成了ca.csr, ca-key.pen, ca.pem三个私钥与证书文件。
(5) 生成etcd用的证书文件
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=k8s-server-client etcd-peer-csr.json | cfssljson -bare etcd-peer
注意,-profile=k8s-server-client,表示客户端与服务端要双向通讯。| cfssljson -bare etcd-peer 表示生成证书文件名为 etcd-peer 。
这次生成了etcd-peer.csr, etcd-peer-key.pem,etcd-peer.pem 三个文件
最终产生这几个文件: ca-config.json, ca.csr, ca-csr.json , ca-key.pem, ca.pem, etcd-peer.csr, etcd-peer-csr.json ,etcd-peer-key.pem,etcd-peer.pem
(6) copy证书到其它两个节点
把ca.pem, etcd-peer.pem, etcd-peer-key.pem三个证书拷贝到etcd02与etcd03节点的【/opt/etcd/certs】目录,etcd只用到三个证书。
cd /opt/certs
scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.102:/opt/etcd/certs/ scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.103:/opt/etcd/certs/
安装etcd (不使用SSL证书安装) |
将etcd集群安装在三个节点上,三个实例节点信息为:
etcd实例名称 | IP地址 | Hostname |
---|---|---|
etcd01 | 192.168.0.102 | yyee-centos-2 |
ctcd02 | 192.168.0.103 | yyee-centos-3 |
etcd03 | 192.168.0.104 | yyee-centos-4 |
(1) 在三个节点上创建工作目录
mkdir -p /var/lib/etcd/data
(2) 编写etcd启动文件
编写 etcd01, etcd02, etcd03 三个节点的etcd启动文件,然后三个节点要同时启动才能启动etcd集群成功。
【编写 etcd01节点的 etcd.service 文件】
vi /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
--name=etcd01 \\
--data-dir=/var/lib/etcd/data \\
--listen-peer-urls=http://192.168.0.102:2380 \\
--listen-client-urls=http://192.168.0.102:2379,http://127.0.0.1:2379 \\
--initial-advertise-peer-urls=http://192.168.0.102:2380 \\
--advertise-client-urls=http://192.168.0.102:2379,http://127.0.0.1 \\
--initial-cluster=etcd01=http://192.168.0.102:2380,etcd02=http://192.168.0.103:2380,etcd03=http://192.168.0.104:2380 \\
--initial-cluster-token=k8s-etcd-cluster \\
--initial-cluster-state=new"
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
【编写 etcd02节点的 etcd.service 文件】
vi /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
--name=etcd02 \\
--data-dir=/var/lib/etcd/data \\
--listen-peer-urls=http://192.168.0.103:2380 \\
--listen-client-urls=http://192.168.0.103:2379,http://127.0.0.1:2379 \\
--initial-advertise-peer-urls=http://192.168.0.103:2380 \\
--advertise-client-urls=http://192.168.0.103:2379,http://127.0.0.1 \\
--initial-cluster=etcd01=http://192.168.0.102:2380,etcd02=http://192.168.0.103:2380,etcd03=http://192.168.0.104:2380 \\
--initial-cluster-token=k8s-etcd-cluster \\
--initial-cluster-state=new"
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
【编写 etcd03节点的 etcd.service 文件】
vi /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
--name=etcd03 \\
--data-dir=/var/lib/etcd/data \\
--listen-peer-urls=http://192.168.0.104:2380 \\
--listen-client-urls=http://192.168.0.104:2379,http://127.0.0.1:2379 \\
--initial-advertise-peer-urls=http://192.168.0.104:2380 \\
--advertise-client-urls=http://192.168.0.104:2379,http://127.0.0.1 \\
--initial-cluster=etcd01=http://192.168.0.102:2380,etcd02=http://192.168.0.103:2380,etcd03=http://192.168.0.104:2380 \\
--initial-cluster-token=k8s-etcd-cluster \\
--initial-cluster-state=new"
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
(3) 启动etcd
然后三个节点要同时执行训动命令才能成功启动etcd。
systemctl daemon-reload systemctl enable etcd #这条命令要在三个节点上同时执行,第一个执行的节点会最多等待30秒让其它两个节点加入集群。 systemctl start etcd
启动如果没报错的话就启动成功了,查看集群状态
etcdctl member list
查看etcd监听端口
netstat -tunlp | grep etcd
安装etcd (使用SSL证书安装) |
将etcd集群安装在三个节点上,三个实例节点信息为:
etcd实例名称 | IP地址 | Hostname |
---|---|---|
etcd01 | 192.168.0.102 | yyee-centos-2 |
ctcd02 | 192.168.0.103 | yyee-centos-3 |
etcd03 | 192.168.0.104 | yyee-centos-4 |
(1) 在三个节点上创建工作目录
mkdir -p /var/lib/etcd/data mkdir -p /opt/certs
(2) copy证书到其它两个节点
把192.168.0.102:/opt/certs/ 目录下的ca.pem, etcd-peer.pem, etcd-peer-key.pem三个证书文件拷贝到etcd02节点与etcd03节点上。
cd /opt/certs
scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.103:/opt/certs/ scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.104:/opt/certs/
(3) 编写etcd启动文件
编写 etcd01, etcd02, etcd03 三个节点的etcd启动文件,然后三个节点要同时启动才能启动etcd集群成功。
【编写 etcd01节点的 etcd.service 文件】
vi /usr/lib/systemd/system/etcd.service
[Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\ --name=etcd01 \\ --data-dir=/var/lib/etcd/data \\ --listen-peer-urls=https://192.168.0.102:2380 \\ --listen-client-urls=https://192.168.0.102:2379,http://127.0.0.1:2379 \\ --initial-advertise-peer-urls=https://192.168.0.102:2380 \\ --advertise-client-urls=https://192.168.0.102:2379 \\ --initial-cluster=etcd01=https://192.168.0.102:2380,etcd02=https://192.168.0.103:2380,etcd03=https://192.168.0.104:2380 \\ --initial-cluster-token=k8s-etcd-cluster \\ --initial-cluster-state=new \\ --cert-file=/opt/etcd/certs/etcd-peer.pem \\ --key-file=/opt/etcd/certs/etcd-peer-key.pem \\ --peer-cert-file=/opt/etcd/certs/etcd-peer.pem \\ --peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \\ --trusted-ca-file=/opt/etcd/certs/ca.pem \\ --peer-trusted-ca-file=/opt/etcd/certs/ca.pem" Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target
【编写 etcd02节点的 etcd.service 文件】
vi /usr/lib/systemd/system/etcd.service
[Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\ --name=etcd02 \\ --data-dir=/var/lib/etcd/data \\ --listen-peer-urls=https://192.168.0.103:2380 \\ --listen-client-urls=https://192.168.0.103:2379,http://127.0.0.1:2379 \\ --initial-advertise-peer-urls=https://192.168.0.103:2380 \\ --advertise-client-urls=https://192.168.0.103:2379 \\ --initial-cluster=etcd01=https://192.168.0.102:2380,etcd02=https://192.168.0.103:2380,etcd03=https://192.168.0.104:2380 \\ --initial-cluster-token=k8s-etcd-cluster \\ --initial-cluster-state=new \\ --cert-file=/opt/etcd/certs/etcd-peer.pem \\ --key-file=/opt/etcd/certs/etcd-peer-key.pem \\ --peer-cert-file=/opt/etcd/certs/etcd-peer.pem \\ --peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \\ --trusted-ca-file=/opt/etcd/certs/ca.pem \\ --peer-trusted-ca-file=/opt/etcd/certs/ca.pem" Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target
【编写 etcd03节点的 etcd.service 文件】
vi /usr/lib/systemd/system/etcd.service
[Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\ --name=etcd03 \\ --data-dir=/var/lib/etcd/data \\ --listen-peer-urls=https://192.168.0.104:2380 \\ --listen-client-urls=https://192.168.0.104:2379,http://127.0.0.1:2379 \\ --initial-advertise-peer-urls=https://192.168.0.104:2380 \\ --advertise-client-urls=https://192.168.0.104:2379 \\ --initial-cluster=etcd01=https://192.168.0.102:2380,etcd02=https://192.168.0.103:2380,etcd03=https://192.168.0.104:2380 \\ --initial-cluster-token=k8s-etcd-cluster \\ --initial-cluster-state=new \\ --cert-file=/opt/etcd/certs/etcd-peer.pem \\ --key-file=/opt/etcd/certs/etcd-peer-key.pem \\ --peer-cert-file=/opt/etcd/certs/etcd-peer.pem \\ --peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \\ --trusted-ca-file=/opt/etcd/certs/ca.pem \\ --peer-trusted-ca-file=/opt/etcd/certs/ca.pem" Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target
(4) 启动etcd
然后三个节点要同时执行训动命令才能成功启动etcd。
systemctl daemon-reload
systemctl enable etcd
#这条命令要在三个节点上同时执行,第一个执行的节点会最多等待30秒让其它两个节点加入集群。
systemctl start etcd
启动如果没报错的话就启动成功了,查看集群状态
etcdctl member list
查看etcd监听端口
netstat -tunlp | grep etcd
以上是关于centos搭建k8s集群,安装Docker,, Harbor, etcd的主要内容,如果未能解决你的问题,请参考以下文章
Kubernetes第二篇:从零开始搭建k8s集群(亲测可用)