centos搭建k8s集群,安装Docker,, Harbor, etcd

Posted 民工黑猫

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了centos搭建k8s集群,安装Docker,, Harbor, etcd相关的知识,希望对你有一定的参考价值。

【前期准备】

下载 etcd 二进制安装包:https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz

下载 kubernetes 1.18.3 二进制安装包:https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1183 

注意:打开链接有很多下载包,包含kubernetes-client 、kubernetes-server 、kubernetes-node ,下载其中一个64位的就行。

 

安装证书签发工具cfssl

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64

wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64

wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64

chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64

cp cfssl_linux-amd64 /usr/local/bin/cfssl

cp cfssljson_linux-amd64 /usr/local/bin/cfssljson

cp cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

下载etcd安装包

 

下载etcd-v3.4.9二进制包

下载etcd-v.3.4.9二进制安装包,

其它版本下载地址: https://github.com/etcd-io/etcd/tags 

wget  https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz

 

下载完成后,解压etcd安装包,复制解压目录里的etcd与etcdctl 两个文件到 /usr/local/bin 目录下面,并且赋予两个文件可执行权限

tar xfv etcd-v3.4.9-linux-amd64.tar.gz
cd etcd-v3.4.9-linux-amd64
cp etcd /usr/local/bin 
cp etcdctl /usr/local/bin
#赋予执行权限
chmod +x /usr/local/bin/etcd
chmod +x /usr/local/bin/etcdctl

 

签发etcd证书

创建证书存放目录。

mkdir -p /opt/certs

 

【创建证书】

首先安装cfssl 证书制作工具,安装方法参考:https://www.cnblogs.com/yyee/p/13189331.html

在etcd01 (192.168.0.102)节点上创建证书。

(1) 创建根证书配置文件

vi  /opt/certs/ca-config.json

{
  "signing": {
    "default": {
      "expiry": "175200h"
     },
    "profiles":{
      "k8s-server": {
       "expiry": "175200h",
        "usages": [
          "signing",
          "key encipherment",
          "server auth"
        ]
      },
      "k8s-client": {
       "expiry": "175200h",
        "usages": [
          "signing",
          "key encipherment",
          "client auth"
        ]
      },
      "k8s-server-client": {
       "expiry": "175200h",
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ]
      }
    }
  }
}

 

signing:表示该证书可用于签名其它证书(生成的 ca.pem 证书中 CA=TRUE);

server auth:表示 client 可以用该该证书对 server 提供的证书进行验证;
client auth:表示 server 可以用该该证书对 client 提供的证书进行验证;
"expiry": "175200h":证书有效期设置为 20 年; 
 
(2) 创建根证书请求文件 ca-csr.json
vi /opt/certs/ca-csr.json
{
  "CN": "k8s", 
  "key": {
     "algo": "rsa",
     "size": 2048
   },
  "names": [
    {
      "C": "CN",
      "L": "Beijing",
      "ST": "Beijing" ,
      "O": "k8s",
      "OU": "system"
     }
  ]
}

CN:Common Name:kube-apiserver 从证书中提取该字段作为请求的用户名 (User Name),

浏览器使用该字段验证网站是否合法;
O:Organization:kube-apiserver 从证书中提取该字段作为请求用户所属的组 (Group);
kube-apiserver 将提取的 User、Group 作为 RBAC 授权的用户标识; 
 

(3) 创建etcd证书请求文件etcd-peer-csr.json

vi /opt/certs/etcd-peer-csr.json

{
  "CN": "k8s-etcd",
  "hosts": [
    "192.168.0.101",
    "192.168.0.102",
    "192.168.0.103",
    "192.168.0.104",
    "192.168.0.105",
    "192.168.0.106"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "k8s",
      "OU": "system"
    }
  ]
}

  

三个json文件编辑完在之后,/opt/certs 目录有三个json文件。

  

(4) 生成 ca 证书和私钥 
cd /opt/certs
#生成 ca 证书和私钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca

 

 

 生成了ca.csr, ca-key.pen, ca.pem三个私钥与证书文件。

 

(5) 生成etcd用的证书文件

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=k8s-server-client etcd-peer-csr.json | cfssljson -bare etcd-peer

注意,-profile=k8s-server-client,表示客户端与服务端要双向通讯。| cfssljson -bare etcd-peer 表示生成证书文件名为 etcd-peer 。

这次生成了etcd-peer.csr, etcd-peer-key.pem,etcd-peer.pem 三个文件

 最终产生这几个文件: ca-config.json, ca.csr, ca-csr.json , ca-key.pem, ca.pem, etcd-peer.csr, etcd-peer-csr.json ,etcd-peer-key.pem,etcd-peer.pem 

  

(6) copy证书到其它两个节点 

 把ca.pem, etcd-peer.pem, etcd-peer-key.pem三个证书拷贝到etcd02与etcd03节点的【/opt/etcd/certs】目录,etcd只用到三个证书。

cd  /opt/certs
scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.102:/opt/etcd/certs/ scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.103:/opt/etcd/certs/
 
 
安装etcd可以使用SSL证书安装,也可以不使用SSL证书安装。
安装etcd (不使用SSL证书安装)

将etcd集群安装在三个节点上,三个实例节点信息为:

 

etcd实例名称IP地址Hostname
etcd01 192.168.0.102 yyee-centos-2
ctcd02 192.168.0.103 yyee-centos-3
etcd03 192.168.0.104 yyee-centos-4

 

 

 

 

 

 

(1) 在三个节点上创建工作目录

 

mkdir -p /var/lib/etcd/data

  

 (2) 编写etcd启动文件

编写 etcd01, etcd02, etcd03 三个节点的etcd启动文件,然后三个节点要同时启动才能启动etcd集群成功。

【编写 etcd01节点的 etcd.service 文件】

vi  /usr/lib/systemd/system/etcd.service 

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
 --name=etcd01 \\
 --data-dir=/var/lib/etcd/data \\
 --listen-peer-urls=http://192.168.0.102:2380 \\
 --listen-client-urls=http://192.168.0.102:2379,http://127.0.0.1:2379 \\
 --initial-advertise-peer-urls=http://192.168.0.102:2380 \\
 --advertise-client-urls=http://192.168.0.102:2379,http://127.0.0.1 \\
 --initial-cluster=etcd01=http://192.168.0.102:2380,etcd02=http://192.168.0.103:2380,etcd03=http://192.168.0.104:2380 \\
 --initial-cluster-token=k8s-etcd-cluster \\
 --initial-cluster-state=new"

Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

 

【编写 etcd02节点的 etcd.service 文件】

vi  /usr/lib/systemd/system/etcd.service 

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
 --name=etcd02 \\
 --data-dir=/var/lib/etcd/data \\
 --listen-peer-urls=http://192.168.0.103:2380 \\
 --listen-client-urls=http://192.168.0.103:2379,http://127.0.0.1:2379 \\
 --initial-advertise-peer-urls=http://192.168.0.103:2380 \\
 --advertise-client-urls=http://192.168.0.103:2379,http://127.0.0.1 \\
 --initial-cluster=etcd01=http://192.168.0.102:2380,etcd02=http://192.168.0.103:2380,etcd03=http://192.168.0.104:2380 \\
 --initial-cluster-token=k8s-etcd-cluster \\
 --initial-cluster-state=new"

Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

 

【编写 etcd03节点的 etcd.service 文件】

vi  /usr/lib/systemd/system/etcd.service 

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
 --name=etcd03 \\
 --data-dir=/var/lib/etcd/data \\
 --listen-peer-urls=http://192.168.0.104:2380 \\
 --listen-client-urls=http://192.168.0.104:2379,http://127.0.0.1:2379 \\
 --initial-advertise-peer-urls=http://192.168.0.104:2380 \\
 --advertise-client-urls=http://192.168.0.104:2379,http://127.0.0.1 \\
 --initial-cluster=etcd01=http://192.168.0.102:2380,etcd02=http://192.168.0.103:2380,etcd03=http://192.168.0.104:2380 \\
 --initial-cluster-token=k8s-etcd-cluster \\
 --initial-cluster-state=new"

Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target 

 

(3) 启动etcd

 然后三个节点要同时执行训动命令才能成功启动etcd。

systemctl daemon-reload
systemctl enable etcd

#这条命令要在三个节点上同时执行,第一个执行的节点会最多等待30秒让其它两个节点加入集群。
systemctl start etcd  

 

启动如果没报错的话就启动成功了,查看集群状态

etcdctl member list

 

 

 

查看etcd监听端口

netstat -tunlp | grep etcd

 

 

  

安装etcd (使用SSL证书安装)

将etcd集群安装在三个节点上,三个实例节点信息为:

 

etcd实例名称IP地址Hostname
etcd01 192.168.0.102 yyee-centos-2
ctcd02 192.168.0.103 yyee-centos-3
etcd03 192.168.0.104 yyee-centos-4

 

 

 

 

 

 

(1) 在三个节点上创建工作目录

 

mkdir -p /var/lib/etcd/data
mkdir -p /opt/certs

 

 

(2) copy证书到其它两个节点 

 把192.168.0.102:/opt/certs/  目录下的ca.pem, etcd-peer.pem, etcd-peer-key.pem三个证书文件拷贝到etcd02节点与etcd03节点上。

cd  /opt/certs
scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.103:/opt/certs/ scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.0.104:/opt/certs/
 

 (3) 编写etcd启动文件

编写 etcd01, etcd02, etcd03 三个节点的etcd启动文件,然后三个节点要同时启动才能启动etcd集群成功。

【编写 etcd01节点的 etcd.service 文件】

vi  /usr/lib/systemd/system/etcd.service 

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
 --name=etcd01 \\
 --data-dir=/var/lib/etcd/data \\
 --listen-peer-urls=https://192.168.0.102:2380 \\
 --listen-client-urls=https://192.168.0.102:2379,http://127.0.0.1:2379 \\
 --initial-advertise-peer-urls=https://192.168.0.102:2380 \\
 --advertise-client-urls=https://192.168.0.102:2379 \\
 --initial-cluster=etcd01=https://192.168.0.102:2380,etcd02=https://192.168.0.103:2380,etcd03=https://192.168.0.104:2380 \\
 --initial-cluster-token=k8s-etcd-cluster \\
 --initial-cluster-state=new \\
 --cert-file=/opt/etcd/certs/etcd-peer.pem \\
 --key-file=/opt/etcd/certs/etcd-peer-key.pem \\
 --peer-cert-file=/opt/etcd/certs/etcd-peer.pem \\
 --peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \\
 --trusted-ca-file=/opt/etcd/certs/ca.pem \\
 --peer-trusted-ca-file=/opt/etcd/certs/ca.pem"


Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

 

【编写 etcd02节点的 etcd.service 文件】

vi  /usr/lib/systemd/system/etcd.service 

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
 --name=etcd02 \\
 --data-dir=/var/lib/etcd/data \\
 --listen-peer-urls=https://192.168.0.103:2380 \\
 --listen-client-urls=https://192.168.0.103:2379,http://127.0.0.1:2379 \\
 --initial-advertise-peer-urls=https://192.168.0.103:2380 \\
 --advertise-client-urls=https://192.168.0.103:2379 \\
 --initial-cluster=etcd01=https://192.168.0.102:2380,etcd02=https://192.168.0.103:2380,etcd03=https://192.168.0.104:2380 \\
 --initial-cluster-token=k8s-etcd-cluster \\
 --initial-cluster-state=new \\
 --cert-file=/opt/etcd/certs/etcd-peer.pem \\
 --key-file=/opt/etcd/certs/etcd-peer-key.pem \\
 --peer-cert-file=/opt/etcd/certs/etcd-peer.pem \\
 --peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \\
 --trusted-ca-file=/opt/etcd/certs/ca.pem \\
 --peer-trusted-ca-file=/opt/etcd/certs/ca.pem"


Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

 

【编写 etcd03节点的 etcd.service 文件】

vi  /usr/lib/systemd/system/etcd.service 

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd \\
 --name=etcd03 \\
 --data-dir=/var/lib/etcd/data \\
 --listen-peer-urls=https://192.168.0.104:2380 \\
 --listen-client-urls=https://192.168.0.104:2379,http://127.0.0.1:2379 \\
 --initial-advertise-peer-urls=https://192.168.0.104:2380 \\
 --advertise-client-urls=https://192.168.0.104:2379 \\
 --initial-cluster=etcd01=https://192.168.0.102:2380,etcd02=https://192.168.0.103:2380,etcd03=https://192.168.0.104:2380 \\
 --initial-cluster-token=k8s-etcd-cluster \\
 --initial-cluster-state=new \\
 --cert-file=/opt/etcd/certs/etcd-peer.pem \\
 --key-file=/opt/etcd/certs/etcd-peer-key.pem \\
 --peer-cert-file=/opt/etcd/certs/etcd-peer.pem \\
 --peer-key-file=/opt/etcd/certs/etcd-peer-key.pem \\
 --trusted-ca-file=/opt/etcd/certs/ca.pem \\
 --peer-trusted-ca-file=/opt/etcd/certs/ca.pem"


Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target 

 

(4) 启动etcd

 然后三个节点要同时执行训动命令才能成功启动etcd。

systemctl daemon-reload
systemctl enable etcd

#这条命令要在三个节点上同时执行,第一个执行的节点会最多等待30秒让其它两个节点加入集群。
systemctl start etcd  

 

启动如果没报错的话就启动成功了,查看集群状态

etcdctl member list

 

 

 

查看etcd监听端口

netstat -tunlp | grep etcd

 

 

 

以上是关于centos搭建k8s集群,安装Docker,, Harbor, etcd的主要内容,如果未能解决你的问题,请参考以下文章

使用docker搭建flink集群

docker 搭建es+es基本使用

Kubernetes第二篇:从零开始搭建k8s集群(亲测可用)

Kubernetes第二篇:从零开始搭建k8s集群(亲测可用)

k8s安装部署

k8s集群安装部署实战详细手册