k8s搭建
Posted xuzhongtao
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了k8s搭建相关的知识，希望对你有一定的参考价值。
K8s官方文档地址:https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/ 如果用云主机部署，一定要提前开启端口
1. 服务器规划
|
角色 |
IP |
组件 |
|
k8s-master1 |
192.168.31.63 |
kube-apiserver kube-controller-manager kube-scheduler etcd |
|
k8s-master2 |
192.168.31.64 |
kube-apiserver kube-controller-manager kube-scheduler
|
|
k8s-node1 |
192.168.31.65 |
kubelet kube-proxy docker etcd |
|
k8s-node2 |
192.168.31.66 |
kubelet kube-proxy docker etcd |
|
Load Balancer（Master） |
192.168.31.61 192.168.31.60 (VIP) |
nginx L4 |
|
Load Balancer（Backup） |
192.168.31.62 |
Nginx L4 |
1.系统初始化
修改主机名称：
hostnamectl set-hostname k8s-master1
关闭防火墙：
# systemctl stop firewalld
# systemctl disable firewalld
关闭selinux：
# setenforce 0 # 临时
# sed -i ‘s/enforcing/disabled/‘ /etc/selinux/config # 永久
关闭swap：
# swapoff -a # 临时
# vim /etc/fstab # 永久
同步系统时间：
# ntpdate time.windows.com
2.2 部署三个Etcd节点
TLS、etcd地址:
链接：https://pan.baidu.com/s/1kyC5KgsF5DB2fZK5UGPaQg
提取码：o101
# tar zxvf etcd.tar.gz
# cd etcd
# cp TLS/etcd/ssl/{ca,server,server-key}.pem ssl
分别拷贝到Etcd三个节点：
# scp –r etcd root@192.168.31.63:/opt
# scp etcd.service root@192.168.31.63:/usr/lib/systemd/system/
登录三个节点修改配置文件 名称和IP：
# vi /opt/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-1" 名称一定要替换
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.31.63:2380" 内网ip
ETCD_LISTEN_CLIENT_URLS="https://192.168.31.63:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.31.63:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.31.63:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.31.63:2380,etcd-2=https://192.168.31.64:2380,etcd-3=https://192.168.31.65:2380" 部署3个节点的内网ip
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new" 集群状态
# systemctl daemon-reload
# systemctl start etcd
# ps -ef|grep etcd 查看etcd进程
# systemctl enable etcd 设置开机启动
# tail /var/log/messages -f 查看系统日志
2.3 查看集群状态
# /opt/etcd/bin/etcdctl
> --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem
> --endpoints="https://192.168.31.63:2379,https://192.168.31.64:2379,https://192.168.31.65:2379" 部署3个节点的内网ip一定要替换
> cluster-health
如果出现下面字段，说明集群状态是健康的
member 37f20611ff3d9209 is healthy: got healthy result from https://192.168.31.63:2379
member b10f0bac3883a232 is healthy: got healthy result from https://192.168.31.64:2379
member b46624837acedac9 is healthy: got healthy result from https://192.168.31.65:2379
cluster is healthy
1.部署Master Node
1.1 生成apiserver证书
# cd TLS/k8s
修改请求文件中hosts字段包含所有etcd节点IP：
# vi server-csr.json
{
"CN": "kubernetes",
"hosts": [
"10.0.0.1",
"127.0.0.1",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local",
"192.168.31.60", 你的内网ip
"192.168.31.61",
"192.168.31.62",
"192.168.31.63",
"192.168.31.64",
"192.168.31.65",
"192.168.31.66"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
# ./generate_k8s_cert.sh
# ls *pem
ca-key.pem ca.pem kube-proxy-key.pem kube-proxy.pem server-key.pem server.pem
3.2 部署apiserver，controller-manager和scheduler
在Master节点完成以下操作。
二进制包下载地址：https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.16.md#v1161
master安装包下载地址:链接：https://pan.baidu.com/s/1kyC5KgsF5DB2fZK5UGPaQg
提取码：o101
二进制文件位置：kubernetes/serverr/bin
# tar zxvf k8s-master.tar.gz
# cd kubernetes
# cp TLS/k8s/ssl/*.pem ssl
# cp –r kubernetes /opt
# cp kube-apiserver.service kube-controller-manager.service kube-scheduler.service /usr/lib/systemd/system
# cat /opt/kubernetes/cfg/kube-apiserver.conf
KUBE_APISERVER_OPTS="--logtostderr=false
--v=2
--log-dir=/opt/kubernetes/logs
--etcd-servers=https://192.168.31.63:2379,https://192.168.31.64:2379,https://192.168.31.65:2379 替换etcd节点的内网ip
--bind-address=192.168.31.63 替换master节点的ip
--secure-port=6443
--advertise-address=192.168.31.63 替换master节点的ip
……
# systemctl start kube-apiserver
# systemctl start kube-controller-manager
# systemctl start kube-scheduler
# systemctl enable kube-apiserver
# systemctl enable kube-controller-manager
# systemctl enable kube-scheduler
# systemctl start kube-apiserver
# ls /opt/kubernetes/logs 查看日志
# less /opt/kubernetes/logs/kube-apiserver.INFO
# tail -f /opt/kubernetes/logs/kube-controller-manager.INFO
# for i in $(ls /opt/kubernetes/bin);do systemctl enable $i;done开机启动
# mv /opt/kubernetes/bin/kubectl /usr/local/bin/kubectl 移动到环境变量
# chmod a+x /usr/local/bin/kubect
# kubectl get cs查看组件状态
# 查看3个组件的进程 ps -ef|grep kube
3.3 启用TLS Bootstrapping
为kubelet TLS Bootstrapping 授权：
# cat /opt/kubernetes/cfg/token.csv
c47ffb939f5ca36231d9e3121a252940,kubelet-bootstrap,10001,"system:node-bootstrapper"
格式：token,用户,uid,用户组
给kubelet-bootstrap授权：
kubectl create clusterrolebinding kubelet-bootstrap
--clusterrole=system:node-bootstrapper
--user=kubelet-bootstrap
token也可自行生成替换：
head -c 16 /dev/urandom | od -An -t x | tr -d ‘ ‘
但apiserver配置的token必须要与node节点bootstrap.kubeconfig配置里一致。
1.部署Worker Node
1.1 安装Docker
二进制包下载地址：https://download.docker.com/linux/static/stable/x86_64/
docker下载地址: 链接：https://pan.baidu.com/s/1kyC5KgsF5DB2fZK5UGPaQg
提取码：o101
# tar zxvf k8s-node.tar.gz
# tar zxvf docker-18.09.6.tgz
# mv docker/* /usr/bin
# mkdir /etc/docker
# mv daemon.json /etc/docker
# mv docker.service /usr/lib/systemd/system
# systemctl start docker
# systemctl enable docker
# docker info 通过docker info查看docker是否启动成功
执行docker info出现如下警告
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
解决办法：
vi /etc/sysctl.conf
添加以下内容
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
最后再执行
sysctl -p
此时docker info就看不到此报错了
执行docker info出现如下警告
4.2 部署kubelet和kube-proxy
拷贝证书到Node：
# cd TLS/k8s
# scp ca.pem kube-proxy*.pem root@192.168.31.65:/opt/kubernetes/ssl/
# cp kube-apiserver.service kube-controller-manager.service kube-
# tar zxvf k8s-node.tar.gz
# mv kubernetes /opt
# cp kubelet.service kube-proxy.service /usr/lib/systemd/system
查看以下三个文件中IP地址：
[root@k8s-node2 kubernetes]# grep 192 *
修改以下两个文件中主机名：
[root@k8s-node2 cfg]# vim bootstrap.kubeconfig
[root@k8s-node2 cfg]# vim kubelet.conf
[root@k8s-node2 cfg]# vim kubelet.kubeconfig
[root@k8s-node2 cfg]# vim kube-proxy-config.yml
[root@k8s-node2 cfg]# vim kube-proxy.kubeconfig
# systemctl start kubelet
# systemctl start kube-proxy
# systemctl enable kubelet
# systemctl enable kube-proxy
# tail /opt/kubernetes/logs/kubelet.INFO 查看日志
4.3 允许给Node颁发证书
# kubectl get csr
# kubectl certificate approve node-csr-MYUxbmf_nmPQjmH3LkbZRL2uTO-_FCzDQUoUfTy7YjI 替换你的node名称
# kubectl get node
以上是关于k8s搭建的主要内容，如果未能解决你的问题，请参考以下文章