K8SStatefulSet
Posted 礁之
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了K8SStatefulSet相关的知识，希望对你有一定的参考价值。
文章目录
一、StatefulSet概述
-
StatefulSet（有状态集），通常用于部署有状态的并且需要有序启动的应用程序 -
StatefulSet主要用于管理有状态应用程序的工作负载API对象，例如：
生产环境中部署ElasticSearch集群、Mongodb集群，以及需要持久化的RabbitMQ集群、Redis集群、Kafka集群、ZooKeeper集群等
-
而StatefuleSet创建的Pod一般都会使用
Headless Service（无头服务）进行通信，与普通的Service的区别在于：1、Headless不分配ClusterIP，也就是不使用ClusterIP，而是使用Endpoint进行通信，一般的格式为：
statefulSetName-0..N-1.serviceName.namespace.svc.cluster.local #注释 1.statefulSetName：statefulSet的名称 2.0..N-1：Pod所在的序号，从0开始到n-1 3.serviceName：Headless Service的名称 4.namespace：服务所在的命名空间 5.cluster.local：Cluster Domain集群域2、Headless Service可以通过解析Service的DNS，返回所有Pod的地址和DNS==（StatefulSet部署的Pod才有DNS）==
3、普通的Service，只能通过解析Service的DNS返回Service的ClusterIP
二、StatefulSet注意事项
- StatefulSet一般应用于符合以下一个或多个要求的应用程序：
- 需要持久化数据
- 需要有序的、优雅的部署和扩展
- 需要有序的自动安装更新
注释：如果应用程序不需要任何稳定的标识符或者有序部署、删除、扩展等，应该使用无状态的控制器，例如Deployment、ReplicaSet，通常都会使用Deployment
三、创建StatefulSet
- 创建一个简单的
StatefulSet：
- kind: StatefulSet：这个参数定义了一个名叫nginx的StatefulSet，replicas表示部署Pod的副本数，也就是数量，这里写的副本数是1
- kind: Service：这个参数定义了一个名称为nginx的
Headless Service，创建的Service格式为nginx-0.nginx.default.svc.cluster.local，因为没有指定命名空间，所以默认是部署在default
[root@master test]# cat nginx.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nginx
spec:
serviceName: "nginx"
selector:
matchLabels:
run: nginx
replicas: 1
template:
metadata:
labels:
run: nginx
spec:
containers:
- name: nginx
image: nginx:1.15.2
ports:
- containerPort: 80
name: nginx
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
run: nginx
spec:
ports:
- port: 80
name: nginx
clusterIP: None
selector:
run: nginx
- 查看对象资源
[root@master test]# kubectl apply -f nginx.yaml #创建
statefulset.apps/nginx created
service/nginx created
[root@master test]# kubectl get pods #查看pod，可以看到nginx-0，这个名称是有编号的
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 8s
[root@master test]# kubectl get svc -o wide #创建的是无头服务，所以这里ip是none
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 172.16.0.1 <none> 443/TCP 8d <none>
nginx ClusterIP None <none> 80/TCP 37s run=nginx
[root@master test]# kubectl get statefulsets.apps
NAME READY AGE
nginx 1/1 24s
[root@master test]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-0 1/1 Running 0 95s 10.244.1.35 node <none> <none>
[root@master test]# kubectl describe svc nginx
Name: nginx
Namespace: default
Labels: run=nginx
Annotations: Selector: run=nginx
Type: ClusterIP
IP: None
Port: nginx 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.1.35:80
Session Affinity: None
Events: <none>
#访问endpoints地址，可以成功访问
[root@master test]# curl 10.244.1.35
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
四、StatefulSet的扩容和缩容
- 与Deployment差不多，StatefulSet同样可以通过更新
replicas参数扩容、缩容StatefulSet，也可以使用kubectlscale或者kubectlpatch来扩容、缩容一个StatefulSet
（1）扩容
- 在上面创建的statefulSet基础上，将pod的副本数扩容到5个
注意：扩容之前必须保证有创建完成的静态PV、动态PV以及emptyDir
[root@master test]# kubectl scale statefulset nginx --replicas=5 #修改副本数为5个
statefulset.apps/nginx scaled
[root@master test]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 18m
nginx-1 1/1 Running 0 1s
nginx-2 0/1 ContainerCreating 0 0s
[root@master test]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 18m
nginx-1 1/1 Running 0 4s
nginx-2 1/1 Running 0 3s
nginx-3 0/1 ContainerCreating 0 1s
[root@master test]# kubectl get pods #可以看到是依次创建的
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 18m
nginx-1 1/1 Running 0 7s
nginx-2 1/1 Running 0 6s
nginx-3 1/1 Running 0 4s
nginx-4 0/1 ContainerCreating 0 2s
[root@master test]# kubectl get pods #可以看到创建的pod有顺序
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 18m
nginx-1 1/1 Running 0 30s
nginx-2 1/1 Running 0 29s
nginx-3 1/1 Running 0 27s
nginx-4 1/1 Running 0 25s
[root@master test]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-0 1/1 Running 0 27m 10.244.1.35 node <none> <none>
nginx-1 1/1 Running 0 9m13s 10.244.1.36 node <none> <none>
nginx-2 1/1 Running 0 9m12s 10.244.1.37 node <none> <none>
nginx-3 1/1 Running 0 9m10s 10.244.1.38 node <none> <none>
nginx-4 1/1 Running 0 9m8s 10.244.1.39 node <none> <none>
#可以
[root@master test]# nslookup nginx.default.svc.cluster.local 172.16.0.10
Server: 172.16.0.10
Address: 172.16.0.10#53
Name: nginx.default.svc.cluster.local
Address: 10.244.1.37
Name: nginx.default.svc.cluster.local
Address: 10.244.1.35
Name: nginx.default.svc.cluster.local
Address: 10.244.1.38
Name: nginx.default.svc.cluster.local
Address: 10.244.1.39
Name: nginx.default.svc.cluster.local
Address: 10.244.1.36
（2）缩容
[root@master test]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-0 1/1 Running 0 48m 10.244.1.35 node <none> <none>
nginx-1 1/1 Running 0 30m 10.244.1.36 node <none> <none>
nginx-2 1/1 Running 0 30m 10.244.1.37 node <none> <none>
nginx-3 1/1 Running 0 30m 10.244.1.38 node <none> <none>
nginx-4 1/1 Running 0 30m 10.244.1.39 node <none> <none>
#修改副本数为3
[root@master test]# kubectl patch statefulset nginx -p '"spec":"replicas":3'
statefulset.apps/nginx patched
[root@master test]# kubectl get pods -w #加-w参数，可以实时查看，可以发现正在删除，并且是以序号从大到小删除
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 54m
nginx-1 1/1 Running 0 35m
nginx-2 1/1 Running 0 35m
nginx-3 1/1 Running 0 50s
nginx-4 0/1 Terminating 0 48s
nginx-4 0/1 Terminating 0 57s
nginx-4 0/1 Terminating 0 57s
nginx-3 1/1 Terminating 0 59s
nginx-3 0/1 Terminating 0 59s
nginx-3 0/1 Terminating 0 69s
nginx-3 0/1 Terminating 0 69s
[root@master test]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-0 1/1 Running 0 52m 10.244.1.35 node <none> <none>
nginx-1 1/1 Running 0 34m 10.244.1.36 node <none> <none>
nginx-2 1/1 Running 0 34m 10.244.1.37 node <none> <none>
（3）更新策略
- 先来查看yaml文件，使用
kubectl get statefulset -o yaml
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate #这个就是更新策略
- 修改更新策略
kubectl patch statefulset web -p '"spec":"updateStrategy":"type":"RollingUpdate"' #type选择更新的策略
- 更新策略
On Delete策略：OnDelete更新策略实现了传统（1.7版本之前）的行为。当我们选择这个更新策略并修改StatefulSet的.spec.template字段时，StatefulSet控制器不会自动更新Pod，我们必须手动删除Pod才能使控制器创建新的Pod。RollingUpdate策略：RollingUpdate（滚动更新）更新策略会更新一个StatefulSet中所有的Pod，采用与序号索引相反的顺序进行滚动更新。注释：
使用RollingUpdate策略更新下一个Pod前，StatefulSet控制器会终止每一个Pod并等待它们变成Running和Ready状态。在当前顺序变成Running和Ready状态之前，StatefulSet控制器不会更新下一个Pod，但它仍然会重建任何在更新过程中发生故障的Pod，使用它们当前的版本。已经接收到请求的Pod将会被恢复为更新的版本，没有收到请求的Pod则会被恢复为之前的版本。注释：在更新过程中可以使用
kubectl rollout status sts/<name>来查看滚动更新的状态
（4）分段更新
- StatefulSet可以使用RollingUpdate更新策略的
partition参数来分段更新一个StatefulSet，分段更新将会使pod序号小于partition（分区）数字的Pod保持当前版本，只更新序号大于分区的Pod，相当于是灰度发布 - 下面来看案例
#更新配置，更新分区数字为3，意思是每次更新使，不更新序号小于3的
[root@master test]# kubectl patch statefulset nginx -p '"spec":"updateStrategy":"type":"RollingUpdate","rollingUpdate":"partition":3'
statefulset.apps/nginx patched
#这里可以更新一下镜像，我这里下载了新的nginx镜像，nginx:1.21.0
[root@master test]# docker pull nginx:1.21.0
1.21.0: Pulling from library/nginx
b4d181a07f80: Pull complete
edb81c9bc1f5: Pull complete
b21fed559b9f: Pull complete
03e6a2452751: Pull complete
b82f7f888feb: Pull complete
5430e98eba64: Pull complete
Digest: sha256:47ae43cdfc7064d28800bc42e79a429540c7c80168e8c8952778c0d5af1c09db
Status: Downloaded newer image for nginx:1.21.0
docker.io/library/nginx:1.21.0
#因为上面定的分区数字有点大，所以这里再扩容一下
[root@master test]kubectl patch statefulset nginx -p '"spec":"replicas":7'
statefulset.apps/nginx patched
#修改镜像为nginx:1.21.0
[root@master test]# kubectl patch statefulset nginx --type='json' -p '["op":"replace","path":"/spec/template/spec/containers/0/image","value":"nginx:1.21.0"]'
#因为我有一个node节点，pod都创建再node上了，所以去node查看，对应容器使用的镜像id，可以发现序号大于等于3的容器使用的镜像都是新镜像
[root@node ~]# docker ps -a | grep nginx
ca0156f6b0e1 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0
00f167982405 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0
ef28b0660a24 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0
cea1be3e6055 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0
acbe6faf965c 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0
ffdc06a8961b registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0
697436878a91 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0
fde7dc2cf8ef registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0
8a6130d190af c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-2_default_107adf0c-856f-4aaa-b7a0-53515ac84d59_0
1720f7571d11 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-2_default_107adf0c-856f-4aaa-b7a0-53515ac84d59_0
62d3bde4de36 c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-1_default_3336ee35-efa0-4b76-922d-c93146853f3b_0
db81735a8307 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-1_default_3336ee35-efa0-4b76-922d-c93146853f3b_0
f652001cd901 c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0
6d781716840d registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0
[root@node ~]# docker images | grep nginx
nginx 1.21.0 4f380adfc10f 13 months ago 133MB
nginx 1.15.2 c82521676580 3 years ago 109MB
#修改分区数字再次查看
[root@master test]# kubectl patch statefulset nginx -p '"spec":"updateStrategy":"type":"RollingUpdate","rollingUpdate":"partition":1'
statefulset.apps/nginx patched
#可以看到，1号和2号，这两个pod重启了
[root@master test]# kubectl get pods -w
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 120m
nginx-1 1/1 Running 0 16s
nginx-2 1/1 Running 0 21s
nginx-3 1/1 Running 0 5m51s
nginx-4 1/1 Running 0 5m49s
nginx-5 1/1 Running 0 5m47s
nginx-6 1/1 Running 0 5m45s
#查看容器，发现容器的镜像也更新了
[root@node ~]# docker ps -a | grep nginx
1b3a6390da17 4f380adfc10f "/docker-entrypoint.…" About a minute ago Up About a minute k8s_nginx_nginx-1_default_1a858475-b5ec-4e11-a740-6e3f86e6a65d_0
340a0193dfbd registry.aliyuncs.com/google_containers/pause:3.2 "/pause" About a minute ago Up About a minute k8s_POD_nginx-1_default_1a858475-b5ec-4e11-a740-6e3f86e6a65d_0
72ad32af50e8 4f380adfc10f "/docker-entrypoint.…" About a minute ago Up About a minute k8s_nginx_nginx-2_default_71559ce2-20df-4591-9f6c-ccc16886e0f8_0
848c4c4c6264 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" About a minute ago Up About a minute k8s_POD_nginx-2_default_71559ce2-20df-4591-9f6c-ccc16886e0f8_0
ca0156f6b0e1 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0
00f167982405 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0
ef28b0660a24 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0
cea1be3e6055 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0
acbe6faf965c 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0
ffdc06a8961b registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0
697436878a91 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0
fde7dc2cf8ef registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0
f652001cd901 c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0
6d781716840d registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0
[root@node ~]# docker images | grep nginx
nginx 1.21.0 4f380adfc10f 13 months ago 133MB
nginx 1.15.2 c82521676580 3 years ago 109MB
五、dns解析
#service名称，nginx
[root@master test]# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 172.16.0.1 <none> 443/TCP 8d <none>
nginx ClusterIP None <none> 80/TCP 35m run=nginx
#kube-des的ip地址，172.16.0.10
[root@master test]# kubectl get svc -o wide -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
grafana NodePort 172.16.195.186 <none> 3000:30931/TCP 6d5h app=grafana,component=core
kube-dns ClusterIP 172.16.0.10 <none> 53/UDP,53/TCP,9153/TCP 8d k8s-app=kube-dns
metrics-server ClusterIP 172.16.122.15 <none> 443/TCP 27h k8s-app=metrics-server
node-exporter NodePort 172.16.201.35 <none> 9100:31672/TCP 6d22h k8s-app=node-exporter
prometheus NodePort 172.16.176.125 <none> 9090:30003/TCP 6d21h app=prometheus
#通过解析dns，可以获取到pod的所有ip
[root@master test]# nslookup nginx.default.svc.cluster.local 172.16.0.10
Server: 172.16.0.10
Address: 172.16.0.10#53
Name: nginx.default.svc.cluster.local
Address: 10.244.1.37
Name: nginx.default.svc.cluster.local
Address: 10.244.1.35
Name: nginx.default.svc.cluster.local
Address: 10.244.1.38
Name: nginx.default.svc.cluster.local
Address: 10.244.1.39
Name: nginx.default.svc.cluster.local
Address: 10.244.1.36
nslookup nginx.default.svc.cluster.local 172.16.0.10格式为：
nginx：service的名称
default：命名空间名称
svc.cluster.local：固定的
172.16.0.10：kube-dns的ip地址
六、删除
- 删除的命令都是一样的
#直接这么删除，只会删除statefulSet资源，无法把创建的service删除
kubectl delete statefulset nginx
#而这种删除可以把此yaml文件创建的所有资源对象一次性删除，也就是级联删除
kubectl delete -f nginx.yaml
以上是关于K8SStatefulSet的主要内容，如果未能解决你的问题，请参考以下文章
想必大家在做参数验证的时候，都会遇到一个问题，就是如何验证枚举？ 自定义annotation 自定义Validator
安装proe5.0之后，总是会出现错误日志，但还可以使用。就是每次打开也都会产生一个附带的日志？？？
使用手机地图时，比如搜狗地图、百度地图，都会出现“ERR:Network creat fail” ，这个是由于啥原因呢？