k8s实践liveness与readiness 2种探针使用

Posted 2021-09-25 dl528888

0.简介

本文主要介绍v1.20版本k8s使用2种探针实例。

1.liveness

存活探针，检测服务是否运行正常

1.1 exec方式

采用以下过exec方式例子，启动dp

cat liveness-exec.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-exec
  labels:
    app: liveness
spec:
  containers:
  - name: liveness
    image: busybox
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 600
    livenessProbe:
      initialDelaySeconds: 10
      periodSeconds: 3
      failureThreshold: 3
      successThreshold: 1
      timeoutSeconds: 1
      exec:
        command:
        - cat
        - /tmp/healthy

启动

kubectl appley -f  liveness-exec.yaml

结论：删除pod里的/tmp/healthy,可以看到检测很快，但killing状态持续30秒左右

 Normal   Scheduled  2m2s               default-scheduler  Successfully assigned default/liveness-exec to k8s-node01
  Normal   Pulled     116s               kubelet            Successfully pulled image "busybox" in 5.486523665s
  Normal   Created    116s               kubelet            Created container liveness
  Normal   Started    116s               kubelet            Started container liveness
  Warning  Unhealthy  32s (x3 over 38s)  kubelet            Liveness probe failed: cat: can\'t open \'/tmp/healthy\': No such file or directory
  Normal   Killing    32s                kubelet            Container liveness failed liveness probe, will be restarted

1.2 http方式

cat liveness-http.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-http
  labels:
    test: liveness
spec:
  containers:
  - name: liveness
    image: mydlqclub/springboot-helloworld:0.0.1
    livenessProbe:
      initialDelaySeconds: 20
      periodSeconds: 3
      failureThreshold: 3
      timeoutSeconds: 2
      httpGet:
        scheme: HTTP
        port: 8081
        path: /actuator/health

运行后查看

[root@iz0jl52ythz64abom43swwz ~]#  kubectl  get pod -o wide|grep liveness-http
liveness-http                  1/1     Running       1          16h     172.16.1.34   k8s-node01   <none>           <none>
[root@iz0jl52ythz64abom43swwz ~]# curl 172.16.1.34^C
[root@iz0jl52ythz64abom43swwz ~]# curl 172.16.1.34:8081/actuator/health
{"status":"UP"}
[root@iz0jl52ythz64abom43swwz ~]# curl 172.16.1.34:8081/actuator/health
{"status":"UP"}
[root@iz0jl52ythz64abom43swwz ~]#

结论：直接kill容器里服务后，也是30秒后自动重建

1.3 tcp方式

cat liveness-tcp2.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-tcp
  labels:
    app: liveness
spec:
  containers:
  - name: liveness
    image: cloudnativelabs/whats-my-ip
    livenessProbe:
      initialDelaySeconds: 15
      periodSeconds: 3
      failureThreshold: 3
      timeoutSeconds: 1

      tcpSocket:
        port: 8080

结论：killall容器里服务后，也是30s左右

1.4 总结

使用exec、http、tcp方式，均在30s左右重建与恢复。

2、readiness

就绪探针，验证容器服务启动是否正常

2.1 exec方式

cat readliness-exec.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: readliness-exec
  name: readliness-exec
spec:
  replicas: 1
  selector:
    matchLabels:
      run: readliness-exec
  template:
    metadata:
      labels:
        run: readliness-exec
    spec:
      containers:
      - image: cloudnativelabs/whats-my-ip
        name: readliness-exec
        ports:
        - containerPort: 8080
        readinessProbe:
          initialDelaySeconds: 60
          periodSeconds: 3
          failureThreshold: 3
          successThreshold: 1
          timeoutSeconds: 1
          exec:
            command:
            - cat
            - /tmp/healthy
      terminationGracePeriodSeconds: 30

运行后可以看懂ready是0/1，此时容器虽然在running，但实际为未就绪状态。原因是我没有创建 /tmp/healthy文件，所以检测失败，符合预期

[root@iz0jl52ythz64abom43swwz ~]# kubectl  get pod
NAME                              READY   STATUS        RESTARTS   AGE
readliness-exec-59cfc454d-dgxx8   0/1     Running       0          3m21s

可以看到监控检测是失败的

  Normal   Scheduled  3m24s                 default-scheduler  Successfully assigned default/readliness-exec-59cfc454d-dgxx8 to k8s-node01
  Normal   Pulling    3m25s                 kubelet            Pulling image "cloudnativelabs/whats-my-ip"
  Normal   Pulled     3m24s                 kubelet            Successfully pulled image "cloudnativelabs/whats-my-ip" in 851.613807ms
  Normal   Created    3m24s                 kubelet            Created container readliness-exec
  Normal   Started    3m24s                 kubelet            Started container readliness-exec
  Warning  Unhealthy  82s (x21 over 2m22s)  kubelet            Readiness probe failed: cat: can\'t open \'/tmp/healthy\': No such file or directory

在容器里创建/tmp/healthy文件,可以看到状态已经为1/1

[root@iz0jl52ythz64abom43swwz ~]# kubectl  get pod
NAME                              READY   STATUS        RESTARTS   AGE
readliness-exec-59cfc454d-dgxx8   1/1     Running       0          3m31s

在删除此文件后，又变为了0/1

  Warning  Unhealthy  6s (x47 over 8m24s)  kubelet            Readiness probe failed: cat: can\'t open \'/tmp/healthy\': No such file or directory
[root@iz0jl52ythz64abom43swwz ~]# kubectl  get pod
NAME                              READY   STATUS        RESTARTS   AGE
readliness-exec-59cfc454d-dgxx8   0/1     Running       0          9m28s

添加svc后观察

kubectl expose deployment readliness-exec --name=readliness-exec --port=80 --target-port=8080

​访问失败

[root@iz0jl52ythz64abom43swwz ~]# kubectl get svc
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes        ClusterIP   10.96.0.1       <none>        443/TCP   12d
readliness-exec   ClusterIP   10.99.194.102   <none>        80/TCP    5m46s
whats-my-ip       ClusterIP   10.103.25.20    <none>        80/TCP    11d
[root@iz0jl52ythz64abom43swwz ~]# curl 10.99.194.102
curl: (7) Failed connect to 10.99.194.102:80; Connection refused

在实例里添加监控检测文件后,访问正常

kubectl  get pod
NAME                              READY   STATUS        RESTARTS   AGE
readliness-exec-59cfc454d-dgxx8   1/1     Running       0          36m
[root@iz0jl52ythz64abom43swwz ~]# curl 10.99.194.102
HOSTNAME:readliness-exec-59cfc454d-dgxx8 IP:172.16.1.42
[root@iz0jl52ythz64abom43swwz ~]# curl 10.99.194.102
HOSTNAME:readliness-exec-59cfc454d-dgxx8 IP:172.16.1.42

2.2 结论

如果监听的文件不存在，那么容器READY状态会有异常，并且访问有问题。