kubernetes 1.14安装部署metrics-server插件

Posted 2021-11-01 ding2016

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了kubernetes 1.14安装部署metrics-server插件相关的知识，希望对你有一定的参考价值。

简单介绍：

如果使用kubernetes的自动扩容功能的话，那首先得有一个插件，然后该插件将收集到的信息（cpu、memory..）与自动扩容的设置的值进行比对，自动调整pod数量。关于该插件，在kubernetes的早些版本中采用的是heapster，1.13版本正式发布后，丢弃了heapster，官方推荐采用metrics-sever。

测试环境：

同之前部署的集群环境，如下：

System	Hostname	IP
CentOS 7.6	k8s-master	138.138.82.14
CentOS 7.6	k8s-node1	138.138.82.15
CentOS 7.6	k8s-node2	138.138.82.16

操作步骤：

1. 下载相关yaml文件

https://github.com/kubernetes-incubator/metrics-server

~]# git clone https://github.com/kubernetes-incubator/metrics-server.git

[[email protected] ~]# cd metrics-server/deploy/1.8+/
[[email protected]-master 1.8+]# ll
总用量 28
-rw-r--r-- 1 root root 384 4月  28 09:46 aggregated-metrics-reader.yaml
-rw-r--r-- 1 root root 308 4月  28 09:46 auth-delegator.yaml
-rw-r--r-- 1 root root 329 4月  28 09:46 auth-reader.yaml
-rw-r--r-- 1 root root 298 4月  28 09:46 metrics-apiservice.yaml
-rw-r--r-- 1 root root 815 4月  28 09:46 metrics-server-deployment.yaml
-rw-r--r-- 1 root root 291 4月  28 09:46 metrics-server-service.yaml
-rw-r--r-- 1 root root 502 4月  28 09:46 resource-reader.yaml

2. 修改其中的metrics-server-deployment.yaml文件（用红色标亮处）

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: metrics-server
  namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: metrics-server
  namespace: kube-system
  labels:
    k8s-app: metrics-server
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  template:
    metadata:
      name: metrics-server
      labels:
        k8s-app: metrics-server
    spec:
      serviceAccountName: metrics-server
      volumes:
      # mount in tmp so we can safely use from-scratch images and/or read-only containers
      - name: tmp-dir
        emptyDir: {}
      containers:
      - name: metrics-server
        image: mirrorgooglecontainers/metrics-server-amd64:v0.3.2
      # image: k8s.gcr.io/metrics-server-amd64:v0.3.2
        imagePullPolicy: IfNotPresent
        command:
            - /metrics-server
            - --kubelet-preferred-address-types=InternalIP
            - --kubelet-insecure-tls
        volumeMounts:
        - name: tmp-dir
          mountPath: /tmp

注意：以上修改了3个地方：①是被墙的镜像；②镜像拉取策略；③添加命令和相关参数；

如果不修改command区域的参数则会报如下错误：

E0428 19:35:05.894325       1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:k8s-node2: unable to fetch metrics from Kubelet k8s-node2 (138.138.82.16): Get https://138.138.82.16:10250/stats/summary/: x509: cannot validate certificate for 138.138.82.16 because it doesn‘t contain any IP SANs, unable to fully scrape metrics from source kubelet_summary:k8s-node1: unable to fetch metrics from Kubelet k8s-node1 (138.138.82.15): Get https://138.138.82.15:10250/stats/summary/: x509: cannot validate certificate for 138.138.82.15 because it doesn‘t contain any IP SANs, unable to fully scrape metrics from source kubelet_summary:k8s-master: unable to fetch metrics from Kubelet k8s-master (138.138.82.14): Get https://138.138.82.14:10250/stats/summary/: x509: cannot validate certificate for 138.138.82.14 because it doesn‘t contain any IP SANs]

附上metrcs-server命令参数详解：

Usage:
[flags]

Flags:
--alsologtostderr log to standard error as well as files
--authentication-kubeconfig string kubeconfig file pointing at the ‘core‘ kubernetes server with enough rights to create tokenaccessreviews.authentication.k8s.io.
--authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster.
--authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s)
--authorization-kubeconfig string kubeconfig file pointing at the ‘core‘ kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io.
--authorization-webhook-cache-authorized-ttl duration The duration to cache ‘authorized‘ responses from the webhook authorizer. (default 10s)
--authorization-webhook-cache-unauthorized-ttl duration The duration to cache ‘unauthorized‘ responses from the webhook authorizer. (default 10s)
--bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces). (default 0.0.0.0)
--cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "apiserver.local.config/certificates")
--client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
--contention-profiling Enable lock contention profiling, if profiling is enabled
--enable-swagger-ui Enables swagger ui on the apiserver at /swagger-ui
-h, --help help for this command
--http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang‘s default.
--kubeconfig string The path to the kubeconfig used to connect to the Kubernetes API server and the Kubelets (defaults to in-cluster config)
--kubelet-certificate-authority string Path to the CA to use to validate the Kubelet‘s serving certificates.
--kubelet-insecure-tls Do not verify CA of serving certificates presented by Kubelets. For testing purposes only.
--kubelet-port int The port to use to connect to Kubelets. (default 10250)
--kubelet-preferred-address-types strings The priority of node address types to use when determining which address to use to connect to a particular node (default [Hostname,InternalDNS,InternalIP,ExternalDNS,ExternalIP])
--log-flush-frequency duration Maximum number of seconds between log flushes (default 5s)
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory
--logtostderr log to standard error instead of files (default true)
--metric-resolution duration The resolution at which metrics-server will retain metrics. (default 1m0s)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.
--requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests.
--requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-])
--requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group])
--requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user])
--secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don‘t serve HTTPS at all. (default 443)
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
--tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir.
--tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be use. Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA
--tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12
--tls-private-key-file string File containing the default x509 private key matching --tls-cert-file.
--tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default [])
-v, --v Level log level for V logs
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging

View Code

3. 应用所有配置文件到系统中

[[email protected] 1.8+]# kubectl apply -f .
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.extensions/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created

过个一两分钟（下载镜像和获取数据都会耗时）检查metrics-server的状态

[[email protected] 1.8+]# kubectl get po -n kube-system
NAME                                 READY   STATUS    RESTARTS   AGE
calico-node-b78m4                    1/1     Running   0          176m
calico-node-r5mlj                    1/1     Running   0          3h6m
calico-node-z5tdh                    1/1     Running   0          176m
coredns-fb8b8dccf-6mgks              1/1     Running   0          3h21m
coredns-fb8b8dccf-cbtlx              1/1     Running   0          3h21m
etcd-k8s-master                      1/1     Running   0          3h20m
kube-apiserver-k8s-master            1/1     Running   0          3h20m
kube-controller-manager-k8s-master   1/1     Running   0          3h20m
kube-proxy-c9xd2                     1/1     Running   0          3h21m
kube-proxy-fp2r2                     1/1     Running   0          176m
kube-proxy-lrsw7                     1/1     Running   0          176m
kube-scheduler-k8s-master            1/1     Running   0          3h20m
metrics-server-7579f696d8-pgcc4      1/1     Running   0          99s
[[email protected]-master 1.8+]# kubectl top node
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8s-master   179m         8%     1660Mi          43%       
k8s-node1    81m          4%     908Mi           23%       
k8s-node2    78m          3%     1036Mi          26%

看的出来，metrics-server已经正常running，并且能够获取节点的信息。

结束.

以上是关于kubernetes 1.14安装部署metrics-server插件的主要内容，如果未能解决你的问题，请参考以下文章

kubernetes 1.14安装部署dashboard

kubeadm安装Kubernetes 1.14最佳实践

Kubernetes1.15.2集群部署并部署Metrics Server插件

kubernetes部署metrics-server

Kubernetes 集群部署 Metrics Server 获取集群 Metric 数据