K8S 日志收集：在K8S创建fluentd的ds

Posted 2020-11-01 不忘初心-铃

下载fluentd 的images

https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset/

docker pull 192.168.19.111/baseimages/fluentd-kubernetes-daemonset:v0.12-alpine-elasticsearch

由于镜像没有kafka插件修改镜像，dockerfile文件如下

FROM 192.168.19.111/baseimages/fluentd-kubernetes-daemonset:v0.12-alpine-elasticsearch
RUN gem install  fluent-plugin-kafka
COPY fluent.conf /fluentd/etc/fluent.conf
ENTRYPOINT ["fluentd","-c","/fluentd/etc/fluent.conf","-p","/fluentd/plugins"]

最后生成镜像：192.168.19.111/baseimages/fluentd:201803101322

创建fluentd的yaml文件：

由于fluentd要从kube-apiserver抓取镜像的相关信息，所以必须配置sa
sa文件：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd-es
  namespace: kube-system
  labels:
    k8s-app: fluentd-es
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile

创建ClusterRole文件：

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: fluentd-es
  labels:
    k8s-app: fluentd-es
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
rules:
- apiGroups:
  - ""
  resources:
  - "namespaces"
  - "pods"
  verbs:
  - "get"
  - "watch"
  - "list"

创建ClusterRoleBinding角色绑定文件

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: fluentd-es
  labels:
    k8s-app: fluentd-es
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
subjects:
- kind: ServiceAccount
  name: fluentd-es
  namespace: kube-system
  apiGroup: ""
roleRef:
  kind: ClusterRole
  name: fluentd-es
  apiGroup: ""

创建cm配置文件

kind: ConfigMap
apiVersion: v1
metadata:
  name: fluentd-config
  namespace: kube-system
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
data:
  fluent.conf: |-
    <source>     
      @type tail
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      time_format %Y-%m-%dT%H:%M:%S.%NZ
      tag kubernetes.*
      format json
    </source>
    <filter kubernetes.**>
       @type kubernetes_metadata
    </filter>
    <match **>
      @type               kafka_buffered
      brokers             192.168.7.204:9092,192.168.7.204:9093,192.168.7.204:9094
      output_data_type    json
      default_topic       test-panjunbai
      compression_codec   gzip
      required_acks       1
    </match>

创建ds文件（由于/var/log/containers能被fluentd和kube-apiserver连接做处理，必须挂载/var/log/containers的软件目的地文件路径）

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: fluentd-ds
  namespace: kube-system
  labels:
    k8s-app: fluentd
    kubernetes.io/cluster-service: "true"
spec:
  selector:
    matchLabels:
      k8s-app: fluentd
  template:
    metadata:
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ""
      labels:
        k8s-app: fluentd
        kubernetes.io/cluster-service: "true"
    spec:
      serviceAccountName: fluentd-es
      containers:
      - name: fluentd-ds
        image: 192.168.19.111/baseimages/fluentd:201803101322
        resources:
          limits:
            memory: 500Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - mountPath: /var/log
          name: varlog
        - mountPath: /fluentd/etc
          name: fluentd-cm
        - name: tz-config
          mountPath: /etc/localtime
        - name: real-dir
          mountPath: /opt/docker/containers
      terminationGracePeriodSeconds: 30
      volumes:
      - name: real-dir
        hostPath:
          path: /opt/docker/containers
      - name: varlog
        hostPath:
          path: /var/log
      - configMap:
          defaultMode: 420
          items:
          - key: fluent.conf
            path: fluent.conf
          name: fluentd-config
        name: fluentd-cm
      - name: tz-config
        hostPath:
          path: /usr/share/zoneinfo/Asia/Shanghai 

https://docs.fluentd.org/v0.12/articles/kubernetes-fluentd
https://github.com/fluent/fluent-plugin-kafka
https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter
https://kubernetes.io/docs/concepts/cluster-administration/logging/