[k8s]coredns/kube-dns配置subdomain

Posted 2020-10-16 毛台

思想: kube-dns或coredns本质上是一个dns服务软件.都需要配置配置文件.要控制怎么查询,即控制他的配置文件即可.
本文先说下coredns怎么配置,然后在配下kube-dns(包含了外建dnsmasq搭建,模拟集群访问公司私有域情景)

参考:
https://coredns.io/2017/03/01/coredns-for-kubernetes-service-discovery-take-2/

https://coredns.io/2017/05/08/custom-dns-entries-for-kubernetes/
https://coredns.io/2017/06/08/how-queries-are-processed-in-coredns/

默认的kube-dns策略

本次模拟架构如下图:

coredns配置文件:

• 1.访问cluster.local后缀的,去查10.254.0.2
• 2.访问out-of.kubernetes的如server.out-of.kubernetes去查192.168.x.x
• 3.访问互联网的,走resolve.conf的地址

    .:53 {
        errors      # show errors
        log stdout  # show query logs
        health
        kubernetes cluster.local 10.254.0.0/16
        proxy out-of.kubernetes 192.168.x.x
        proxy . /etc/resolv.conf

参考:
https://coredns.io/2017/03/01/coredns-for-kubernetes-service-discovery-take-2/

$ cat coredns.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        log stdout
        health
        kubernetes cluster.local 10.254.0.0/16
        proxy out-of.kubernetes 192.168.x.x
        proxy . /etc/resolv.conf
        cache 30
    }
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: coredns
  namespace: kube-system
  labels:
    k8s-app: coredns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "CoreDNS"
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: coredns
  template:
    metadata:
      labels:
        k8s-app: coredns
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: \'\'
        scheduler.alpha.kubernetes.io/tolerations: \'[{"key":"CriticalAddonsOnly", "operator":"Exists"}]\'
    spec:
      containers:
      - name: coredns
        image: coredns/coredns:latest
        imagePullPolicy: Always
        args: [ "-conf", "/etc/coredns/Corefile" ]
        volumeMounts:
        - name: config-volume
          mountPath: /etc/coredns
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
      dnsPolicy: Default
      volumes:
        - name: config-volume
          configMap:
            name: coredns
            items:
            - key: Corefile
              path: Corefile
---
apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  labels:
    k8s-app: coredns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "CoreDNS"
spec:
  selector:
    k8s-app: coredns
  clusterIP: 10.254.0.2
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP

kube-dns配置文件

部署参考: http://www.cnblogs.com/iiiiher/p/7891713.html

有3个文件,修改cm即可.

$ ls
kubedns-cm.yaml  kubedns-deployment.yaml  kubedns-svc.yaml

$ cat dns-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-dns
  namespace: kube-system
data:
  stuDomains: |
    {"out-of.kubernetes": {"192.168.x.x"}
  upsteamNameservers: |
    {"114.114.114.114","9.9.9.9"}

配置dnsmasq--外建:模拟公司私有的dns服务器

yum install -y dnsmasq tcpdump

echo "192.168.8.191 server.out-of.kubernetes" > /tmp/hosts


启动:
dnsmasq -q -d -h -R -H /tmp/hosts

-d debug模式
-q 输出查询记录
-h 不使用/etc/hosts
-R 不使用/etc/resolve.conf
-H 使用自定义的文件作为DNS记录


tcpdump -i eth0 udp port 53 -nnv
host -t A server.out-of.kubernetes 192.168.x.x

测试:本地我用coredns

$ kubectl run -it --rm --restart=Never busybox --image=busybox sh
/ # nslookup  server.out-of.kubernetes
Server:    10.254.0.2
Address 1: 10.254.0.2 kube-dns.kube-system.svc.cluster.local

Name:      server.out-of.kubernetes
Address 1: 192.168.x.x

分别访问集群/out-of.kubernetes/外网都可以通信.

同一个域名解析到2个ip如何实现?(headless svc)

参考:http://www.cnblogs.com/cuihongyu3503319/archive/2012/07/09/2583129.html
同一个域名 添加2条不同ip即可.

$ cat /tmp/hosts 
192.168.x.191 server.out-of.kubernetes
192.168.x.192 server.out-of.kubernetes

$ dnsmasq -q -d -h -R -H /tmp/hosts

$ host -t A server.out-of.kubernetes 192.168.x.x
Using domain server:
Name: 192.168.x.x
Address: 192.168.x.x#53
Aliases: 

server.out-of.kubernetes has address 192.168.x.191
server.out-of.kubernetes has address 192.168.x.192

• todo
  cordns k8s插件细节,将集群内的dns移到集群外.
  coredns放在集群里,logs -f看不到日志,目测得抽时间细究下.
  coredns只开放A记录,关闭AAAA功能.

每个svc添加两条记录

<service_name>.<namespace_name>.<domain>        # 没想到这种有什么用
<service_name>.<namespace_name>.svc.<domain>  # 好像kube-dns解析出的都是这种带svc的.


$ cat nginx-svc.yaml 
kind: Service
apiVersion: v1
metadata:
  name: svc-nginx
spec:
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 80


/ # nslookup  svc-nginx
Address 1: 10.254.164.42 svc-nginx.default.svc.cluster.local

/ # nslookup  svc-nginx.default.svc.cluster.local
Address 1: 10.254.164.42 svc-nginx.default.svc.cluster.local

/ # nslookup  svc-nginx.default.cluster.local
nslookup: can\'t resolve \'svc-nginx.default.cluster.local\'