[svc]logstash和filebeat之间ssl加密

Posted 2020-10-18 毛台

cfssl生成证书

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O     /usr/local/bin/cfssl
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O      /usr/local/bin/cfssljson
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -O /usr/local/bin/cfssl-certinfo
chmod +x /usr/local/bin/cfssl*


cd;mkdir keys;cd keys
cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "app": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "8760h"
      }
    }
  }
}
EOF


cat > ca-csr.json <<EOF
{
  "CN": "k8s",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF


cfssl gencert -initca ca-csr.json | cfssljson -bare ca



cd /root/keys
cat > app-csr.json <<EOF
{
  "CN": "app",
  "hosts": [
    "127.0.0.1",
    "192.168.1.11",
    "192.168.1.12"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=/root/keys/ca.pem \\
  -ca-key=/root/keys/ca-key.pem \\
  -config=/root/keys/ca-config.json \\
  -profile=app app-csr.json | cfssljson -bare app

openssl x509  -noout -text -in  app.pem

可以看到san里包含了n1 和 n2的ip. 这里计划logstash(的ip)和filebeat(的ip)使用同一套证书

实验环境

logstash&filebeat之间传数据-明文

logstash配置

cat > pipeline.conf <<EOF
input {
  beats {
    port => 5044
  }
  
  stdin {
    codec => "json"
  }
}

output {
  stdout { codec => rubydebug }
}
EOF

bin/logstash -f pipeline.conf --config.reload.automatic

filebeat多输入(不能多输出)参考: https://www.elastic.co/guide/en/beats/filebeat/current/migration-configuration.html

cat > filebeat.yml <<EOF
filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /tmp/ma.txt
- type: stdin
  
output.logstash:
  hosts: ["192.168.1.11:5044"]
# output.console:
#  pretty: true
EOF


./filebeat -e -c filebeat.yml -d "publish"

测试文字

helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld helloworld 

wireshark抓包: 不加密的时候,可以看到这玩意依稀可以看到依稀传输内容,如果互联网传输的话会有隐患.

logstash&filebeat之间传数据-ssl加密

• logstash配置ssl

参考: https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ssl-logstash.html

cat > pipeline.conf <<EOF
input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/root/keys/ca.pem"]
    ssl_certificate => "/root/keys/app.pem"
    ssl_key => "/root/keys/app-key.pem"
    ssl_verify_mode => "force_peer"
  }
  
  stdin {
    codec => "json"
  }
}

output {
  stdout { codec => rubydebug }
}
EOF


bin/logstash -f pipeline.conf --config.reload.automatic

• filebeat配置ssl

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /tmp/ma.txt
    
output.logstash:
  hosts: ["192.168.1.12:5043"]

output.logstash.ssl.certificate_authorities: ["/root/keys/ca.pem"]
output.logstash.ssl.certificate: "/root/keys/app.pem"
output.logstash.ssl.key: "/root/keys/app-key.pem"


output.console:
  pretty: true

  
./filebeat -e -c filebeat.yml -d "publish"

wireshark抓包: 看不到任何传输内容,依稀看到证书的subject(公开的).

报错doesn\'t contain any IP SANs

2017/12/24 02:33:59.242540 output.go:74: ERR Failed to connect: x509: cannot validate certificate for 192.168.1.11 because it doesn\'t contain any IP SANs
2017/12/24 02:34:15.289558 output.go:74: ERR Failed to connect: x509: cannot validate certificate for 192.168.1.11 because it doesn\'t contain any IP SANs

ssl验证流程:

报错原因: 我生成证书请求的时候 hosts字段(即san)为空.

cd /root/keys
cat > app-csr.json <<EOF
{
  "CN": "192.168.1.12",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=/root/keys/ca.pem \\
  -ca-key=/root/keys/ca-key.pem \\
  -config=/root/keys/ca-config.json \\
  -profile=app app-csr.json | cfssljson -bare app

openssl x509  -noout -text -in  app.pem