logstash之filter处理中括号包围的内容

Posted 2020-12-27 zhzhang

 

如题，logstash之filter处理中括号包围的内容：

$grep -v "#" config/logstash-nlp.yml
input {
    kafka {
        bootstrap_servers => "datacollect-1:9092,datacollect-2:9092,datacollect-3:9092"
        codec => "json"
        group_id => "logstash-newtrace-nlptemp"
        topics => ["ot-nlp"]
    }
}
filter {
    grok {
        match => {
            "message" => "^[%{GREEDYDATA:request}]$"
        }
    }
    json {
        source => "request"
    }
    ruby {
        code => "event.set(‘temptime‘, event.get(‘@timestamp‘).time.localtime + 8*60*60);"
    }
    grok {
        match => ["temptime", "%{DATA:thedate}T%{NOTSPACE:thetime}Z"]
    }
}
output {
    if ([kafka][topic] =~ "^ot-nlp*") {
        if [name] == "nlp" {
            file {
                codec => line {format => "%{request}"}
                path => "/tmp/newtrace_nlp.log.%{thedate}"
            }
        }
    }
}

 

 

摘出来数组中的所有fields

filter {
    grok {
        match => {
            "message" => "^[%{GREEDYDATA:request}]$"
        }
    }
    json {
        source => "request"
        remove_field => [ "message", "request" ]
    }
    split {
        field => "binaryAnnotations"
    }
    json {
        source => "[binaryAnnotations][value]"
    }
}

 

感谢时总的大力支持！！！

 

核心就是正则！

 

        match => {

 

            "message" => "^[%{GREEDYDATA:request}]$"

 

        }