logstash-input-jdbc 同时同步多个表的情况

Posted 兔六哥

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了logstash-input-jdbc 同时同步多个表的情况相关的知识,希望对你有一定的参考价值。

input {
    jdbc {
      jdbc_connection_string => "jdbc:mysql://localhost:3306/crm?zeroDateTimeBehavior=convertToNull"
      jdbc_user => "root"
      jdbc_password => ""
      jdbc_driver_library => "D:/siyang/elasticsearch-5.2.2/logstash-5.2.2/mysql-connector-java-5.1.30.jar"
      jdbc_driver_class => "com.mysql.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      statement_filepath => "filename.sql"
      schedule => "* * * * *"
      type => "jdbc_office"
    }
    jdbc {
      jdbc_connection_string => "jdbc:mysql://localhost:3306/crm?zeroDateTimeBehavior=convertToNull"
      jdbc_user => "root"
      jdbc_password => "" 
      jdbc_driver_library => "D:/siyang/elasticsearch-5.2.2/logstash-5.2.2/mysql-connector-java-5.1.30.jar"
      jdbc_driver_class => "com.mysql.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      statement => "SELECT * from sys_user"
      schedule => "* * * * *"
      type => "jdbc_user"
    }
}
output {
 stdout {
        codec => json_lines
    }
    if[type] == "jdbc_office"{
        elasticsearch {
        hosts  => "localhost:9200"
        index => "contacts4"
        document_type => "office1"
        document_id => "%{id}"
        }
    }
    if[type] == "jdbc_user"{
        elasticsearch {
        hosts  => "localhost:9200"
        index => "contacts4"
        document_type => "user1"
        document_id => "%{id}"
        }
    }
}

注意事项,数据库里面的字段不要用type,如果有,as成其他的名字,不然的话,这里判断会有异常

以上是关于logstash-input-jdbc 同时同步多个表的情况的主要内容,如果未能解决你的问题,请参考以下文章

利用 Logstash-input-jdbc同步sqlserver数据到elasticsearch

和我一起打造个简单搜索之Logstash实时同步建立索引

Windows系统上安装logstash和logstash-input-jdbc

logstsh | logstash-input-jdbc 启动错误收集

mysql准实时同步数据到Elasticsearch

logstash-input-jdbc配置说明