Cassandra和Solr集成使用JDBC:org.apache.thrift.transport.TTransportException
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Cassandra和Solr集成使用JDBC:org.apache.thrift.transport.TTransportException相关的知识,希望对你有一定的参考价值。
我正在尝试整合cassandra和solr。我正在使用solr -6.6.0版本,cassandra 3.10版本和java 8。
添加所有必要的配置文件后,我无法进行集成。我错过了任何罐子或冲突的罐子吗?我也尝试使用solr 7.1.0并遇到同样的问题。任何指针都会有所帮助。
添加到solarconfig.xml
<lib dir="/Office/solr-6.6.0/dist" regex="solr-dataimporthandler-6.6.0.jar" />
<lib dir="/Office/solr-6.6.0/dist" regex="solr-dataimporthandler-extras-6.6.0.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="asm-5.0.3.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="asm-analysis-5.0.3.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="asm-commons-5.0.3.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="asm-tree-5.0.3.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="asm-util-5.0.3.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="cassandra-driver-core-3.3.0.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="cassandra-driver-extras-3.3.0.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="cassandra-jdbc-1.2.5.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="cassandra-jdbc-driver-0.6.4.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="cassandrajdbc1.1.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="guava-19.0.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jffi-1.2.10-native.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jffi-1.2.10.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jnr-constants-0.9.0.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jnr-ffi-2.0.7.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jnr-posix-3.0.27.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jnr-x86asm-1.0.2.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="joda-time-2.9.9.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="jsqlparser-0.9.6.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="lz4-1.3.0.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="metrics-core-3.2.2.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="netty-buffer-4.0.47.Final.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="netty-codec-4.0.47.Final.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="netty-common-4.0.47.Final.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="netty-handler-4.0.47.Final.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="netty-transport-4.0.47.Final.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="slf4j-api-1.7.25.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="snakeyaml-1.18.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="snappy-java-1.1.4.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc/Workingcombo2" regex="tinylog-1.2.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc" regex="cassandra-all-3.11.1.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc" regex="cassandra-thrift-3.11.1.jar" />
<lib dir="/Office/solr-6.6.0/cassandrajdbc" regex="libthrift-0.10.0.jar" />
dataconfig.xml
<dataConfig>
<dataSource type="JdbcDataSource"
driver="org.apache.cassandra.cql.jdbc.CassandraDriver"
url="jdbc:cassandra://127.0.0.1:9042/test"
autoCommit="true"/>
<document name="content">
<entity name="test"
query="SELECT a,b from test" autoCommit="true">
<!--
<field column="is_first" name="is_first" />
<field column="is_last" name="is_last" />
-->
<field column="id" name="id" />
<field column="a" name="a_s" />
<field column="b" name="b_s" />
</entity>
</document>
</dataConfig>
schema.xml中
<schema name="cassandra" version="1.1">
<uniqueKey>id</uniqueKey>
<types>
<fieldType name="string" class="solr.StrField" omitNorms="true"/>
<fieldType name="double" class="solr.DoublePointField" omitNorms="true"/>
<fieldType name="tdoubles" class="solr.DoublePointField" omitNorms="true"/>
<fieldType name="int" class="solr.IntPointField" omitNorms="true"/>
<fieldType name="long" class="solr.LongPointField" omitNorms="true"/>
<fieldType name="tlongs" class="solr.LongPointField" omitNorms="true"/>
<fieldType name="uuid" class="solr.UUIDField" indexed="true" />
<fieldType name="text_general" class="solr.TextField" indexed="true" />
<fieldType name="booleans" class="solr.BoolField" indexed="true" />
<fieldType name="tdates" class="solr.DatePointField" docValues="true" multiValued="true"/>
</types>
<fields>
<field name="id" type="string" indexed="true" stored="true" required="true"/>
<field name="a" type="string" indexed="true" stored="true" required="true"/>
<field name="b" type="string" indexed="true" stored="true"/>
<field name="_version_" type="long" indexed="true" stored="true"/>
</fields>
</schema>
Error:
2017-12-19 00:29:08.675 INFO (Thread-14) [ x:cassandraconnect] o.a.s.u.p.LogUpdateProcessorFactory [cassandraconnect] webapp=/solr path=/dataimport params={core=cassandraconnect&optimize=false&indent=on&commit=true&name=dataimport&clean=true&wt=json&command=full-import&entity=test&_=1513643343716&verbose=false} status=0 QTime=25{deleteByQuery=*:* (-1587170087790444544)} 0 275
2017-12-19 00:29:08.677 ERROR (Thread-14) [ x:cassandraconnect] o.a.s.h.d.DataImporter Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: SELECT a,b from test Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:415)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:474)
at org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:457)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: SELECT a,b from test Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
... 4 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: SELECT a,b from test Processing Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:327)
at org.apache.solr.handler.dataimport.JdbcDataSource.createResultSetIterator(JdbcDataSource.java:288)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:283)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:52)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:267)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
... 6 more
Caused by: java.sql.SQLNonTransientConnectionException: org.apache.thrift.transport.TTransportException: Read a negative frame size (-2062548992)!
at org.apache.cassandra.cql.jdbc.CassandraConnection.<init>(CassandraConnection.java:159)
at org.apache.cassandra.cql.jdbc.CassandraDriver.connect(CassandraDriver.java:92)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:192)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:172)
at org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:528)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:317)
... 14 more
Caused by: org.apache.thrift.transport.TTransportException: Read a negative frame size (-2062548992)!
at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:134)
at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:424)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:321)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:225)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.cassandra.thrift.Cassandra$Client.recv_describe_cluster_name(Cassandra.java:1247)
at org.apache.cassandra.thrift.Cassandra$Client.describe_cluster_name(Cassandra.java:1235)
at org.apache.cassandra.cql.jdbc.CassandraConnection.<init>(CassandraConnection.java:130)
... 19 more
请帮忙!
答案
您正尝试使用CQL端口9042上的JDBC协议连接到Cassandra。您需要通过调用nodetool enablethrift
启用Thrift支持,然后您应该能够使用端口9160(或Cassandra配置中的任何内容)进行连接
以上是关于Cassandra和Solr集成使用JDBC:org.apache.thrift.transport.TTransportException的主要内容,如果未能解决你的问题,请参考以下文章
使用 datastax 进行错误的 SOlr 索引。从 1 个节点扩展到 2 个 cassandra 节点和 1 个搜索节点
无法使用 cassandra datastax 生成 solr 架构