问题：使用ogg for bigdata实现数据传输到kafka，在做update操做时出现问题

Posted 2023-03-22

问题：使用ogg for bigdata实现数据传输到kafka，在做update操做时，before数据没有显示，after中只有被修改的那一列数据，其它列也不显示，需要怎么配置才能使显示的消息更完整。此处使用的是json格式。

官方显示的json格式时update操作的显示：
6.3.2.2 Sample Update Message

"table":"GG.TCUSTORD",
"op_type":"U",
"op_ts":"2013-06-02 22:14:41.000000",
"current_ts":"2015-09-18T13:39:35.748000",
"pos":"00000000000000002891",
"tokens":
"R":"AADPkvAAEAAEqLzAAA"
,
"before":
"CUST_CODE":"BILL",
"ORDER_DATE":"1995-12-31:15:00:00",
"PRODUCT_CODE":"CAR",
"ORDER_ID":"765",
"PRODUCT_PRICE":15000.00,
"PRODUCT_AMOUNT":3,
"TRANSACTION_ID":"100"
,
"after":
"CUST_CODE":"BILL",
"ORDER_DATE":"1995-12-31:15:00:00",
"PRODUCT_CODE":"CAR",
"ORDER_ID":"765",
"PRODUCT_PRICE":14000.00,
"PRODUCT_AMOUNT":3,
"TRANSACTION_ID":"100"



自己测试时update的显示：

"table":"FAFASCHEMA.P",
"op_type":"U",
"op_ts":"2017-02-07 09:09:19.996232",
"current_ts":"2017-02-07T17:09:26.783000",
"pos":"00000000260000002118",
"primary_keys":[
"ID"
],
"tokens":
"TK-HOST":"ray",
"TK-OSUSER":"oracle",
"TK-SCN":"1346678"
,
"before":
,
"after":
"ID":"10",
"NAME":"qqqqqqqq"



我的kafka.props配置：
gg.handlerlist = kafkahandler
gg.handler.kafkahandler.type = kafka
gg.handler.kafkahandler.KafkaProducerConfigFile = custom_kafka_producer.properties
gg.handler.kafkahandler.TopicName = ggtopic
gg.handler.kafkahandler.BlockingSend = false
gg.handler.kafkahandler.includeTokens = true

gg.handler.kafkahandler.Mode = tx
gg.handler.kafkah andler.topicPartitioning = none

#json
gg.handler.kafkahandler.format = json
gg.handler.kafkahandler.format.insertOpKey = I
gg.handler.kafkahandler.format.updateOpKey = U
gg.handler.kafkahandler.format.deleteOpKey = D
gg.handler.kafkahandler.format.prettyPrint = true
gg.handler.kafkahandler.format.jsonDelimiter = CDATA[]
gg.handler.kafkahandler.format.generateSchema = true
gg.handler.kafkahandler.format.schemaDirectory = dirdef
#gg.handler.kafkahandler.format.treatAllColumnsAsString = true
gg.handler.kafkahandler.format.includePrimaryKeys = true

#gg.handler.kafkahandler.maxGroupSize =100, 1Mb
#gg.handler.kafkahandler.minGroupSize =50, 500Kb
goldengate.userexit.timestamp=utc
goldengate.userexit.writers=javawriter
javawriter.stats.display=TRUE
javawriter.stats.full=TRUE
gg.log=log4j
gg.log.level=INFO
gg.report.time=30sec
gg.classpath=dirprm/:/root/kafka/kafka_2.10-0.9.0.1/libs/*:
javawriter.bootoptions=-Xmx512m -Xms32m -Djava.class.path=ggjava/ggjava.jar

参考技术A 加一个参数getUpdateBefores，
另外可能还要修改日志模式为全日志模式本回答被提问者和网友采纳