发送到kafka主题时序列化消息出错
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了发送到kafka主题时序列化消息出错相关的知识,希望对你有一定的参考价值。
我需要测试一个包含标题的消息,所以我需要使用MessageBuilder,但我无法序列化。
我尝试在生产者道具上添加序列化设置,但它不起作用。
有人能帮我吗?
这个错误:
org.apache.kafka.common.errors.SerializationException: Can't convert value of class org.springframework.messaging.support.GenericMessage to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
我的考试班:
public class TransactionMastercardAdapterTest extends AbstractTest
@Autowired
private KafkaTemplate<String, Message<String>> template;
@ClassRule
public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1);
@BeforeClass
public static void setUp()
System.setProperty("spring.kafka.bootstrap-servers", embeddedKafka.getBrokersAsString());
System.setProperty("spring.cloud.stream.kafka.binder.zkNodes", embeddedKafka.getZookeeperConnectionString());
@Test
public void sendTransactionCommandTest()
String payload = "\"o2oTransactionId\" : \"" + UUID.randomUUID().toString().toUpperCase() + "\","
+ "\"cardId\" : \"11\","
+ "\"transactionId\" : \"20110405123456\","
+ "\"amount\" : 200.59,"
+ "\"partnerId\" : \"11\"";
Map<String, Object> props = KafkaTestUtils.producerProps(embeddedKafka);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
Producer<String, Message<String>> producer = new KafkaProducer<>(props);
producer.send(new ProducerRecord<String, Message<String>> ("notification_topic", MessageBuilder.withPayload(payload)
.setHeader("status", "RECEIVED")
.setHeader("service", "MASTERCARD")
.build()));
Map<String, Object> configs = KafkaTestUtils.consumerProps("test1", "false", embeddedKafka);
configs.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
ConsumerFactory<byte[], byte[]> cf = new DefaultKafkaConsumerFactory<>(configs);
Consumer<byte[], byte[]> consumer = cf.createConsumer();
consumer.subscribe(Collections.singleton("transaction_topic"));
ConsumerRecords<byte[], byte[]> records = consumer.poll(10_000);
consumer.commitSync();
assertThat(records.count()).isEqualTo(1);
答案
我说错误是显而易见的:
Can't convert value of class org.springframework.messaging.support.GenericMessage to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
你的价值是GenericMessage
,但StringSerializer
只适用于字符串。
你需要的是所谓的JavaSerializer
,它不存在,但写起来并不那么难:
public class JavaSerializer implements Serializer<Object>
@Override
public byte[] serialize(String topic, Object data)
try
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ObjectOutputStream objectStream = new ObjectOutputStream(byteStream);
objectStream.writeObject(data);
objectStream.flush();
objectStream.close();
return byteStream.toByteArray();
catch (IOException e)
throw new IllegalStateException("Can't serialize object: " + data, e);
@Override
public void configure(Map<String, ?> configs, boolean isKey)
@Override
public void close()
并为该value.serializer
属性配置它。
另一答案
private void configureProducer()
Properties props = new Properties();
props.put("key.serializer",
"org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer",
"org.apache.kafka.common.serialization.ByteArraySerializer");
producer = new KafkaProducer<String, String>(props);
这将完成这项工作。
另一答案
使用@XmlRootElement
注释JSON类
以上是关于发送到kafka主题时序列化消息出错的主要内容,如果未能解决你的问题,请参考以下文章
Kafka Connect:如何使用 hdfs sink 连接器将 Kafka 主题的 protobuf 数据发送到 HDFS?