flink13.5整合hudi10
Posted wudl5566
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了flink13.5整合hudi10相关的知识，希望对你有一定的参考价值。
1. 版本
| 组件 | 版本 |
|---|---|
| hudi | 10.0 |
| flink | 13.5 |
2.hudi 源码下载
https://github.com/apache/hudi/releases
2.1 需要改flink 版本为13.5
根目录下面的pom 文件
<flink.version>1.13.5</flink.version>
<hive.version>3.1.0</hive.version>
<hadoop.version>3.1.1</hadoop.version>
2.2 编译命令
mvn clean package -DskipTests
# 或者指定scala 版本
#编译后的包
包的路径在packaging/hudi-flink-bundle/target/hudi-flink-bundle_2.12-*.*.*-SNAPSHOT.jar
2.3编译遇到一个错误
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Hudi 0.10.0:
[INFO]
[INFO] Hudi ............................................... SUCCESS [ 1.642 s]
[INFO] hudi-common ........................................ SUCCESS [ 9.808 s]
[INFO] hudi-aws ........................................... SUCCESS [ 1.306 s]
[INFO] hudi-timeline-service .............................. SUCCESS [ 1.623 s]
[INFO] hudi-client ........................................ SUCCESS [ 0.082 s]
[INFO] hudi-client-common ................................. SUCCESS [ 8.027 s]
[INFO] hudi-hadoop-mr ..................................... SUCCESS [ 2.825 s]
[INFO] hudi-spark-client .................................. SUCCESS [ 13.891 s]
[INFO] hudi-sync-common ................................... SUCCESS [ 0.718 s]
[INFO] hudi-hive-sync ..................................... SUCCESS [ 3.027 s]
[INFO] hudi-spark-datasource .............................. SUCCESS [ 0.066 s]
[INFO] hudi-spark-common_2.12 ............................. SUCCESS [ 7.706 s]
[INFO] hudi-spark2_2.12 ................................... SUCCESS [ 9.535 s]
[INFO] hudi-spark_2.12 .................................... SUCCESS [ 25.923 s]
[INFO] hudi-utilities_2.12 ................................ FAILURE [ 2.638 s]
[INFO] hudi-utilities-bundle_2.12 ......................... SKIPPED
[INFO] hudi-cli ........................................... SKIPPED
[INFO] hudi-java-client ................................... SKIPPED
[INFO] hudi-flink-client .................................. SKIPPED
[INFO] hudi-spark3_2.12 ................................... SKIPPED
[INFO] hudi-dla-sync ...................................... SKIPPED
[INFO] hudi-sync .......................................... SKIPPED
[INFO] hudi-hadoop-mr-bundle .............................. SKIPPED
[INFO] hudi-hive-sync-bundle .............................. SKIPPED
[INFO] hudi-spark-bundle_2.12 ............................. SKIPPED
[INFO] hudi-presto-bundle ................................. SKIPPED
[INFO] hudi-timeline-server-bundle ........................ SKIPPED
[INFO] hudi-hadoop-docker ................................. SKIPPED
[INFO] hudi-hadoop-base-docker ............................ SKIPPED
[INFO] hudi-hadoop-namenode-docker ........................ SKIPPED
[INFO] hudi-hadoop-datanode-docker ........................ SKIPPED
[INFO] hudi-hadoop-history-docker ......................... SKIPPED
[INFO] hudi-hadoop-hive-docker ............................ SKIPPED
[INFO] hudi-hadoop-sparkbase-docker ....................... SKIPPED
[INFO] hudi-hadoop-sparkmaster-docker ..................... SKIPPED
[INFO] hudi-hadoop-sparkworker-docker ..................... SKIPPED
[INFO] hudi-hadoop-sparkadhoc-docker ...................... SKIPPED
[INFO] hudi-hadoop-presto-docker .......................... SKIPPED
[INFO] hudi-integ-test .................................... SKIPPED
[INFO] hudi-integ-test-bundle ...................以上是关于flink13.5整合hudi10的主要内容，如果未能解决你的问题，请参考以下文章