OGG 问题

Posted 2020-10-04 月饼馅饺子

1.启动复制时报

"ERROR OGG-15050 Oracle GoldenGate Delivery, l***.prm: Error loading Java VM runtime library: (2 No such file or directory)."

设置环境变量LD_LIBRARY_PATH,然后重启mgr,再启动rhdfs.如果不重启mgr直接启动rhdfs,则会报这个错

2. 对主键的修改无法更新，但是通过在复制上加

ERROR 2016-07-30 14:47:40,180 [main] The Delimited Text Formatter encountered a update including a primary key. The behavior is configured to ABEND in this scenario. ERROR 2016-07-30 14:47:40,181 [main] The Delimited Text Formatter formatOp operation failed. java.lang.RuntimeException: The Delimited Text Formatter encountered a update including a primary key. The behavior is configured to ABEND in this scenario. at oracle.goldengate.format.text.DelimitedTextFormatter.formatOp(DelimitedTextFormatter.java:427) at oracle.goldengate.handler.flume.FlumeHandler.formatOp(FlumeHandler.java:275) at oracle.goldengate.handler.flume.FlumeHandler.operationAdded(FlumeHandler.java:136) at oracle.goldengate.datasource.DsEventManager$3.send(DsEventManager.java:439) at oracle.goldengate.datasource.DsEventManager$EventDispatcher.distributeEvent(DsEventManager.java:231) at oracle.goldengate.datasource.DsEventManager.fireOperationAdded(DsEventManager.java:447) at oracle.goldengate.datasource.AbstractDataSource.fireOperationAdded(AbstractDataSource.java:464) at oracle.goldengate.datasource.UserExitDataSource.addOperationToTransactionAndFireEvent(UserExitDataSource.java:1337) at oracle.goldengate.datasource.UserExitDataSource.createOperation(UserExitDataSource.java:1305) at oracle.goldengate.datasource.UserExitDataSource.createOperation(UserExitDataSource.java:1096)

xxxxxxxxxx

13

 

1

ERROR 2016-07-30 14:47:40,180 [main] The Delimited Text Formatter encountered a update including a primary key.  The behavior is configured to ABEND in this scenario.

2

ERROR 2016-07-30 14:47:40,181 [main] The Delimited Text Formatter formatOp operation failed.

3

java.lang.RuntimeException: The Delimited Text Formatter encountered a update including a primary key.  The behavior is configured to ABEND in this scenario.

4

        at oracle.goldengate.format.text.DelimitedTextFormatter.formatOp(DelimitedTextFormatter.java:427)

5

        at oracle.goldengate.handler.flume.FlumeHandler.formatOp(FlumeHandler.java:275)

6

        at oracle.goldengate.handler.flume.FlumeHandler.operationAdded(FlumeHandler.java:136)

7

        at oracle.goldengate.datasource.DsEventManager$3.send(DsEventManager.java:439)

8

        at oracle.goldengate.datasource.DsEventManager$EventDispatcher.distributeEvent(DsEventManager.java:231)

9

        at oracle.goldengate.datasource.DsEventManager.fireOperationAdded(DsEventManager.java:447)

10

        at oracle.goldengate.datasource.AbstractDataSource.fireOperationAdded(AbstractDataSource.java:464)

11

        at oracle.goldengate.datasource.UserExitDataSource.addOperationToTransactionAndFireEvent(UserExitDataSource.java:1337)

12

        at oracle.goldengate.datasource.UserExitDataSource.createOperation(UserExitDataSource.java:1305)

13

        at oracle.goldengate.datasource.UserExitDataSource.createOperation(UserExitDataSource.java:1096)

Logdump 25 >pos 14848654 Reading forward from RBA 14848654 Logdump 26 >n ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x04) UndoFlag : . (x00) BeforeAfter: A (x41) RecLength : 147 (x0093) IO Time : 2016/07/30 13:34:23.006.490 IOType : 115 (x73) OrigNode : 255 (xff) TransInd : . (x03) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 13456 AuditPos : 525723664 Continued : N (x00) RecCount : 1 (x01) 2016/07/30 13:34:23.006.490 GGSPKUpdate Len 147 RBA 14848654

xxxxxxxxxx

14

 

1

Logdump 25 >pos 14848654

2

Reading forward from RBA 14848654 

3

Logdump 26 >n

4

___________________________________________________________________ 

5

Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)  

6

UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)  

7

RecLength  :   147  (x0093)   IO Time    : 2016/07/30 13:34:23.006.490   

8

IOType     :   115  (x73)     OrigNode   :   255  (xff) 

9

TransInd   :     .  (x03)     FormatType :     R  (x52) 

10

SyskeyLen  :     0  (x00)     Incomplete :     .  (x00) 

11

AuditRBA   :      13456       AuditPos   : 525723664 

12

Continued  :     N  (x00)     RecCount   :     1  (x01) 

13

14

2016/07/30 13:34:23.006.490 GGSPKUpdate          Len   147 RBA 14848654 

REPLICAT RFLUME Last Started 2016-07-30 14:47 Status ABENDED Checkpoint Lag 00:00:00 (updated 02:12:47 ago) Log Read Checkpoint File /opt/ogg/dirdat/nn000000268 2016-07-30 13:34:20.005220 RBA 14732512

x

 

1

2

REPLICAT   RFLUME    Last Started 2016-07-30 14:47   Status ABENDED

3

Checkpoint Lag       00:00:00 (updated 02:12:47 ago)

4

Log Read Checkpoint  File /opt/ogg/dirdat/nn000000268

5

                     2016-07-30 13:34:20.005220  RBA 14732512

IOTPY:

http://docs.oracle.com/goldengate/c1221/gg-winux/GWUAD/wu_fileformats.htm#g997818

捕获主键更新：

http://docs.oracle.com/goldengate/c1221/gg-winux/GWURF/gg_parameters066.htm#GWURF491

3.源表更新主键的问题

默认是会abend，可以配置为：

gg.handler.name.format.pkUpdateHandling

Optional

abend | update | delete-insert

abend

Provides configuration for how the formatter should handle update operations that change a primary key. Primary key operations can be problematic for the text formatter and require special consideration by you. abend - indicates the process will abend update - indicates the process will treat this as a normal update delete-insert - indicates the process will treat this as a delete and an insert. Full supplemental logging needs to be enabled for this to work. Without full before and after row images the insert data will be incomplete. 来源： http://docs.oracle.com/goldengate/bd1221/gg-bd/GADBD/GUID-F0FA2781-0802-4530-B1F0-5E102B982EC0.htm#GADBD494

4.ERROR   OGG-01875  Classic Extract doesn‘t support RAC parallel direct load of table.

这个问题没弄清楚,抽取是DBA那边弄的.

参考这个:

https://community.oracle.com/thread/3942758

5.kafka handler

当使用tx模式时,如果做了partition,数据会随机的发送到某个partition中,而不是按表hash,导致数据丢失.

op模式时不会,数据按表进入特定的partition

来自为知笔记(Wiz)