OGG,大表拆分

Posted dataroot123

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了OGG,大表拆分相关的知识,希望对你有一定的参考价值。

OGG进程拆分(单表拆成多个进程)
 
概要:
《OGG进程拆分》介绍了如何将一个入库进程中的多个表拆分到其他进程中。本篇将着重介绍如何使用多个进程同时入库一张表。
适用条件:
1)入库进程只同步一张表,但仍有延时
2)目标段主机CPU、内存压力不大,以便有足够的资源添加新的入库进程
 
本示例将RZG_CXI2中的HX_SB.SB_CWBB_XQYKJZZ_ZCFZB表拆分到RZG_CXI1-RZG_CXI8这8个进程中
1. 创建新添加进程的配置文件
edit  params RZG_CXI1
 ##添加以下内容##
REPLICAT RZG_CXI1
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI1.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (1, 8));
##结束##
 
 edit  params RZG_CXI2
 ##开始##
REPLICAT RZG_CXI2
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI2.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (2, 8));
 ##结束##
 
 edit  params RZG_CXI3
 ##开始##
REPLICAT RZG_CXI3
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI3.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (3, 8));
 ##结束##
 
 edit  params RZG_CXI4
 ##开始##
REPLICAT RZG_CXI4
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI4.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
 MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (4, 8));
 ##结束##
 
 edit  params RZG_CXI5
 ##开始##
REPLICAT RZG_CXI5
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI5.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
  MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (5, 8));
 ##结束##
 
 edit  params RZG_CXI6
 ##开始##
REPLICAT RZG_CXI6
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI6.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (6, 8));                   
 ##结束##
 
 edit  params RZG_CXI7
 ##开始##
REPLICAT RZG_CXI7
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI7.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
 MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (7, 8));
 ##结束##
 
 edit  params RZG_CXI8
 ##开始##
REPLICAT RZG_CXI8
SETENV (ORACLE_SID="hasthxcx1")
SETENV (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
REPORT AT 01:59
REPORTROLLOVER AT 02:00
CACHEMGR, CACHESIZE 256MB
REPERROR DEFAULT,ABEND
DISCARDFILE ./dirrpt/RZG_CXI8.dsc,APPEND,MEGABYTES 100
DISCARDROLLOVER AT 06:00
GETTRUNCATES
NUMFILES 3000
EOFDELAYCSECS 30
ASSUMETARGETDEFS
ALLOWNOOPUPDATES
DYNAMICRESOLUTION
GETUPDATEBEFORES
NOCOMPRESSDELETES
MAP HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , TARGET HX_SB.SB_CWBB_XQYKJZZ_ZCFZB , colmap (usedefaults ,  &
    sjtb_sj = @date ("yyyy-mm-dd hh:mi:ss.ffffff", "JTS", @GETENV("JULIANTIMESTAMP"))),  FILTER (@RANGE (8, 8));
 ##结束##
 
 
  1,查看 日志序列号和rba (添加进程时注意修改trail文件名,日志序号,rba)
 
   等stop后查看 info RZG_CX2I
 
 
GGSCI (hasthxcxdb01) 33>  info RZG_CX2I
 
REPLICAT   RZG_CX2I  Last Started 2016-06-02 11:39   Status RUNNING
Checkpoint Lag       91:53:11 (updated 03:12:24 ago)
Log Read Checkpoint  File ./dirdat/za003124
                     2016-05-30 19:43:03.015049  RBA 114276026
 
   2,登录数据库
dblogin Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT ##密文部分为OGG用户ggs的密码,也可用明文
 
  3.,添加进程 RZG_CXI
上面已经编辑过8个进程的参数文件,这里添加进程即可
   $ ggsci
   add replicat RZG_CXI1, exttrail ./dirdat/za
   add replicat RZG_CXI2, exttrail ./dirdat/za
   add replicat RZG_CXI3, exttrail ./dirdat/za
   add replicat RZG_CXI4, exttrail ./dirdat/za
   add replicat RZG_CXI5, exttrail ./dirdat/za
   add replicat RZG_CXI6, exttrail ./dirdat/za
   add replicat RZG_CXI7, exttrail ./dirdat/za
   add replicat RZG_CXI8, exttrail ./dirdat/za
 
 
  4, 修改进程检查点
   $ ggsci
    alter replicat RZG_CXI1, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI2, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI3, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI4, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI5, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI6, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI7, extseqno 3124, extrba 114276026
   alter replicat RZG_CXI8, extseqno 3124, extrba 114276026   
 
 
  5,启动新增的8个进程
 start RZG_CXI*
 
 
  6,再次登录数据库,删除原进程
  dblogin Userid ggs,PASSWORD 密码, BLOWFISH, ENCRYPTKEY DEFAULT
   delete RZG_CX2I

以上是关于OGG,大表拆分的主要内容,如果未能解决你的问题,请参考以下文章

OGG进程拆分(单表拆成多个进程)

SQL - 根据访问频率拆分大表?

mysql把一个大表拆分多个表后,如何解决跨表查询效率问题

根据每行列的内容将一个大表拆分为多个表

将大表查询拆分为对表子集的多个查询是不是有意义?

SQL性能问题.现在表设计可以把一个大表按类型(各类型字段不相同)拆分成多个小表.拆分后比较方便.