TimesTen 数据库复制学习：16. 一个缓存组，复制，客户端自动切换的串烧实验

Posted 2020-08-02 dingdingfish

简介

这时一个集只读，AWT缓存组，Active Standby 复制，client auto failover为一体的集成实验。
整个过程来至于Doc ID 1359840.1, 本文基于此文档按照自己的环境重做了一遍，并更正了其中的小错误，增加了自己的理解。
本文省略了在Oracle端设置缓存组的过程，可以参见前面的文章。

搭建的环境为虚拟机 timesten-hol 上安装了两个TimesTen实例, 实例名分别为tt1122和ttnew, 分别驻留在端口53392（缺省）和55555
tt1122上数据库为cachedb1，ttnew上数据库为cachedb2
虚拟机上还有一个Oracle数据库。

连接实例tt1122的语句为:

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"

连接实例ttnew的语句为:

$  . /u01/TimesTen/ttnew/bin/ttenv.sh
$ ttisql -v1 -e "set prompt ‘cachedb2> ‘" "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle"

连接Oracle的语句为:

$ sqlplus tthr/oracle@ttorcl

实例与数据库DSN

两个实例的定义为：

$ cat /etc/TimesTen/instance_info 
#SUM 55602     1
[ tt1122 ]
Product=TimesTen11.2.2.6.2
InstallDir=/home/oracle/TimesTen/tt1122
InstanceAdministrator=oracle
DaemonHome=/home/oracle/TimesTen/tt1122/info
BitLevel=64
Component=Client/Server and DataManager
TT_PORT=53392

[ ttnew ]
Product=TimesTen11.2.2.8.11
InstallDir=/u01/TimesTen/ttnew
InstanceAdministrator=oracle
DaemonHome=/u01/TimesTen/ttnew/info
BitLevel=64
Component=Client/Server and DataManager
TT_PORT=55555

实例tt1122上的数据库cachedb1的DSN，为复制源

more /home/oracle/TimesTen/tt1122/info/sys.odbc.ini
[cachedb1]
Driver=/home/oracle/TimesTen/tt1122/lib/libtten.so
DataStore=/home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb1
PermSize=32
TempSize=64
LogFileSize=32
LogBufMB=32
DatabaseCharacterSet=AL32UTF8
OracleNetServiceName=ttorcl

实例ttnew上的数据库cachedb2的DSN，为复制目标
这里有一点必须强调，即Driver必须写自己路径下的Driver，即/u01/TimesTen/ttnew/lib/libtten.so, 由于最初cachedb2的DSN是从cachedb1拷贝而来，因此最初的Driver写成了Driver=/home/oracle/TimesTen/tt1122/lib/libtten.so, 即使这个.so文件和之前的是一样的，这种写法在后续做client auto failover实验时会产生错误，

more /u01/TimesTen/ttnew/info/sys.odbc.ini
[cachedb2]
Driver=/u01/TimesTen/ttnew/lib/libtten.so
#Driver=/home/oracle/TimesTen/tt1122/lib/libtten.so
DataStore=/home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb2
PermSize=32
TempSize=64
LogFileSize=32
LogBufMB=32
DatabaseCharacterSet=AL32UTF8
OracleNetServiceName=ttorcl

在Oracle数据库中建表

其中t1用于只读缓存组，t2用于AWT缓存组:

$ sqlplus tthr/[email protected]
drop table t1;
drop table t2;
create table t1 (c1 number(22) not null primary key, c2 date, c3 varchar(40));
insert into t1 values (1, sysdate, ‘t1 data inserted in oracle‘);
insert into t1 values (2, sysdate, ‘t1 data inserted in oracle‘);
commit;
create table t2 (c1 number(22) not null primary key, c2 date, c3 varchar(40));
insert into t2 values (1, sysdate, ‘t2 data inserted in oracle‘);
insert into t2 values (2, sysdate, ‘t2 data inserted in oracle‘);
commit;

建立Readonly Autorefresh缓存组

tthr赋予了admin权限是用于克隆active数据库，赋予cache_manager权限是为了做cache admin, 不过我们的例子中使用cacheadm用户来做。

$ ttisql cachedb1

drop user tthr;
create user tthr identified by timesten;
grant admin, create session, cache_manager, create any table to tthr; ???
exit;

建立只读缓存组t1_roa

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"

call ttcacheuidpwdset(‘cacheadm‘,‘oracle‘);
call ttcachestart;
create readonly cache group t1_roa autorefresh interval 10 seconds
from t1 (c1 number(22) not null primary key, c2 date, c3 varchar(40));
load cache group t1_roa commit every 100 rows;

select * from t1;
< 1, 2016-07-01 20:04:51, t1 data inserted in oracle >
< 2, 2016-07-01 20:04:51, t1 data inserted in oracle >
exit;

建立AWT缓存组t2_awt

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"

create asynchronous writethrough cache group t2_awt
from t2 (c1 number(22) not null primary key, c2 date, c3 varchar(40));
load cache group t2_awt commit every 100 rows;
call ttrepstart;

select * from t2;
< 1, 2016-07-01 20:04:51, t2 data inserted in oracle >
< 2, 2016-07-01 20:04:51, t2 data inserted in oracle >

exit;

创建Active Standby Pair复制

注意指定了固定的复制端口，而不是自动协商的端口。因为ttnew daemon并非在缺省的端口启动

alter cache group t1_roa set autorefresh state paused;
call ttrepstop;
create active standby pair
cachedb1 on "timesten-hol",
cachedb2 on "timesten-hol"
return receipt
store cachedb1 on "timesten-hol" port 11102
store cachedb2 on "timesten-hol" port 11202;
call ttrepstart;
call ttrepstateget;
call ttrepstateset(‘ACTIVE‘);
alter cache group t1_roa set autorefresh state on;
call ttrepstateget; 
exit;

克隆Standby数据库, 注意-verbosity 2，给出了很有用的信息

$ ttRepAdmin -verbosity 2 -duplicate -from cachedb1 -host timesten-hol -remotedaemonport 53392 -dsn cachedb2 -uid tthr -pwd timesten -keepcg -cacheuid cacheadm -cachepwd oracle
20:32:38 Contacting remote main daemon at 127.0.0.1 port 53392
20:32:38 Contacting the replication agent for CACHEDB1 ON TIMESTEN-HOL (127.0.0.1) port 11102
20:32:38 Beginning transfer from CACHEDB1 ON TIMESTEN-HOL to CACHEDB2 ON TIMESTEN-HOL
20:33:06 Checkpoint transfer 10 percent complete
20:33:06 Checkpoint transfer 20 percent complete
20:33:06 Checkpoint transfer 30 percent complete
20:33:06 Checkpoint transfer 100 percent complete
20:33:06 Checkpoint transfer phase complete
20:33:09 Log transfer 100 percent complete
20:33:09 Log transfer phase complete
20:33:10 Transfer complete

Subscriber                                                       State
CACHEDB1 ON TIMESTEN-H                                           START
_ORACLE ON TIMESTEN-H                                            START

20:33:16 Duplicate Operation Ends

$ ttisql -v1 -e "set prompt ‘cachedb2> ‘" "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle"
call ttrepstart;
call ttcachestart;
call ttrepstateget;
< STANDBY, NO GRID >
exit;

确认只读缓存组正常工作（在Oracle端插入）

$ sqlplus tthr/[email protected]
insert into t1 values (3, sysdate, ‘t1 data inserted in oracle‘);
commit;
$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"
sleep 70; <- refresh interval定义的是60s
select * from t1;
Command> select * from t1;
< 1, 2016-07-01 20:04:51, t1 data inserted in oracle >
< 2, 2016-07-01 20:04:51, t1 data inserted in oracle >
< 3, 2016-07-02 01:26:34, t1 data inserted in oracle >

确认AWT缓存组正常工作（在TimesTen端插入）

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"
insert into t2 values (3, sysdate, ‘t2 data inserted in timesten‘);
commit;
$ sqlplus tthr/[email protected]
select * from t2;
SQL> select * from t2;

        C1 C2        C3
---------- --------- ----------------------------------------
         1 01-JUL-16 t2 data inserted in oracle
         2 01-JUL-16 t2 data inserted in oracle
         3 02-JUL-16 t2 data inserted in timesten

在active 节点上创建C/S连接

客户端自动切换相关属性为ttc_server2, ttc_server_dns2 和 tcp_port2

TCP_Port指的是timesten server的端口，而非daemon的端口：

The TCP/IP port number where the TimesTen Server is running. Default for TimesTen release 11.2.2 is 53393 for 32-bit platforms and 53397 for 64-bit platforms.

不带自动切换的连接，指定连接到active：

ttIsqlCS -connstr "ttc_server=timesten-hol;tcp_port=53393;ttc_server_dsn=cachedb1;uid=tthr;pwd=timesten;connectionname=cs_without_failover1" -e "set prompt ‘cs_without_failover1> ‘"
cs_without_failover1> call ttrepstateget;
< ACTIVE, NO GRID >

带自动切换的连接, client auto failover只支持C/S连接，因此必须用ttIsqlCS, 并且总是连接到active 节点：

ttIsqlCS -connstr "ttc_server=timesten-hol;tcp_port=53393;ttc_server_dsn=cachedb1;ttc_server2=timesten-hol;tcp_port2=55556;ttc_server_dsn2=cachedb2;uid=tthr;pwd=timesten;connectionname=cs_with_failover1" -e "set prompt ‘cs_with_failover1> ‘"
cs_with_failover1> call ttrepstateget;
< ACTIVE, NO GRID >

在active和standby节点上确定有哪些连接

至此，在active数据库cachedb1上有两个连接，standby数据库cachedb2上没有连接

ACTIVE:必须在tt1122的环境变量下执行

ttisql -connstr "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle" -e "call ttrepstateget;call ttdatastorestatus;exit" | egrep -i "failover1|active|standby"
< ACTIVE, NO GRID >
< /home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb1, 15427, 000000000138DB90, application     , 5900C901, cs_without_failover1          , 1 >
< /home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb1, 15438, 0000000001CB4B90, application     , 5900C901, cs_with_failover1             , 2 >

STANDBY: 必须在ttnew的环境变量下执行

ttisql -connstr "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle" -e "call ttrepstateget;call ttdatastorestatus;exit" | egrep -i "failover1|active|standby"
< STANDBY, NO GRID >

调换active和standby的角色

在ACTIVE节点上：

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"
call ttrepsubscriberwait(,,,,-1); <- 必须返回 < 00 >
call ttrepstop;
alter cache group t1_roa set autorefresh state paused;
call ttrepdeactivate;
call ttrepstateget;

输出为：
< 00 >
< IDLE, NO GRID >

在STANDBY节点上，使STANDBY成为新的ACTIVE：

$ ttisql -v1 -e "set prompt ‘cachedb2> ‘" "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle"
call ttrepstateset(‘ACTIVE‘);
exit;

在老的ACTIVE节点上：

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"
call ttrepstart;
sleep 10;
call ttrepstateget; <- 输出应为< STANDBY, NO GRID >
exit;

至此，cachedb2成为新的active，cachedb1成为standby

在active和standby节点上确定有哪些连接

新STANDBY:必须在tt1122的环境变量下执行

$ ttisql -connstr "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle" -e "call ttrepstateget;call ttdatastorestatus;exit" | egrep -i "failover1|active|standby"
< STANDBY, NO GRID >
< /home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb1, 15427, 000000000138DB90, application     , 5900C901, cs_without_failover1          , 1 >

cachedb1成了standby，不具备failover功能的c/s连接仍保留在其上

新ACTIVE: 必须在ttnew的环境变量下执行

这时有了一个连接，而之前cachedb2上是没有连接的，这个连接就是从之前的cachedb1上通过auto client failover切换过来的

$ ttisql -connstr "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle" -e "call ttrepstateget;call ttdatastorestatus;exit" | egrep -i "failover1|active|standby"
< ACTIVE, NO GRID >
< /home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb2, 15518, 00000000021F8C60, application     , 0A020081, cs_with_failover1             , 9 >

模拟主节点失效

确保cache agent和rep agent不会自动重启，即重启策略为’manual’ 或 ‘norestart

The daemon restart while there are active connections to the database will cause a database invalidation.

由于目前的active数据库为cachedb2，因此需要重启ttnew实例，重启后，ACTIVE状态变为IDLE

$ . /u01/TimesTen/ttnew/bin/ttenv.sh
$ daemonadmin -restart
$ ttisql -v1 -e "set prompt ‘cachedb2> ‘" "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle"
cachedb2> call ttrepstateget;
< IDLE, NO GRID >

提升standby节点为active，并标记之前的active为失效

$ ttisql -v1 -e "set prompt ‘cachedb1> ‘" "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle"
call ttrepstateset(‘ACTIVE‘);
call ttRepStateSave(‘FAILED‘, ‘cachedb2‘, ‘timesten-hol‘);
call ttrepstateget;
< ACTIVE, NO GRID >

恢复老的active作为standby

$ ttdestroy -force cachedb2
$ ttRepAdmin -verbosity 2 -duplicate -from cachedb1 -host timesten-hol -remotedaemonport 53392 -dsn cachedb2 -uid tthr -pwd timesten -keepcg -cacheuid cacheadm -cachepwd oracle

$ ttisql -v1 -e "set prompt ‘cachedb2> ‘" "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle"cachedb2> call ttrepstart;
cachedb2> call ttcachestart;
cachedb2> call ttrepstateget;
< STANDBY, NO GRID >

确认客户端连接连接的数据库

$ ttisql -connstr "dsn=cachedb1;uid=tthr;pwd=timesten;oraclepwd=oracle" -e "call ttrepstateget;call ttdatastorestatus;exit" | egrep -i "failover1|active|standby"
< ACTIVE, NO GRID >
< /home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb1, 15427, 000000000138DB90, application     , 5900C901, cs_without_failover1          , 1 >
< /home/oracle/TimesTen/tt1122/info/DemoDataStore/cachedb1, 16209, 00000000013D4B90, application     , 5900C901, cs_with_failover1             , 9 >


$ ttisql -connstr "dsn=cachedb2;uid=tthr;pwd=timesten;oraclepwd=oracle" -e "call ttrepstateget;call ttdatastorestatus;exit" | egrep -i "failover1|active|standby"
< STANDBY, NO GRID >

可以看到，如果是auto client failover，连接总是指向active节点。

参考

• HOWTO : Add Active Standby Pair and Automatic Client Failover To Database With Oracle Cache Connect (Doc ID 1359840.1)