环境：AIX7.1
DB：ORACLE11.2.0.4
OGG：12.3.0.1.4
问题：因不能重复同步数据到目标库，关闭归档，恢复数据后不能启动OGG
#=========================================================#
Connecting to 153.12.72.110:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.
1 unsuccessful login attempt since last login.
Last unsuccessful login: Fri Dec 6 18:38:24 BEIST 2019 on ssh from 89.12.74.46
Last login: Fri Dec 6 18:38:05 BEIST 2019 on ssh from 89.12.88.71
*******************************************************************************
* *
* *
* Welcome to AIX Version 7.1! *
* *
* *
* Please see the README file in /usr/lpp/bos for information pertinent to *
* this release of the AIX Operating System. *
* *
* *
*******************************************************************************
[YOU HAVE NEW MAIL]
#==============================停止OGG===============================
#======================停止顺序：EX->PP->mgr=========================
nskxt1:/home/oracle$cd /ogg/ogg12c
nskxt1:/ogg/ogg12c$ggsci
Oracle GoldenGate Command Interpreter for Oracle
Version 12.3.0.1.4 OGGCORE_12.3.0.1.0_PLATFORMS_180415.0359_FBO
AIX 6, ppc, 64bit (optimized), Oracle 11g on Apr 19 2018 05:30:29
Operating system character set identified as US-ASCII.
Copyright (C) 1995, 2018, Oracle and/or its affiliates. All rights reserved.
GGSCI (nskxt1) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EX_DZFP 00:00:00 00:00:00
EXTRACT RUNNING EX_PTFP 00:00:01 00:00:03
EXTRACT RUNNING EX_QTFP 00:00:01 00:00:09
EXTRACT RUNNING EX_ZYFP 00:00:00 00:00:00
EXTRACT RUNNING PP_DZFP 00:00:00 00:00:00
EXTRACT RUNNING PP_PTFP 00:00:00 00:00:03
EXTRACT RUNNING PP_QTFP 00:00:00 00:00:07
EXTRACT RUNNING PP_ZYFP 00:00:00 00:00:04
GGSCI (nskxt1) 3> stop ex_dzfp
Sending STOP request to EXTRACT EX_DZFP ...
Request processed.
GGSCI (nskxt1) 5> stop ex_ptfp
Sending STOP request to EXTRACT EX_PTFP ...
Request processed.
GGSCI (nskxt1) 6> stop ex_qtfp
Sending STOP request to EXTRACT EX_QTFP ...
Request processed.
GGSCI (nskxt1) 7> stop ex_zyfp
Sending STOP request to EXTRACT EX_ZYFP ...
Request processed.
GGSCI (nskxt1) 8> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT STOPPED EX_DZFP 00:00:01 00:00:49
EXTRACT STOPPED EX_PTFP 00:00:01 00:00:27
EXTRACT STOPPED EX_QTFP 00:00:01 00:00:16
EXTRACT STOPPED EX_ZYFP 00:00:02 00:00:03
EXTRACT RUNNING PP_DZFP 00:00:00 00:00:02
EXTRACT RUNNING PP_PTFP 00:00:00 00:00:05
EXTRACT RUNNING PP_QTFP 00:00:00 00:00:10
EXTRACT RUNNING PP_ZYFP 00:00:00 00:00:07
GGSCI (nskxt1) 12> stop PP_DZFP
Sending STOP request to EXTRACT PP_DZFP ...
Request processed.
GGSCI (nskxt1) 14> stop PP_PTFP
Sending STOP request to EXTRACT PP_PTFP ...
Request processed.
GGSCI (nskxt1) 15> stop PP_QTFP
Sending STOP request to EXTRACT PP_QTFP ...
Request processed.
GGSCI (nskxt1) 16> stop PP_ZYFP
Sending STOP request to EXTRACT PP_ZYFP ...
Request processed.
GGSCI (nskxt1) 17> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT STOPPED EX_DZFP 00:00:01 00:04:05
EXTRACT STOPPED EX_PTFP 00:00:01 00:03:43
EXTRACT STOPPED EX_QTFP 00:00:01 00:03:32
EXTRACT STOPPED EX_ZYFP 00:00:02 00:03:18
EXTRACT STOPPED PP_DZFP 00:00:00 00:00:52
EXTRACT STOPPED PP_PTFP 00:00:00 00:00:27
EXTRACT STOPPED PP_QTFP 00:00:00 00:00:16
EXTRACT STOPPED PP_ZYFP 00:00:00 00:00:05
GGSCI (nskxt1) 18> stop mgr
Manager process is required by other GGS processes.
Are you sure you want to stop it (y/n)?y
Sending STOP request to MANAGER ...
Request processed.
Manager stopped.
GGSCI (nskxt1) 19> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED
EXTRACT STOPPED EX_DZFP 00:00:01 00:04:39
EXTRACT STOPPED EX_PTFP 00:00:01 00:04:17
EXTRACT STOPPED EX_QTFP 00:00:01 00:04:06
EXTRACT STOPPED EX_ZYFP 00:00:02 00:03:52
EXTRACT STOPPED PP_DZFP 00:00:00 00:01:26
EXTRACT STOPPED PP_PTFP 00:00:00 00:01:01
EXTRACT STOPPED PP_QTFP 00:00:00 00:00:50
EXTRACT STOPPED PP_ZYFP 00:00:00 00:00:39
GGSCI (nskxt1) 20>exit
#============================停止数据库归档============================
# RAC节点都要关闭掉
# 在110中执行和在111中执行，都要执行
sqlplus / as sysdba
SQL>shutdown immediate;
# 在110中执行
SQL>startup mount;
SQL>alter database noarchivelog;
#===========================记录停止归档时间===========================
nskxt2:/home/oracle$date
Fri Dec 6 19:10:01 BEIST 2019
SQL>alter database open;
SQL>alter system set cluster_database=true scope=spfile;
# 查看归档状态
SQL>archive log list;
#============================处理、恢复数据============================
略
#==========================启动数据库归档和数据库======================
#================================启动OGG===============================
#==========================顺序：MGR->PP->EX===========================
GGSCI (nskxt1) 1> start mgr
GGSCI (nskxt1) 2> start PP_DZFP
Sending START request to MANAGER ...
EXTRACT PP_DZFP starting
GGSCI (nskxt1) 3> start PP_PTFP
Sending START request to MANAGER ...
EXTRACT PP_PTFP starting
GGSCI (nskxt1) 4> start PP_QTFP
Sending START request to MANAGER ...
EXTRACT PP_QTFP starting
GGSCI (nskxt1) 5> start PP_ZYFP
Sending START request to MANAGER ...
EXTRACT PP_ZYFP starting
GGSCI (nskxt1) 6> start EX_DZFP
Sending START request to MANAGER ...
EXTRACT EX_DZFP starting
GGSCI (nskxt1) 7> start EX_PTFP
Sending START request to MANAGER ...
EXTRACT EX_PTFP starting
GGSCI (nskxt1) 8> start EX_QTFP
Sending START request to MANAGER ...
EXTRACT EX_QTFP starting
GGSCI (nskxt1) 9> start EX_ZYFP
Sending START request to MANAGER ...
EXTRACT EX_ZYFP starting
GGSCI (nskxt1) 10> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT STOPPED EX_DZFP 00:00:01 00:04:05
EXTRACT STOPPED EX_PTFP 00:00:01 00:03:43
EXTRACT STOPPED EX_QTFP 00:00:01 00:03:32
EXTRACT STOPPED EX_ZYFP 00:00:02 00:03:18
EXTRACT RUNNING PP_DZFP 00:00:00 00:00:52
EXTRACT RUNNING PP_PTFP 00:00:00 00:00:27
EXTRACT RUNNING PP_QTFP 00:00:00 00:00:16
EXTRACT RUNNING PP_ZYFP 00:00:00 00:00:05
#如果直接启动OGG不行，在试指定关闭归档时间来启动
GGSCI (nskxt1) 11>alter extract EX_DZFP,tranlog,begin 2019-12-06 19:10:01
GGSCI (nskxt1) 12>alter extract EX_PTFP,tranlog,begin 2019-12-06 19:10:01
GGSCI (nskxt1) 13>alter extract EX_QTFP,tranlog,begin 2019-12-06 19:10:01
GGSCI (nskxt1) 14>alter extract EX_ZYFP,tranlog,begin 2019-12-06 19:10:01
#发现有部份进程不能启动
GGSCI (nskxt1) 15> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT STOPPED EX_DZFP 00:00:01 00:04:05
EXTRACT STOPPED EX_PTFP 00:00:01 00:03:43
EXTRACT STOPPED EX_QTFP 00:00:01 00:03:32
EXTRACT STOPPED EX_ZYFP 00:00:02 00:03:18
EXTRACT RUNNING PP_DZFP 00:00:00 00:00:52
EXTRACT RUNNING PP_PTFP 00:00:00 00:00:27
EXTRACT RUNNING PP_QTFP 00:00:00 00:00:16
EXTRACT RUNNING PP_ZYFP 00:00:00 00:00:05
#=================执行启动日志查看================
GGSCI (nskxt1) 16> view report EX_PTFP
....
2019-12-06 20:41:55 ERROR OGG-00446 Opening file +Fra/2_2519_1014763310.dbf inDBLOGREADER ..
ORA-17503:ksfdopn:2 Failed to open file +FRA/2_2519_1014763310.dbf
ORA-15173:entry '2_2519_1014763310.dbf' does not exist in directory '/'
....
#日志中有以上错误信息，进入ASM查看2_2519_1014763310.dbf归档日志是否存在。
GGSCI (nskxt1) 17>exit
nskxt1:/ogg/ogg12c$su - grid
grid's Password:
nskxt1:/home/grid$asmcmd
ASMCMD> ls
CRS/
DATA/
FRA/
OGG/
ASMCMD> cd fra
ASMCMD> ls
NSKXT/
ASMCMD> cd nskxt
ASMCMD> ls
ARCHIVELOG/
ONLINELOG/
ASMCMD> cd ARCHIVELOG
ASMCMD> ls
2019_11_30/
2019_12_01/
2019_12_02/
2019_12_03/
2019_12_04/
2019_12_05/
2019_12_06/
ASMCMD> cd 2019_12_06
ASMCMD> ls -l
...
ARCHIVELOG UNPROT COARSE DEC 06 21:00:00 Y thread_2_seq_2516.562.1026325513
ARCHIVELOG UNPROT COARSE DEC 06 21:00:00 Y thread_2_seq_2517.859.1026325771
ARCHIVELOG UNPROT COARSE DEC 06 21:00:00 Y thread_2_seq_2518.299.1026326019
ARCHIVELOG UNPROT COARSE DEC 06 21:00:00 Y thread_2_seq_2521.1565.1026329405
ARCHIVELOG UNPROT COARSE DEC 06 21:00:00 Y thread_2_seq_2522.1627.1026329407
ARCHIVELOG UNPROT COARSE DEC 06 21:00:00 Y thread_2_seq_2523.966.1026334027
ARCHIVELOG UNPROT COARSE DEC 06 22:00:00 Y thread_2_seq_2524.329.1026334807
...
#=======2518后直接到2521了，怀疑2519、2520归档日志进入黑洞，试着使用thread_2_seq_2521.1565.1026329405的时间DEC 06 21:00:00来启动OGG
ASMCMD> exit
nskxt1:/ogg/ogg12c$su - grid
GGSCI (nskxt1) 1>alter extract EX_DZFP,tranlog,begin 2019-12-06 21:00:00
GGSCI (nskxt1) 2>alter extract EX_PTFP,tranlog,begin 2019-12-06 21:00:00
GGSCI (nskxt1) 3>alter extract EX_QTFP,tranlog,begin 2019-12-06 21:00:00
GGSCI (nskxt1) 4>alter extract EX_ZYFP,tranlog,begin 2019-12-06 21:00:00
#=====只启动捕获进程，投递进程直接start PP_DZFP...
#=====持续查看info all，所有OGG进程正常
GGSCI (nskxt1) 10> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EX_DZFP 00:00:01 00:04:05
EXTRACT RUNNING EX_PTFP 00:00:01 00:03:43
EXTRACT RUNNING EX_QTFP 00:00:01 00:03:32
EXTRACT RUNNING EX_ZYFP 00:00:02 00:03:18
EXTRACT RUNNING PP_DZFP 00:00:00 00:00:52
EXTRACT RUNNING PP_PTFP 00:00:00 00:00:27
EXTRACT RUNNING PP_QTFP 00:00:00 00:00:16
EXTRACT RUNNING PP_ZYFP 00:00:00 00:00:05
GGSCI (nskxt1) 18> exit
#=====开启应用应用程序
