Whoops! There was a problem loading more pages. Retrying... Whoops! There was a problem previewing this document. Retryi
A simple step by step procedure on how to configure and achieve data replication between MS SQL Server database and Oracle database using Oracle GoldenGate software
MS SQL Server to Oracle data replication using Oracle GoldenGate – A simple demo
Selvakumar Nagulan - http://orabliss.blogspot.com
Table of Contents 1. 2.
Environment Details ....................................................................................................................................................................... 2 GoldenGate Installation .................................................................................................................................................................. 4
3.
Environment preparation .............................................................................................................................................................. 7
3.1
Source Table and Data: ............................................................................................................................................................................. 7
3.2
Data Server Name (DSN) creation ........................................................................................................................................................... 8
3.3
Backup Type and performing Backup................................................................................................................................................... 12
3.4
Target environment setup ....................................................................................................................................................................... 15
4.
Enable Trandata............................................................................................................................................................................. 16
5.
Param Files Setup .......................................................................................................................................................................... 19
5.1 5.2
Source files: .............................................................................................................................................................................................. 19 Target Files: ............................................................................................................................................................................................. 20
6.
Final Destination – The replication............................................................................................................................................ 21
7.
References ....................................................................................................................................................................................... 36
Color coding: GGSCI Commands OS Commands SQL/TSQL Commands Website link Links to reference pages.
Note: I have provided the simple demo on how to perform a replication from an Oracle to Oracle database in my previous blog post Oracle GoldenGate simple hands-on demo. Most of the Oracle related steps will be redirected to refer the blog for details. This document is to provide information mostly on the prerequisites and how to configure GoldenGate software on the MS SQL Server database and perform replication (minimum mandatory setup to achieve replication where you have multiple options on the parameters available to be used). Hence when using this document it is required to have a working internet connection or a cached page of my blog post for reference.
1. Environment Details Source: MS SQL Server Enterprise 2012 running on Windows 10 enterprise
Target: Oracle 12.1.0.2.0 database running on openSUSE 13.2 (Harlequin) (x86_64) under Oracle VM Virtual box with Windows 10 enterprise host.
GoldenGate software used: Source: Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401 Windows x64 (optimized) Target: Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO Linux, x64, 64bit (optimized) GoldenGate software can be downloaded from this link.
2. GoldenGate Installation Once the software has been downloaded from the link, let’s proceed with the installation of the software on both source and target and prepare the environments for the replication.
Source: Unzip the file to a directory and that would be the installation of Oracle GoldenGate software for SQL Server on Windows.
Once the software is extracted, we can then login to the GoldenGate Software Command Interface (GGSCI) and create all the sub directories required. Note: You should change your present working directory to the software location or the GoldenGate Home location and start ggsci.
C:\Windows\system32>cd E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit C:\Windows\system32>e: E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>ggsci Oracle GoldenGate Command Interpreter for SQL Server Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401 Windows x64 (optimized), Microsoft SQL Server on Dec 11 2015 15:50:22 Operating system character set identified as windows-1252. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (Prinky) 1> create subdirs Creating subdirectories under current directory E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit Parameter files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirprm: created Report files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirrpt: created Checkpoint files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirchk: created Process status files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirpcs: created SQL script files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirsql: created Database definitions files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdef: created Extract data files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdat: created Temporary files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirtmp: created Credential store files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dircrd: created Masterkey wallet files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirwlt: created Dump files E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdmp: created
GGSCI (Prinky) 2> exit E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>
Optional step: To create manager process as a Windows service to run in background even when user logs off the system E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit> INSTALL ADDSERVICE Service 'GGSMGR' created. Install program terminated normally.
E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>
Target: For installation on the Target Linux based system, refer my blog post and follow the steps intended to be performed on the Destination Refer Oracle GoldenGate Installation Once installation is complete, login to the GoldenGate Software Command Interface (GGSCI) and create all the sub directories required. Note: You should change your present working directory to the software location or the GoldenGate Home location and start ggsci. oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> ggsci Oracle GoldenGate Command Interpreter for Oracle Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO Linux, x64, 64bit (optimized), Oracle 12c on Nov 11 2015 03:53:23 Operating system character set identified as UTF-8. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (opensuse.selvapc.com) 1> create subdirs We are now done with the installation and configuration of Oracle GoldenGate software on both source and target systems.
3. Environment preparation Once the GoldenGate installation is successful, we should prepare both the environments to be replication ready. Now let’s prepare the environment.
3.1 Source Table and Data: I will be using GGATE database in my MS SQL Server for the purpose of this demo. You can your own existing database. Under GGATE database create schema gg_user go
create table [gg_user].[emp] ( [id] [smallint] not null, [first_name] varchar(50) not null, [last_name] varchar(50) not null, constraint [emp_pk] primary key clustered ( [id] asc ) with (pad_index = off, statistics_norecompute=off, ignore_dup_key=off, allow_row_locks=on, allow_page_locks=on) on [primary] ) on [primary] go
Insert a few data into the table so that we have our source table set up. Query the table to see the records. use ggate select * from gg_user.emp
Result: id 1 2 3 4
first_name Selva Rajini David Manic
last_name Kumar Kanth Billa Badsha
Now our Test DB with its data is ready.
3.2 Data Server Name (DSN) creation Create DSN for the MSSQL data to be accessed by Oracle GoldenGate software. This step is very important as Oracle GoldenGate software will communicate to the database using DSN Steps: Type “ODBC Data Source” in the “Search the web and Windows” tab in the task bar and click on “ODBC Data Sources (64 bit)”
Click on System DSN and fill data as below. Use the database you are trying to connect.
Click on Next and enter details as below
Click on Next and provide the default database to connect
Click on Next and then Click Finish
Now Test the connectivity whether check if it works. Click on Test Data Source
Now we are done with the setup of DSN successfully.
3.3 Backup Type and performing Backup It is a mandatory step to set up “Full recovery model” as backup strategy for the MS SQL database and a backup has to be taken and available at disk to start with the real time extraction. Note: Initial replication doesn’t need the above to be done but is necessary for real time synchronization. So now let’s turn on the Full recovery model and take a backup as below. Check the current setting and change to Full recovery model if required. In SQL Server Management Studio, right click on database name (GGATE in our case) Properties Options Recovery Model
Once set, click on OK so the setting be saved. Take a backup of database with transaction logs. Transaction logs backup is mandatory to be taken and it will be taken along with the database backup by default in the Full recovery mode. Oracle GoldenGate gets the changed data from transaction logs in most cases and if it not found in the logs then it would get it from the transaction log backup and for this reason the backup should be of native MS SQL Server backup and Oracle GoldenGate doesn’t support third party backup. Use the graphical method to take backup by Right click on database (GGATE in our case) Tasks Backup as shown below.
Edit the fields as desired and click OK to perform the backup.
If you would like to use command line to take the backup, open New Query in the SQL Server Management Studio and execute the below commands. USE [master] GO BACKUP DATABASE [Ggate] TO DISK = N'E:\oracle\Ggate_Full.bak' WITH NAME = N'Ggate-Full Database Backup', STATS = 10, INIT GO
3.4 Target environment setup Once software installation is completed on the target system, please refer to Oracle GoldenGate simple hands-on demo, section 3. Prerequisites and Oracle GoldenGate software set up and perform the On Target part of 3.2 OGG software setup Now once this is done, we have set up our source and target environment for the GoldenGate replication and let’s continue with the rest of the operations.
4. Enable Trandata Enabling trandata on the source database would require login to the database and hence use the created DSN name (Command line 1) to connect to the database using GGSCI. The defgen parameter file will be created in the command line 3. E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>ggsci Oracle GoldenGate Command Interpreter for SQL Server Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401 Windows x64 (optimized), Microsoft SQL Server on Dec 11 2015 15:50:22 Operating system character set identified as windows-1252. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (Prinky) 1> dblogin sourcedb GG 2016-05-07 16:27:30 WARNING OGG-05236 ODBC Warning: The specified DSN 'GG' uses a client driver that may be incompatible with the database server. Microsoft SQL Server 2012 requires SQLNCLI11.DLL or a more recent version. 2016-05-07 16:27:30 INFO OGG-03036 Database character set identified as windows-1252. Locale: en_US. 2016-05-07 16:27:30 INFO OGG-03037 Session character set identified as windows-1252. Successfully logged into database. GGSCI (Prinky) 2> add trandata gg_user.emp Logging of supplemental log data is enabled for table gg_user.emp GGSCI (Prinky) 3> edit params defgen GGSCI (Prinky) 4> exit E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit> The contents of the defgen.prm are as below defsfile E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdef\emp.def
sourcedb GG table gg_user.emp; You can see all the tables for which supplement logging (or published for replication) from MS-SQL databases using the below command. SELECT * FROM sys.tables WHERE is_replicated = 1
Result name emp
object_id is_replicated 245575913 1
create_date modify_date 2016-05-07 15:41:04.973
2016-05-07 16:28:03.593
Now we need to invoke defgen utility to create a definitions file for the table EMP under GG_USER schema. So let’s do it. E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>defgen paramfile E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirprm\defgen.prm *********************************************************************** Oracle GoldenGate Table Definition Generator for SQL Server Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401 Windows x64 (optimized), Microsoft SQL Server on Dec 11 2015 16:16:15 Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
Starting at 2016-05-07 16:42:09 *********************************************************************** Operating System Version: Microsoft Windows 8 , on x64 Version 6.2 (Build 9200) Process id: 3116 *********************************************************************** ** Running with the following parameters ** *********************************************************************** defsfile E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdef\emp.def sourcedb GG
2016-05-07 16:42:09 WARNING OGG-05236 ODBC Warning: The specified DSN 'GG' uses a client driver that may be incompatible with the database server. Microsoft SQL Server 2012 requires SQLNCLI11.DLL or a more recent version. 2016-05-07 16:42:09 INFO OGG-03036 Database character set identified as windows-1252. Locale: en_US. 2016-05-07 16:42:09 INFO OGG-03037 Session character set identified as windows-1252. table gg_user.emp; Retrieving definition for gg_user.emp.
Definitions generated for 1 table in E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdef\emp.def.
E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit> We can see that the definition file has been created in the dirdef directory. You can check the contents of the definition file create if you would like to… This definition file is needed for Oracle to understand and translate the data types from MS SQL to Oracle data types. Transfer the generated definition file E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirdef\emp.def to the target server under dirdef directory of the GoldenGate home. I’m using Filezilla client to transfer the file and you are free to use any of your tools such as Filezilla, Winscp, etc. Once the above said steps has been completed, then the rest of the steps are of same like the Oracle to Oracle replication as described in this post. I’ll go through the steps quickly here with explanations wherever required. Rest of the explanation can be read from the above post link.
5. Param Files Setup 5.1 Source files: Place all these files in the dirprm directory under source GoldenGate home. mgr.prm PORT 11000 DYNAMICPORTLIST 15010-15020 msd1e.prm EXTRACT MSD1E SOURCEDB GG TRANLOGOPTIONS MANAGESECONDARYTRUNCATIONPOINT RMTHOST 192.168.56.131, MGRPORT 12000 RMTTRAIL /u01/app/oracle/product/12.1/oggcore_1/dirdat/za TABLE GG_USER.EMP; When you enable supplemental logging with the ADD TRANDATA command for at least one table in a SQL Server database, a secondary truncation point is created in the transaction log that has to be moved for log space to be released as needed, following subsequent log backups. MANAGESECONDARYTRUNCATIONPOINT - Extract will not be running concurrently (for the same source database) with SQL Server transactional replication and/or CDC that is configured for applications other than Oracle GoldenGate. To read more about this parameter see this link. msd1i.prm SOURCEISTABLE SOURCEDB GG RMTHOST 192.168.56.131, MGRPORT 12000 RMTFILE /u01/app/oracle/product/12.1/oggcore_1/dirdat/zi TABLE gg_user.emp; SOURCEISTABLE should be used on initial load extract alone which dictates the process to extract data directly from the table and use created DSN name for SOURCEDB to connect to the database (GG in our case). Note: In this demo I have not used the local trail and pump processes unlike I have used in the Oracle to Oracle replication as explained in this link. It is always best practice to use local trails for real time capture and send to remote trail using data pump process.
5.2 Target Files: I would like to replicate the data on to EMPLOYEE table under OPENSUSE user. I already have a table with the below definition. ID column has a unique constraint enabled. SQL> desc opensuse.employee Name Null? Type ----------------------------------------- -------- ---------------------------ID NOT NULL NUMBER FIRST_NAME VARCHAR2(50) LAST_NAME VARCHAR2(50) Place all these files in the dirprm directory under target GoldenGate home. mgr.prm PORT 12000 DYNAMICPORTLIST 12010-12020 USERID GGS_DBA, PASSWORD ggs_dba msd1r.prm REPLICAT MSD1R SOURCEDEFS /u01/app/oracle/product/12.1/oggcore_1/dirdef/emp.def USERID ggs_dba, PASSWORD ggs_dba HANDLECOLLISIONS MAP GG_USER.emp, TARGET opensuse.employee; msd1i.prm SPECIALRUN END RUNTIME USERID ggs_dba, PASSWORD ggs_dba EXTFILE /u01/app/oracle/product/12.1/oggcore_1/dirdat/zi SOURCEDEFS /u01/app/oracle/product/12.1/oggcore_1/dirdef/emp.def MAP gg_user.emp, TARGET opensuse.employee; SPECIALRUN should be used for initial load parameter file only and END RUNTIME dictates the process to end when the loading from the trial file has been completed.
6. Final Destination – The replication Now once all the files are placed in their respective directories, do the following operations in the Source and Target as shown below. Target: Start the manager process to allow incoming connections for GoldenGate. oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> ggsci Oracle GoldenGate Command Interpreter for Oracle Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO Linux, x64, 64bit (optimized), Oracle 12c on Nov 11 2015 03:53:23 Operating system character set identified as UTF-8. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (opensuse.selvapc.com) 1> start manager Manager started.
GGSCI (opensuse.selvapc.com) 2> info all Program MANAGER
Status
Group
Lag at Chkpt Time Since Chkpt
RUNNING
GGSCI (opensuse.selvapc.com) 3> exit oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> Source: Start the manager, real time extraction process and initial extract process. E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>ggsci Oracle GoldenGate Command Interpreter for SQL Server Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401
Windows x64 (optimized), Microsoft SQL Server on Dec 11 2015 15:50:22 Operating system character set identified as windows-1252. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (Prinky) 1> start manager Manager started.
GGSCI (Prinky) 2> ADD EXTRACT msd1e, TRANLOG, BEGIN NOW EXTRACT added.
GGSCI (Prinky) 3> ADD RMTTRAIL /u01/app/oracle/product/12.1/oggcore_1/dirdat/za, EXTRACT msd1e RMTTRAIL added. GGSCI (Prinky) 4> start ext msd1e Sending START request to MANAGER ... EXTRACT MSD1E starting
GGSCI (Prinky) 5> info all Program
Status
Group
Lag at Chkpt Time Since Chkpt
MANAGER RUNNING EXTRACT RUNNING MSD1E
00:00:47
GGSCI (Prinky) 6> stats ext msd1e Sending STATS request to EXTRACT MSD1E ... No active extraction maps.
00:00:05
-- At this point, we don't have any active changes happening at the database in the real time even though we have data in the table. -- When there is a change in the database (either of an insert, update, delete), then the process will show the status as below. -- Inserted 4 rows to the gg_user.emp table now -- Data in the source table now is as below use ggate select * from gg_user.emp
Result: id 1 2 3 4 5 6 7 8
first_name Selva Rajini David Manic Manne Rolls Big Masala
last_name Kumar Kanth Billa Badsha Pacquo Royse Ben Poori
GGSCI (Prinky) 7> stats ext msd1e Sending STATS request to EXTRACT MSD1E ... Start of Statistics at 2016-05-08 02:19:16. Output to /u01/app/oracle/product/12.1/oggcore_1/dirdat/za: Extracting from gg_user.emp to gg_user.emp: *** Total statistics since 2016-05-08 02:17:42 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00
*** Daily statistics since 2016-05-08 02:17:42 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 *** Hourly statistics since 2016-05-08 02:17:42 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 *** Latest statistics since 2016-05-08 02:17:42 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 End of Statistics.
GGSCI (Prinky) 8> exit -- Start the initial extract using "extract" executable so that the process ends when the initial data is completely extracted. E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit>extract paramfile dirprm\msd1i.prm reportfile dirrpt\msd1i.rpt 2016-05-08 02:23:23 INFO OGG-01017 Wildcard resolution set to IMMEDIATE because SOURCEISTABLE is used. *********************************************************************** Oracle GoldenGate Capture for SQL Server Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401 Windows x64 (optimized), Microsoft SQL Server on Dec 11 2015 16:22:53 Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
Starting at 2016-05-08 02:23:23 *********************************************************************** Operating System Version: Microsoft Windows 8 , on x64 Version 6.2 (Build 9200) Process id: 3328 Description: *********************************************************************** ** Running with the following parameters ** *********************************************************************** 2016-05-08 02:23:23 INFO OGG-03059 Operating system character set identified as windows-1252. 2016-05-08 02:23:23 INFO OGG-02695 ANSI SQL parameter syntax is used for parameter parsing. 2016-05-08 02:23:23 WARNING OGG-05236 ODBC Warning: The specified DSN 'GG' uses a client driver that may be incompatible with the database server. Microsoft SQL Server 2012 requires SQLNCLI11.DLL or a more recent version. 2016-05-08 02:23:23 INFO OGG-03036 Database character set identified as windows-1252. Locale: en_US. 2016-05-08 02:23:23 INFO OGG-03037 Session character set identified as windows-1252. 2016-05-08 02:23:23 INFO OGG-06509 Using the following key columns for source table gg_user.emp: id. 2016-05-08 02:23:23 INFO OGG-01851 filecaching started: thread ID: 72465440. 2016-05-08 02:23:23 INFO OGG-01815 Virtual Memory Facilities for: COM anon alloc: MapViewOfFile anon free: UnmapViewOfFile file alloc: MapViewOfFile file free: UnmapViewOfFile target directories: E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit\dirtmp.
CACHEMGR virtual memory values (may have been adjusted) CACHESIZE: 4G CACHEPAGEOUTSIZE (default): 4M PROCESS VM AVAIL FROM OS (min): 7.97G CACHESIZEMAX (strict force to disk): 6.88G 2016-05-08 02:23:23 WARNING OGG-01842 CACHESIZE PER DYNAMIC DETERMINATION (4G) LESS THAN RECOMMENDED: 64G (64bit system) vm found: 7.97G Check swap space. Recommended swap/extract: 128G (64bit system). Database Version: Microsoft SQL Server Version 11.00.5058 ODBC Version 03.80.0000 Driver Information: SQLSRV32.DLL Version 10.00.10240 ODBC Version 03.52 2016-05-08 02:23:29 INFO OGG-01226 Socket buffer size set to 65536 (flush size 27985). 2016-05-08 02:23:29 INFO OGG-01478 Output file /u01/app/oracle/product/12.1/oggcore_1/dirdat/zi is using format RELEASE 12.2. 2016-05-08 02:23:29 INFO OGG-02911 Processing table gg_user.emp. *********************************************************************** * ** Run Time Statistics ** * ***********************************************************************
Report at 2016-05-08 02:23:29 (activity since 2016-05-08 02:23:23) Output to /u01/app/oracle/product/12.1/oggcore_1/dirdat/zi: From Table gg_user.emp:
# # # #
inserts: updates: deletes: discards:
8 0 0 0
E:\oracle\oggWinx64\ggs_Windows_x64_MSSQL_64bit> Now you can notice that all the 8 records in the table has been extracted by the initial load extract. Target: Start the initial replicat, real time replicat with HANDLECOLLISIONS and without HANDLECOLLISIONS and check for data sync. Initial replicat can be started as a background process so that it can completed and exited when loading is done. oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> sqlplus opensuse/opensuse SQL*Plus: Release 12.1.0.2.0 Production on Sun May 8 02:59:24 2016 Copyright (c) 1982, 2014, Oracle. All rights reserved. Last Successful login time: Sat Apr 30 2016 08:46:44 +05:30 Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options SQL> select * from employee; no rows selected SQL> ! oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> echo $OGG_HOME /u01/app/oracle/product/12.1/oggcore_1 oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> nohup replicat paramfile $OGG_HOME/dirprm/msd1i.prm reportfile $OGG_HOME/dirrpt/msd1i.rpt & [1] 27150 oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> nohup: ignoring input and appending output to ‘nohup.out’
oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> jobs [1]+ Running nohup replicat paramfile $OGG_HOME/dirprm/msd1i.prm reportfile $OGG_HOME/dirrpt/msd1i.rpt & oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> jobs [1]+ Running nohup replicat paramfile $OGG_HOME/dirprm/msd1i.prm reportfile $OGG_HOME/dirrpt/msd1i.rpt & oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> jobs [1]+ Done nohup replicat paramfile $OGG_HOME/dirprm/msd1i.prm reportfile $OGG_HOME/dirrpt/msd1i.rpt oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> exit exit SQL> set lines 300 SQL> select * from employee; ID FIRST_NAME LAST_NAME ---------- -------------------------------------------------- -------------------------------------------------1 Selva Kumar 2 Rajini Kanth 3 David Billa 4 Manic Badsha 5 Manne Pacquo 6 Rolls Royse 7 Big Ben 8 Masala Poori 8 rows selected. SQL> ! Now at this point I have added one record to the table at source by using the below command. use ggate insert into gg_user.emp values (9, 'Mercedez', 'Benz')
Continuing with the replicat operation for real time sync… oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> ggsci Oracle GoldenGate Command Interpreter for Oracle Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO
Linux, x64, 64bit (optimized), Oracle 12c on Nov 11 2015 03:53:23 Operating system character set identified as UTF-8. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (opensuse.selvapc.com) 1> dblogin userid ggs_dba, password ggs_dba Successfully logged into database. GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 2> ADD REPLICAT msd1r, EXTTRAIL /u01/app/oracle/product/12.1/oggcore_1/dirdat/za, CHECKPOINTTABLE ggs_dba.checkpoint REPLICAT added.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 3> alter replicat msd1r, begin 2016-05-08 02:23:23 -- This is the time we started our initial extract. 2016-05-08 19:43:04 INFO OGG-06594 Replicat MSD1R has been altered through GGSCI. Even the start up position might be updated, duplicate suppression remains active in next startup. To override duplicate suppression, start MSD1R with NOFILTERDUPTRANSACTIONS option. REPLICAT altered.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 4> start rep msd1r Sending START request to MANAGER ... REPLICAT MSD1R starting
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 5> info all Program
Status
Group
Lag at Chkpt Time Since Chkpt
MANAGER RUNNING REPLICAT RUNNING MSD1R
00:00:00
00:00:04
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 6> stats rep MSD1R Sending STATS request to REPLICAT MSD1R ... Start of Statistics at 2016-05-08 19:43:22. Replicating from gg_user.emp to OPENSUSE.EMPLOYEE: *** Total statistics since 2016-05-08 19:43:11 *** Total inserts 1.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 1.00 *** Daily statistics since 2016-05-08 19:43:11 *** Total inserts 1.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 1.00 *** Hourly statistics since 2016-05-08 19:43:11 *** Total inserts 1.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 1.00 *** Latest statistics since 2016-05-08 19:43:11 *** Total inserts 1.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 1.00 End of Statistics.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 7> exit oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> exit exit SQL> select * from employee; ID FIRST_NAME LAST_NAME ---------- -------------------------------------------------- -------------------------------------------------1 Selva Kumar 2 Rajini Kanth 3 David Billa 4 Manic Badsha 5 Manne Pacquo 6 Rolls Royse 7 Big Ben 8 Masala Poori 9 Mercedez Benz 9 rows selected. SQL> exit Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> If you take a look at the above command line 3, you might notice I have altered the replicat to start at the exact time the initial extract is started (can be obtained from initial extract report file). This is because, the initial extract will take all the records from the table directly and trying to applying the real time changes before this point is unnecessary and would be an overhead for GoldenGate process as it has to handle any collisions of data. That’s the reason, the real time replicat has inserted only one record that it captured after the mentioned time. Now let’s see what would have happened if we didn’t alter the begin time of the replicat. oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> ggsci Oracle GoldenGate Command Interpreter for Oracle Version 12.2.0.1.0 OGGCORE_12.2.0.1.0_PLATFORMS_151101.1925.2_FBO
Linux, x64, 64bit (optimized), Oracle 12c on Nov 11 2015 03:53:23 Operating system character set identified as UTF-8. Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
GGSCI (opensuse.selvapc.com) 1> dblogin userid ggs_dba, password ggs_dba Successfully logged into database. GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 2> ADD REPLICAT msd1r, EXTTRAIL /u01/app/oracle/product/12.1/oggcore_1/dirdat/za, CHECKPOINTTABLE ggs_dba.checkpoint REPLICAT added.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 3> start replicat msdr1 ERROR: REPLICAT MSDR1 does not exist.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 4> start replicat msd1r Sending START request to MANAGER ... REPLICAT MSD1R starting
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 5> info all Program
Status
Group
Lag at Chkpt Time Since Chkpt
MANAGER RUNNING REPLICAT RUNNING MSD1R
00:00:00
00:00:03
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 6> stats rep msd1r Sending STATS request to REPLICAT MSD1R ...
Start of Statistics at 2016-05-08 13:31:26. Replicating from gg_user.emp to OPENSUSE.EMPLOYEE: *** Total statistics since 2016-05-08 13:31:17 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 Total insert collisions 4.00 *** Daily statistics since 2016-05-08 13:31:17 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 Total insert collisions 4.00 *** Hourly statistics since 2016-05-08 13:31:17 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 Total insert collisions 4.00 *** Latest statistics since 2016-05-08 13:31:17 *** Total inserts 4.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 4.00 Total insert collisions 4.00 End of Statistics.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 7> info all Program
Status
Group
Lag at Chkpt Time Since Chkpt
MANAGER RUNNING REPLICAT RUNNING MSD1R
00:00:00
00:00:08
-- The insert on the source database happened at this point in time. GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 8> stats rep MSD1R Sending STATS request to REPLICAT MSD1R ... Start of Statistics at 2016-05-08 19:40:01. Replicating from gg_user.emp to OPENSUSE.EMPLOYEE: *** Total statistics since 2016-05-08 13:31:17 *** Total inserts 5.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 5.00 Total insert collisions 4.00 *** Daily statistics since 2016-05-08 13:31:17 *** Total inserts 5.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 5.00 Total insert collisions 4.00 *** Hourly statistics since 2016-05-08 19:00:00 *** Total inserts 1.00
Total updates Total deletes Total discards Total operations
0.00 0.00 0.00 1.00
*** Latest statistics since 2016-05-08 13:31:17 *** Total inserts 5.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 5.00 Total insert collisions 4.00 End of Statistics.
GGSCI (opensuse.selvapc.com as ggs_dba@susedb12) 9> exit oracle@opensuse:/u01/app/oracle/product/12.1/oggcore_1> In the above scenario, you can see that we have 4 collisions that the GoldenGate process has handled and inserted only the record which is missing. Once after all the records has been applied, you can stop the replicat process, edit the parameter to remove the HANDLECOLLISIONS and then start the replicat again to reduce the work overhead. The parameter file would look like below. msd1r.prm REPLICAT MSD1R SOURCEDEFS /u01/app/oracle/product/12.1/oggcore_1/dirdef/emp.def USERID ggs_dba, PASSWORD ggs_dba MAP GG_USER.emp, TARGET opensuse.employee; This completes the replication setup between the source MS SQL Server database and the target Oracle database. Hope you have enjoyed the article.
7. References OTN article - Replicating Transactions Between Microsoft SQL Server and Oracle Database Using Oracle GoldenGate by Nikolay Manchev Oracle documentation - Oracle® GoldenGate Installing and Configuring Oracle GoldenGate for SQL Server Oracle documentation - Oracle® GoldenGate Reference for Oracle GoldenGate for Windows and UNIX My blog - Oracle GoldenGate simple hands-on demo
Thank you!
