oracle_goldengate_apache_kafka

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
oracle_goldengate_apache_kafka [2020/11/12 12:44] – andonovjoracle_goldengate_apache_kafka [2020/11/12 12:51] (current) – andonovj
Line 140: Line 140:
  
 </Code> </Code>
 +
 +=====Test=====
 +To test the replication, we will insert some rows and see how they are being replicated.
 +
 +<Code:bash|Insert data on Oracle Source DB>
 +[oracle@edvmr1p0 oggsrc]$ sqlplus oggsrc/oracle@orcl
 +
 +SQL*Plus: Release 12.1.0.2.0 Production on Thu Nov 12 12:46:13 2020
 +
 +Copyright (c) 1982, 2014, Oracle.  All rights reserved.
 +
 +Last Successful login time: Thu Nov 12 2020 12:26:00 +00:00
 +
 +Connected to:
 +Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
 +With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
 +
 +SQL> insert into customer_prod select * from customer where customer_id < 101;
 +
 +100 rows created.
 +
 +SQL> commit;
 +
 +Commit complete.
 +
 +SQL> 
 +</Code>
 +
 +Check if the data is replicated between: GG Extract -> GG Replicat
 +
 +<Code:bash|Check GG Replication>
 +--Source:
 +GGSCI (edvmr1p0) 4> send priex,stats
 +
 +Sending STATS request to EXTRACT PRIEX ...
 +
 +Start of Statistics at 2020-11-12 12:46:37.
 +
 +DDL replication statistics (for all trails):
 +
 +*** Total statistics since extract started     ***
 + Operations                           6.00
 +
 +Output to ./dirdat/in:
 +
 +Extracting from OGGSRC.CUSTOMER_PROD to OGGSRC.CUSTOMER_PROD:
 +
 +*** Total statistics since 2020-11-12 12:46:30 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +*** Daily statistics since 2020-11-12 12:46:30 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +*** Hourly statistics since 2020-11-12 12:46:30 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +*** Latest statistics since 2020-11-12 12:46:30 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +End of Statistics.
 +
 +
 +GGSCI (edvmr1p0) 5> 
 +
 +--Target
 +GGSCI (edvmr1p0) 8> send rkafka,stats
 +
 +Sending STATS request to REPLICAT RKAFKA ...
 +
 +Start of Statistics at 2020-11-12 12:46:47.
 +
 +Replicating from OGGSRC.CUSTOMER_PROD to OGGTRG.CUSTOMER_PROD:
 +
 +*** Total statistics since 2020-11-12 12:46:34 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +*** Daily statistics since 2020-11-12 12:46:34 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +*** Hourly statistics since 2020-11-12 12:46:34 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +*** Latest statistics since 2020-11-12 12:46:34 ***
 + Total inserts                             100.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                         100.00
 +
 +End of Statistics.
 +
 +
 +GGSCI (edvmr1p0) 9> 
 +</Code>
 +
 +Finally, we can consume the data on Kafka:
 +
 +<Code:bash|Consume the data from Kafka>
 +[oracle@edvmr1p0 ~]$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic oggbgtopic --from-beginning --zookeeper localhost:2181
 +{"table":"OGGTRG.CUSTOMER_PROD","op_type":"I","op_ts":"2020-11-12 12:46:28.000148","current_ts":"2020-11-12T12:46:34.075000","pos":"00000000000000002372","after":{"CUSTOMER_ID":1,"FIRST_NAME":"Jaden","LAST_NAME":"Poole","EMAIL_ADDRESS":"[email protected]","SSN":"16060407-7404","ADDRESS":"3640 Ac St.","CITY":"San Juan de Dios","ZIP_CODE":"31014","CUSTOMER_TYPE":1}}{"table":"OGGTRG.CUSTOMER_PROD","op_type":"I","op_ts":"2020-11-12 12:46:28.000148","current_ts":"2020-11-12T12:46:34.076000","pos":"00000000000000002658","after":{"CUSTOMER_ID":2,"FIRST_NAME":"Jack","LAST_NAME":"Cox","EMAIL_ADDRESS":"[email protected]","SSN":"16040918-3654","ADDRESS":"Ap #305-8153 Libero Ave","CITY":"Ca��as","ZIP_CODE":"40813","CUSTOMER_TYPE":4}}{"table":"OGGTRG.CUSTOMER_PROD","op_type":"I","op_ts":"2020-11-12 12:46:28.000148","current_ts":"2020-11-12T12:46:34.077000","pos":"00000000000000002912","after":{"CUSTOMER_ID":3,"FIRST_NAME":"Anastasia","LAST_NAME":"Stewart","EMAIL_ADDRESS":"[email protected]","SSN":"16030603-2822","ADDRESS":"282-2584 Consectetuer Street","CITY":"San Juan (San Juan de Tib��s)","ZIP_CODE":"50409","CUSTOMER_TYPE":7}}{"table":"OGGTRG.CUSTOMER_PROD","op_type":"I","op_ts":"2020-11-12 12:46:28.000148","current_ts":"2020-11-12T12:46:34.077001","pos":"00000000000000003198","after":{"CUSTOMER_ID":4,"FIRST_NAME":"Rowan","LAST_NAME":"Fleming","EMAIL_ADDRESS":"[email protected]","SSN":"16700223-3166","ADDRESS":"7594 Adipiscing. Ave","CITY":"San Vicente","ZIP_CODE":"40102","CUSTOMER_TYPE":2}}{"table":"OGGTRG.CUSTOMER_PROD","op_type":"I","op_ts":"2020-11-12 12:46:28.000148","current_ts":"2020-11-12T12:46:34.077002","pos":"00000000000000003448","after":{"CUSTOMER_ID":5,"FIRST_NAME":"Adena","LAST_NAME":"Williamson","EMAIL_ADDRESS":"[email protected]","SSN":
 +</Code>
 +
 +So, we have configured replication:
 +Oracle RDBMS -> Golden Gate (Extract) -> Golden Gate (Replicat) -> Apache Kafka
  
 =====Appendix===== =====Appendix=====
  • oracle_goldengate_apache_kafka.1605185085.txt.gz
  • Last modified: 2020/11/12 12:44
  • by andonovj