oracle_goldengate_apache_cassandra

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		 Previous revision
oracle_goldengate_apache_cassandra [2020/11/12 16:41] – created andonovjoracle_goldengate_apache_cassandra [2020/11/12 16:54] (current) – [Start Cassandra] andonovj
Line 1: Line 1:
 =====Configuration Destination===== =====Configuration Destination=====
 +To configure cassandra configuration we have to follow the steps as the previous topics.
  
 +
 +====Start Cassandra====
 +Before starting, we have to install and configure Cassandra:
 +
 +<Code:bash|Start Cassandra>
 +[oracle@hostname ~]$ cd /stage
 +[oracle@hostname stage]$ mkdir cassandra
 +[oracle@hostname stage]$ cp cassandra-java-driver-3.1.3.tar.gz cassandra
 +[oracle@hostname stage]$ cd cassandra
 +[oracle@hostname cassandra]$ tar xzvf cassandra-java-driver-3.1.3.tar.gz
 +cassandra-java-driver-3.1.3/apidocs/
 +cassandra-java-driver-3.1.3/apidocs/com/
 +cassandra-java-driver-3.1.3/apidocs/com/datastax/
 +cassandra-java-driver-3.1.3/apidocs/com/datastax/driver/
 +… Many lines omitted for brevity…
 +cassandra-java-driver-3.1.3/src/cassandra-driver-mapping-3.1.3-src.zip
 +cassandra-java-driver-3.1.3/src/cassandra-driver-extras-3.1.3-src.zip
 +[oracle@hostname cassandra]$ ls -1 cassandra-java-driver-3.1.3apidocs
 +cassandra-driver-core-3.1.3.jar
 +cassandra-driver-extras-3.1.3.jar
 +cassandra-driver-mapping-3.1.3.jar
 +changelog
 +lib
 +src
 +upgrade_guide
 +[oracle@ hostname ~]$ cassandra
 +CompilerOracle: dontinline
 +org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset(Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
 +… Many lines omitted for brevity…
 +1335055093669716, 9131915236894670840]
 +INFO 08:25:30 Node localhost/127.0.0.1 state jump to NORMAL
 +-------------------------------------------------------------------
 +[oracle@ hostname ~]$ cqlsh -u cassandra -p cassandra
 +Connected to Test Cluster at 127.0.0.1:9042.
 +[cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
 +Use HELP for help.
 +cassandra@cqlsh> CREATE KEYSPACE oggsrc WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor' : 3 };
 +cassandra@cqlsh> describe keyspaces
 +oggsrc system_schema system_auth system system_distributed system_traces
 +</Code>
 +
 +====Configure Cassandra Properties File====
 +
 +<Code:bash|(e.g. TRG_OGGHOME/dirprm/cass.properties)>
 +gg.handlerlist=cassandra
 +gg.handler.cassandra.type=cassandra
 +gg.handler.cassandra.mode=op
 +gg.handler.cassandra.contactPoints=localhost
 +gg.handler.cassandra.username=cassandra
 +gg.handler.cassandra.password=cassandra
 +gg.handler.cassandra.ddlHandling=CREATE,ADD,DROP
 +gg.handler.cassandra.compressedUpdates=true
 +gg.handler.cassandra.cassandraMode=async
 +gg.handler.cassandra.consistencyLevel=ONE
 +
 +goldengate.userexit.timestamp=utc
 +goldengate.userexit.writers=javawriter
 +javawriter.stats.display=TRUE
 +javawriter.stats.full=TRUE
 +
 +gg.log=log4j
 +gg.log.level=INFO
 +gg.report.time=30sec
 +
 +gg.classpath=/stage/cassandra/cassandra-java-driver-
 +3.1.3/*:/opt/cassandra/lib/*
 +jvm.bootoptions=-Xmx512m -Xms32m -
 +Djava.class.path=.:ggjava/ggjava.jar:./dirprm
 +
 +
 +</Code>
 +
 +
 +====Configure Golden Gate====
 +Now, we have to configure the Golden Gate (Replicat)
 +
 +<Code:bash|Configure Golden Gate>
 +[oracle@edvmr1p0 dirprm]$ trg
 +[oracle@edvmr1p0 oggtrg]$ ggsci
 +
 +Oracle GoldenGate Command Interpreter
 +Version 12.2.0.1.160823 OGGCORE_OGGADP.12.2.0.1.0_PLATFORMS_161019.1437
 +Linux, x64, 64bit (optimized), Generic on Oct 19 2016 16:01:40
 +Operating system character set identified as UTF-8.
 +
 +Copyright (C) 1995, 2016, Oracle and/or its affiliates. All rights reserved.
 +
 +
 +
 +GGSCI (edvmr1p0) 1> info all
 +
 +Program     Status      Group       Lag at Chkpt  Time Since Chkpt
 +
 +MANAGER     RUNNING                                           
 +
 +
 +GGSCI (edvmr1p0) 2> edit param rcass
 +
 +REPLICAT rcass
 +TARGETDB LIBFILE libggjava.so SET
 +property=dirprm/cass.properties
 +REPORTCOUNT EVERY 1 MINUTES, RATE
 +GROUPTRANSOPS 1000
 +MAP OGGSRC.*, TARGET OGGSRC.*;
 +:wq
 +
 +GGSCI (edvmr1p0) 3> add replicat rcass, exttrail ./dirdat/cs
 +REPLICAT added.
 +
 +
 +GGSCI (edvmr1p0) 4> start rcass
 +
 +Sending START request to MANAGER ...
 +REPLICAT RCASS starting
 +
 +
 +GGSCI (edvmr1p0) 5> info all
 +
 +Program     Status      Group       Lag at Chkpt  Time Since Chkpt
 +
 +MANAGER     RUNNING                                           
 +REPLICAT    STARTING    RCASS       00:00:00      00:19:02    
 +
 +
 +GGSCI (edvmr1p0) 6> info all
 +
 +Program     Status      Group       Lag at Chkpt  Time Since Chkpt
 +
 +MANAGER     RUNNING                                           
 +REPLICAT    RUNNING     RCASS       00:00:00      00:00:01    
 +
 +
 +GGSCI (edvmr1p0) 7> 
 +
 +</Code>
 +
 +=====Verify=====
 +To verify the configuration, we will insert data in Oracle and see how this is replicated:
 +
 +
 +<Code:bash|Insert Data in Oracle Source DB>
 +[oracle@edvmr1p0 ~]$ sqlplus oggsrc/oracle@orcl
 +
 +SQL*Plus: Release 12.1.0.2.0 Production on Thu Nov 12 16:47:33 2020
 +
 +Copyright (c) 1982, 2014, Oracle.  All rights reserved.
 +
 +Last Successful login time: Thu Nov 12 2020 16:13:25 +00:00
 +
 +Connected to:
 +Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
 +With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
 +
 +SQL> insert into product_prod select * from product where product_id > 1000 and product_id < 6001;
 +
 +5000 rows created.
 +
 +SQL> commit;
 +
 +Commit complete.
 +
 +SQL> 
 +</Code>
 +
 +Now, we can check if that database has been send over the Golden Gate
 +
 +
 +<Code:bash|Verify the Golden Gate Replication>
 +--Source
 +[oracle@edvmr1p0 oggtrg]$ src
 +[oracle@edvmr1p0 oggsrc]$ ggsci
 +
 +Oracle GoldenGate Command Interpreter for Oracle
 +Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401_FBO
 +Linux, x64, 64bit (optimized), Oracle 12c on Dec 12 2015 02:56:48
 +Operating system character set identified as UTF-8.
 +
 +Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.
 +
 +
 +
 +GGSCI (edvmr1p0) 1> info all
 +
 +Program     Status      Group       Lag at Chkpt  Time Since Chkpt
 +
 +MANAGER     RUNNING                                           
 +EXTRACT     RUNNING     DPUMP       00:00:00      00:00:06    
 +EXTRACT     RUNNING     PRIEX       00:00:10      00:00:06    
 +
 +
 +GGSCI (edvmr1p0) 2> send priex, stats
 +
 +Sending STATS request to EXTRACT PRIEX ...
 +
 +Start of Statistics at 2020-11-12 16:48:10.
 +
 +DDL replication statistics (for all trails):
 +
 +*** Total statistics since extract started     ***
 + Operations                           6.00
 +
 +Output to ./dirdat/in:
 +
 +Extracting from OGGSRC.PRODUCT_PROD to OGGSRC.PRODUCT_PROD:
 +
 +*** Total statistics since 2020-11-12 16:47:45 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +*** Daily statistics since 2020-11-12 16:47:45 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +*** Hourly statistics since 2020-11-12 16:47:45 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +*** Latest statistics since 2020-11-12 16:47:45 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +End of Statistics.
 +
 +
 +GGSCI (edvmr1p0) 3> 
 +
 +--Target
 +GGSCI (edvmr1p0) 6> info all
 +
 +Program     Status      Group       Lag at Chkpt  Time Since Chkpt
 +
 +MANAGER     RUNNING                                           
 +REPLICAT    RUNNING     RCASS       00:00:00      00:00:01    
 +
 +
 +GGSCI (edvmr1p0) 7> send rcass, stats
 +
 +Sending STATS request to REPLICAT RCASS ...
 +
 +Start of Statistics at 2020-11-12 16:48:17.
 +
 +Replicating from OGGSRC.PRODUCT_PROD to OGGSRC.PRODUCT_PROD:
 +
 +*** Total statistics since 2020-11-12 16:47:47 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +*** Daily statistics since 2020-11-12 16:47:47 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +*** Hourly statistics since 2020-11-12 16:47:47 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +*** Latest statistics since 2020-11-12 16:47:47 ***
 + Total inserts                            5000.00
 + Total updates                               0.00
 + Total deletes                               0.00
 + Total discards                             0.00
 + Total operations                        5000.00
 +
 +End of Statistics.
 +
 +
 +GGSCI (edvmr1p0) 8> 
 +</Code>
 +
 +Finally, we can verify the application on Cassandra:
 +
 +<Code:bash|Verify replication on Cassandra>
 +[oracle@edvmr1p0 ~]$ cqlsh -u cassandra -p cassandra
 +Connected to Test Cluster at 127.0.0.1:9042.
 +[cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
 +Use HELP for help.
 +cassandra@cqlsh> CREATE KEYSPACE oggsrc WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor' : 3 };
 +cassandra@cqlsh> describe keyspaces
 +
 +oggsrc  system_schema  system_auth  system  system_distributed  system_traces
 +
 +cassandra@cqlsh> use oggsrc
 +   ... ;     
 +cassandra@cqlsh:oggsrc> describe tables;
 +
 +product_prod
 +
 +cassandra@cqlsh:oggsrc> select count(*) from product_prod;
 +
 + count
 +-------
 +  5000
 +
 +(1 rows)
 +
 +Warnings :
 +Aggregation query used without partition key
 +
 +cassandra@cqlsh:oggsrc> 
 +</Code>
  • oracle_goldengate_apache_cassandra.1605199295.txt.gz
  • Last modified: 2020/11/12 16:41
  • by andonovj