Differences

This shows you the differences between two versions of the page.

--- oracle_goldengate_hadoop_hbase [2020/11/11 15:43] – andonovj
+++ oracle_goldengate_hadoop_hbase [2020/11/22 15:08] (current) – [Configure HBase Handler] andonovj
@@ Line 2: / Line 2: @@
 Again, we have to configure:
+  * Start HBase
   * Replicat config
   * HBase Handler Properties File
@@ Line 8: / Line 9: @@
 So let's get going.
+====Start HBase====
+<Code:bash|Start HBase, HDFS and Yarn>
+[oracle@hostname ~]$ start-dfs.sh
+Starting namenodes on [localhost]
+localhost: starting namenode, logging to /opt/hadoop/logs/hadoop-oraclenamenode-
+edvmr1p0.out
+localhost: starting datanode, logging to /opt/hadoop/logs/hadoop-oracledatanode-
+edvmr1p0.out
+Starting secondary namenodes [0.0.0.0]
+.0.0.0: starting secondarynamenode, logging to /opt/hadoop/logs/hadooporacle-
+secondarynamenode-edvmr1p0.out
+[oracle@hostname ~]$ start-yarn.sh
+starting yarn daemons
+starting resourcemanager, logging to /opt/hadoop/logs/yarn-oracleresourcemanager-
+edvmr1p0.out
+localhost: starting nodemanager, logging to /opt/hadoop/logs/yarn-oraclenodemanager-
+edvmr1p0.out
+[oracle@hostname ~]$ start-hbase.sh
+localhost: starting zookeeper, logging to /opt/hbase/logs/hbase-oraclezookeeper-
+edvmr1p0.out
+starting master, logging to /opt/hbase/logs/hbase-oracle-master-edvmr1p0.out
+starting regionserver, logging to /opt/hbase/logs/hbase-oracle-1-regionserveredvmr1p0.
+out
+[oracle@hostname ~]$
+[oracle@hostname ~]$ $JAVA_HOME/bin/jps
+SecondaryNameNode
+DataNode
+NodeManager
+HMaster
+HQuorumPeer
+NameNode
+Jps
+ResourceManager
+HRegionServer
+[oracle@hostname ~]$
+</Code>
+====Configure HBase Handler====
 <Code:bash |Configure HBase Handler Properties File (TRG_OGGHIME/dirprm/base.properties>
 gg.handlerlist=hbase
@@ Line 43: / Line 83: @@
 Copyright (C) 1995, 2016, Oracle and/or its affiliates. All rights reserved.
 GGSCI (edvmr1p0) 1> info all
@@ Line 64: / Line 102: @@
 REPLICAT rhbase
-TARGETDB LIBFILE libggjava.so SET property=
+TARGETDB LIBFILE libggjava.so SET property=dirprm/hbase.properties
-dirprm/hbase.properties
 REPORTCOUNT EVERY 1 MINUTES, RATE
 GROUPTRANSOPS 10000
@@ Line 90: / Line 127: @@
 GGSCI (edvmr1p0) 7>
+</Code>
+=====Test=====
+So let's check if that will work. We will do two tests:
+  - Test on Insert Operation
+  - Test on Update Operation
+====Test on Insert Operation====
+<Code:bash|Insert data on the source>
+[oracle@edvmr1p0 oggsrc]$ sqlplus oggsrc/oracle@orcl
+SQL*Plus: Release 12.1.0.2.0 Production on Wed Nov 11 15:45:59 2020
+Copyright (c) 1982, 2014, Oracle.  All rights reserved.
+Last Successful login time: Wed Nov 11 2020 15:24:15 +00:00
+Connected to:
+Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
+With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
+SQL> insert into product_prod select * from product where product_id < 1001;
+rows created.
+SQL> commit;
+Commit complete.
+SQL>
+</Code>
+Check on HBase:
+<Code:bash|Check on the HBase>
+[oracle@edvmr1p0 ~]$ hbase shell
+SLF4J: Class path contains multiple SLF4J bindings.
+SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
+SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
+SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
+SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
+HBase Shell; enter 'help<RETURN>' for list of supported commands.
+Type "exit<RETURN>" to leave the HBase Shell
+Version 1.2.3, rbd63744624a26dc3350137b564fe746df7a721a4, Mon Aug 29 15:13:42 PDT 2016
+hbase(main):001:0> list
+TABLE
+OGGSRC:PRODUCT_PROD                     <- Our table
+row(s) in 0.2120 seconds
+=> ["OGGSRC:PRODUCT_PROD"]
+hbase(main):002:0>
+base(main):002:0> scan 'OGGSRC:PRODUCT_PROD'
+ROW COLUMN+CELL
+column=cf:PRODUCT_CODE, timestamp=1486380784291,
+value=RJM75LEM3UY
+column=cf:PRODUCT_ID, timestamp=1486380784291,
+value=1
+column=cf:PRODUCT_NAME, timestamp=1486380784291,
+value=eu, eleifend nec, malesuada
+column=cf:SUPPLIER_ID, timestamp=1486380784291,
+value=1
+column=cf:PRODUCT_CODE, timestamp=1486380784291,
+value=IEF94KVH9EU
+column=cf:PRODUCT_ID, timestamp=1486380784291,
+..............................................................
+</Code>
+====Test on Update Operation====
+On the Source database, update the table in any way you want:
+<Code:bash|Update source table>
+SQL> update product_prod set product_name = 'OGG Big Data HBase' where product_id=999;
+row updated.
+SQL> commit;
+Commit complete.
+SQL>
+</Code>
+On the HBase that should be visible:
+<Code:bash|Check HBase>
+hbase(main):010:0* get 'OGGSRC:PRODUCT_PROD','999'
+COLUMN                           CELL
+ cf:PRODUCT_CODE                 timestamp=1605109589905, value=MTA34WRW7FB
+ cf:PRODUCT_ID                   timestamp=1605111265717, value=999                        <- Our ID (where clause)
+ cf:PRODUCT_NAME                 timestamp=1605111265717, value=OGG Big Data HBase         <- Our change (set clause)
+ cf:SUPPLIER_ID                  timestamp=1605109589905, value=999
+row(s) in 0.0100 seconds
+hbase(main):011:0>
 </Code>