mongo_shrd_rpl

Overview

So far we configured and spoke about two possible configurations with Mongo:

  • Replication
  • Sharding

Let's discuss overview of each at a time:

This configuration is known also as Active / Passive

Pros:

  • Provides redundancy in case you loose your site
  • Can be used as backup site and thus not loading the primary
  • Can be used for select only queries

Cons:

  • Requires additional Server on other location
  • Requires further configuration in case of a failover
  • Secondary servers cannot be used for read/write operations

This configuration is known as Active / Active

Pros:

  • Provides load balanced configuration
  • Increases the performance
  • Can be used to store regional data, thus each region accesses it's data way faster

Cons:

  • Requires additional server on the same location
  • In case of even one server shuts down, its collections aren't available.
  • Backups should be taken separately for each shard server.

Replicated Shards

So, what if we can combine both architectures, as so:

As so, we can combine the PROs of both configuration and eliminates almost all CONs.

Configuration isn't done so differently than the previous implementations, we have to just combine both ways as follows:

The setup for the configuration database is like any other Mongo database and it is started as follows:

[root@lparamongo mongo]# mongod -f /etc/mongod.conf -fork --configsvr
about to fork child process, waiting until server is ready for connections.
forked process: 1336
child process started successfully, parent exiting
[root@lparamongo mongo]#

With this, we started a mongo database on port 9005 with db path /app/mongo and as a CONFIGSVR! You can also replicate that database for more redundancy if you wish.

Shard controller is used for the Shard server re-configuration (Addition or Removal of a node for example) or as a proxy (front point) for the clients to connect and be load-balanced among the shards.

[root@lparbmongo ~]# mongos --configdb lparamongo:9005 --port 9005 --chunkSize 1
2018-06-07T15:02:37.778+0200 warning: running with 1 config server should be done only for testing purposes and is not recommended for production
2018-06-07T15:02:37.781+0200 [mongosMain] MongoS version 2.6.12 starting: pid=1283 port=9005 64-bit host=lparbmongo (--help for usage)
2018-06-07T15:02:37.781+0200 [mongosMain] db version v2.6.12
2018-06-07T15:02:37.781+0200 [mongosMain] git version: d73c92b1c85703828b55c2916a5dd4ad46535f6a
2018-06-07T15:02:37.781+0200 [mongosMain] build info: Linux build5.ny.cbi.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2018-06-07T15:02:37.781+0200 [mongosMain] allocator: tcmalloc
2018-06-07T15:02:37.781+0200 [mongosMain] options: { net: { port: 9005 }, sharding: { chunkSize: 1, configDB: "lparamongo:9005" } }
2018-06-07T15:02:37.829+0200 [mongosMain] creating WriteBackListener for: lparamongo:9005 serverID: 000000000000000000000000
018-04-06T20:11:11.070+0200 [Balancer] warning: could not initialize balancer, please check that all shards and config servers are up: socket exception [CONNECT_ERROR] for lparcmongo:9005
2018-04-06T20:11:11.070+0200 [Balancer] will retry to initialize balancer in one minute
^C2018-04-06T20:11:23.910+0200 [signalProcessingThread] got signal 2 (Interrupt), will terminate after current cmd ends
2018-04-06T20:11:58.751+0200 [mongosMain] writing initial config version at v5
2018-04-06T20:11:58.760+0200 [mongosMain] about to log new metadata event: { _id: "lparbmongo-2018-04-06T18:11:58-5ac7b86e1bf26be6f336303e", server: "lparbmongo", clientAddr: "N/A", time: new Date(1523038318760), what: "finished upgrade of config database", ns: "config.version", details: { from: 0, to: 5 } }
2018-04-06T20:11:58.790+0200 [mongosMain] upgrade of config server to v5 successful
2018-04-06T20:11:58.790+0200 [mongosMain] distributed lock 'configUpgrade/lparbmongo:9005:1523038318:1804289383' unlocked.
2018-04-06T20:11:59.023+0200 [mongosMain] scoped connection to lparamongo:9005 not being returned to the pool
2018-04-06T20:11:59.023+0200 [Balancer] about to contact config servers and shards
2018-04-06T20:11:59.024+0200 [mongosMain] waiting for connections on port 9005
2018-04-06T20:11:59.024+0200 [Balancer] config servers and shards contacted successfully
2018-04-06T20:11:59.024+0200 [Balancer] balancer id: lparbmongo:9005 started at Apr  6 20:11:59
2018-04-06T20:11:59.053+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1523038318:1804289383' acquired, 

Each Replica Set should be started, initialized and configured as follows:

Starting Node 1

[root@lparcmongo app]#  mongod --dbpath /app/mongo2 --logpath /app/mongo2/logmong.log --port 9006 --fork  --replSet repSetA
about to fork child process, waiting until server is ready for connections.
forked process: 1553
child process started successfully, parent exiting

Starting Node 2

[root@lparcmongo mongo]#  mongod --dbpath /app/mongo --logpath /app/mongo/logmong.log --port 9005 --fork  --replSet repSetA
about to fork child process, waiting until server is ready for connections.
forked process: 1899
child process started successfully, parent exiting

Once both Nodes has been started, we can initiate the replica set configuration:

[root@lparcmongo mongo]# mongo --port 9005
MongoDB shell version: 2.6.12
connecting to: 127.0.0.1:9005/test
> use admin
switched to db admin
> rs.initiate()
{
        "info2" : "no configuration explicitly specified -- making one",
        "me" : "lparcmongo:9005",
        "info" : "Config now saved locally.  Should come online in about a minute.",
        "ok" : 1
}
> rs.status()
{
        "set" : "repSetA",
        "date" : ISODate("2018-04-06T18:07:13Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "lparcmongo:9005",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 42,
                        "optime" : Timestamp(1523038024, 1),
                        "optimeDate" : ISODate("2018-04-06T18:07:04Z"),
                        "electionTime" : Timestamp(1523038024, 2),
                        "electionDate" : ISODate("2018-04-06T18:07:04Z"),
                        "self" : true
                }
        ],
        "ok" : 1
}

Once we are done we can add the 2nd node to the First Replica set:

repSetA:PRIMARY> rs.add("lparcmongo:9006")
{ "ok" : 1 }
repSetA:PRIMARY> rs.status()
{
        "set" : "repSetA",
        "date" : ISODate("2018-04-06T18:07:45Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "lparcmongo:9005",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 74,
                        "optime" : Timestamp(1523038061, 1),
                        "optimeDate" : ISODate("2018-04-06T18:07:41Z"),
                        "electionTime" : Timestamp(1523038024, 2),
                        "electionDate" : ISODate("2018-04-06T18:07:04Z"),
                        "self" : true
                },
                {
                        "_id" : 1,
                        "name" : "lparcmongo:9006",
                        "health" : 1,
                        "state" : 5,
                        "stateStr" : "STARTUP2",
                        "uptime" : 4,
                        "optime" : Timestamp(0, 0),
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2018-04-06T18:07:43Z"),
                        "lastHeartbeatRecv" : ISODate("2018-04-06T18:07:44Z"),
                        "pingMs" : 0,
                        "lastHeartbeatMessage" : "initial sync need a member to be primary or secondary to do our initial sync"
                }
        ],
        "ok" : 1

Once the 2nd Server has been added we should see the following output from the rs.status() command:

repSetA:PRIMARY> rs.status()
{
        "set" : "repSetA",
        "date" : ISODate("2018-04-06T18:08:00Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "lparcmongo:9005",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",     <- Primary / Active
                        "uptime" : 89,
                        "optime" : Timestamp(1523038061, 1),
                        "optimeDate" : ISODate("2018-04-06T18:07:41Z"),
                        "electionTime" : Timestamp(1523038024, 2),
                        "electionDate" : ISODate("2018-04-06T18:07:04Z"),
                        "self" : true
                },
                {
                        "_id" : 1,
                        "name" : "lparcmongo:9006",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",    <- Secondary / Passive
                        "uptime" : 19,
                        "optime" : Timestamp(1523038061, 1),
                        "optimeDate" : ISODate("2018-04-06T18:07:41Z"),
                        "lastHeartbeat" : ISODate("2018-04-06T18:07:59Z"),
                        "lastHeartbeatRecv" : ISODate("2018-04-06T18:08:00Z"),
                        "pingMs" : 0,
                        "syncingTo" : "lparcmongo:9005"
                }
        ],
        "ok" : 1
}



The replica B is configured the same way as replica A:

[root@lpardmongo mongo]# mongod --dbpath /app/mongo --logpath /app/mongo/logmong.log --port 9005 --fork  --replSet repSetB
about to fork child process, waiting until server is ready for connections.
forked process: 1290
child process started successfully, parent exiting
[root@lpardmongo mongo2]# mongod --dbpath /app/mongo2 --logpath /app/mongo2/logmong.log --port 9006 --fork  --replSet repSetB
about to fork child process, waiting until server is ready for connections.
forked process: 1341
child process started successfully, parent exiting
[root@lpardmongo mongo2]# mongo --port 9005
MongoDB shell version: 2.6.12
connecting to: 127.0.0.1:9005/test
> use admin
switched to db admin
> rs.initiate()
{
        "info2" : "no configuration explicitly specified -- making one",
        "me" : "lpardmongo:9005",
        "info" : "Config now saved locally.  Should come online in about a minute.",
        "ok" : 1
}
> rs.status()
{
        "set" : "repSetB",
        "date" : ISODate("2018-04-06T18:10:37Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "lpardmongo:9005",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 117,
                        "optime" : Timestamp(1523038227, 1),
                        "optimeDate" : ISODate("2018-04-06T18:10:27Z"),
                        "electionTime" : Timestamp(1523038227, 2),
                        "electionDate" : ISODate("2018-04-06T18:10:27Z"),
                        "self" : true
                }
        ],
        "ok" : 1
}
repSetB:PRIMARY> rs.add("lpardmongo:9006
2018-04-06T20:10:47.691+0200 SyntaxError: Unexpected token ILLEGAL
repSetB:PRIMARY> rs.add("lpardmongo:9006")
{ "ok" : 1 }
repSetB:PRIMARY> rs.status()
{
        "set" : "repSetB",
        "date" : ISODate("2018-04-06T18:10:55Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "lpardmongo:9005",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 135,
                        "optime" : Timestamp(1523038253, 1),
                        "optimeDate" : ISODate("2018-04-06T18:10:53Z"),
                        "electionTime" : Timestamp(1523038227, 2),
                        "electionDate" : ISODate("2018-04-06T18:10:27Z"),
                        "self" : true
                },
                {
                        "_id" : 1,
                        "name" : "lpardmongo:9006",
                        "health" : 1,
                        "state" : 5,
                        "stateStr" : "STARTUP2",
                        "uptime" : 2,
                        "optime" : Timestamp(0, 0),
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2018-04-06T18:10:55Z"),
                        "lastHeartbeatRecv" : ISODate("2018-04-06T18:10:55Z"),
                        "pingMs" : 0,
                        "lastHeartbeatMessage" : "initial sync need a member to be primary or secondary to do our initial sync"
                }
        ],
        "ok" : 1
}
repSetB:PRIMARY> rs.status()
{
        "set" : "repSetB",
        "date" : ISODate("2018-04-06T18:10:57Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "lpardmongo:9005",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 137,
                        "optime" : Timestamp(1523038253, 1),
                        "optimeDate" : ISODate("2018-04-06T18:10:53Z"),
                        "electionTime" : Timestamp(1523038227, 2),
                        "electionDate" : ISODate("2018-04-06T18:10:27Z"),
                        "self" : true
                },
                {
                        "_id" : 1,
                        "name" : "lpardmongo:9006",
                        "health" : 1,
                        "state" : 5,
                        "stateStr" : "STARTUP2",
                        "uptime" : 4,
                        "optime" : Timestamp(0, 0),
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2018-04-06T18:10:57Z"),
                        "lastHeartbeatRecv" : ISODate("2018-04-06T18:10:57Z"),
                        "pingMs" : 0,
                        "lastHeartbeatMessage" : "initial sync need a member to be primary or secondary to do our initial sync"
                }
        ],
        "ok" : 1
}
repSetB:PRIMARY>

So in order to add the replicaSets instead of the servers themselves, we have to add them in a special way:

[root@lparbmongo ~]# mongo --port 9005
MongoDB shell version: 2.6.12
connecting to: 127.0.0.1:9005/test
mongos> sh.addShard("repSetA/lparcmongo:9005")   <- Replica Set Name / one of the members : Port number
{ "shardAdded" : "repSetA", "ok" : 1 }
mongos> sh.addShard("repSetB/lpardmongo:9005")   <- Replica Set Name / one of the members : Port number
{ "shardAdded" : "repSetB", "ok" : 1 }
mongos> sh.enableSharding("ShardDB") <- Create Sharded DB
{ "ok" : 1 }
mongos> sh.shardCollection("ShardDB.shardCollection",{shardKey:1}) <- Create a sharded Collection using Sharded Key.
{ "collectionsharded" : "ShardDB.shardCollection", "ok" : 1 }
mongos> show dbs
ShardDB  0.078GB
admin    (empty)
config   0.016GB
mongos> sh.status()
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "version" : 4,
        "minCompatibleVersion" : 4,
        "currentVersion" : 5,
        "clusterId" : ObjectId("5ac7b86e1bf26be6f336303d")
}
  shards:
        {  "_id" : "repSetA",  "host" : "repSetA/lparcmongo:9005,lparcmongo:9006" }
        {  "_id" : "repSetB",  "host" : "repSetB/lpardmongo:9005,lpardmongo:9006" }
  databases:
        {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
        {  "_id" : "test",  "partitioned" : false,  "primary" : "repSetA" }
        {  "_id" : "ShardDB",  "partitioned" : true,  "primary" : "repSetA" }
                ShardDB.shardCollection
                        shard key: { "shardKey" : 1 }
                        chunks:
                                repSetA 1
                        { "shardKey" : { "$minKey" : 1 } } -->> { "shardKey" : { "$maxKey" : 1 } } on : repSetA Timestamp(1, 0)

mongos>

In the end, the shard controller will display something similar:

2018-06-07T15:14:13.627+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fa5c55b88e9e647af06
2018-06-07T15:14:13.874+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:19.877+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fabc55b88e9e647af07
2018-06-07T15:14:19.887+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:25.890+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fb1c55b88e9e647af08
2018-06-07T15:14:25.899+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:31.903+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fb7c55b88e9e647af09
2018-06-07T15:14:31.935+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:37.942+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fbdc55b88e9e647af0a
2018-06-07T15:14:37.980+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:43.284+0200 [LockPinger] cluster lparamongo:9005 pinged successfully at Thu Jun  7 15:14:43 2018 by distributed lock pinger 'lparamongo:9005/lparbmongo:9005:1528377223:1804289383', sleeping for 30000ms
2018-06-07T15:14:43.983+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fc3c55b88e9e647af0b
2018-06-07T15:14:43.991+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:49.994+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fc9c55b88e9e647af0c
2018-06-07T15:14:50.269+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:14:56.271+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fd0c55b88e9e647af0d
2018-06-07T15:14:56.279+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:15:02.282+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fd6c55b88e9e647af0e
2018-06-07T15:15:02.290+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' unlocked.
2018-06-07T15:15:08.293+0200 [Balancer] distributed lock 'balancer/lparbmongo:9005:1528377223:1804289383' acquired, ts : 5b192fdcc55b88e9e647af0f

  • mongo_shrd_rpl.txt
  • Last modified: 2019/10/18 20:04
  • by 127.0.0.1