Overview
Backup and restore of MongoDB can be achieved in one of the following ways:
- Physical backup of the disk(via. snapshot, rsync or others)
- Using Mongo utilities(mongodump,mongorestore,mongoexport,mongoimport)
Let's start with Mongo utilities:
Mongo Utilities
The mongo utilities, like most of the databases come in pairs:
- Mongodump - Mongorestore (Viable for recovery strategy)
- Mongoexport - Mongoimport (Not Viable for recovery Strategy)
Therefore, backup done with mongodump, has to be restored via mongorestore. Mongodump is able to do backup of:
- Entire Server
- Particular Database
- Particular Collection
Backup
Let's backup a single database as follows:
root# mongodump \
--port 9005 \ <- Port
--db hunter_dev <- Database Name
--out /home/hunter/backups/`date +"%m-%d-%y"` <- Location
The following command will create a directory, which will contain all the collections inside this database.
[root@tain-mt-fhuat1 backups]# mongodump --port 9005 --db monitor --out /home/hunter/backups/`date +"%m-%d-%y"` connected to: 127.0.0.1:9005 2018-05-14T13:19:15.527+0200 DATABASE: monitor to /home/hunter/backups/05-14-18/monitor 2018-05-14T13:19:15.545+0200 monitor.system.indexes to /home/hunter/backups/05-14-18/monitor/system.indexes.bson 2018-05-14T13:19:15.546+0200 3 documents 2018-05-14T13:19:15.546+0200 monitor.monitors to /home/hunter/backups/05-14-18/monitor/monitors.bson 2018-05-14T13:19:15.559+0200 0 documents 2018-05-14T13:19:15.560+0200 Metadata for monitor.monitors to /home/hunter/backups/05-14-18/monitor/monitors.metadata.json 2018-05-14T13:19:15.560+0200 monitor.config to /home/hunter/backups/05-14-18/monitor/config.bson 2018-05-14T13:19:15.560+0200 1 documents 2018-05-14T13:19:15.560+0200 Metadata for monitor.config to /home/hunter/backups/05-14-18/monitor/config.metadata.json [root@tain-mt-fhuat1 backups]# [root@tain-mt-fhuat1 backups]# ls -lart total 4 drwx------. 4 hunter hunter 4096 Mar 20 17:45 .. drwxr-xr-x. 3 root root 22 May 14 13:19 . drwxr-xr-x. 4 root root 39 May 14 13:19 05-14-18
Backup records older than a certain date
mongodump -d database_name -c collection_name -o dump_3mth -q "{\"d\":{\"\$gt\":{\"\$date\":`date -d 2012-12-18 +%s`000}}}" -vvvvvvv
You can also automate it in a script as follows
Backup script
#!/bin/bash
##########################################
# Edit these to define source and destinations
MONGO_DBS=""
BACKUP_TMP=~/tmp
BACKUP_DEST=/apphome/backup/mongo
MONGODUMP_BIN=/usr/bin/mongodump
TAR_BIN=/usr/bin/tar
##########################################
BACKUPFILE_DATE=`date +%Y%m%d-%H%M`
# _do_store_archive <Database> <Dump_dir> <Dest_Dir> <Dest_file>
function _do_store_archive {
mkdir -p $3
cd $2
tar -cvzf $3/$4 dump
}
# _do_backup <Database name>
function _do_backup {
UNIQ_DIR="$BACKUP_TMP/$1"`date "+%s"`
mkdir -p $UNIQ_DIR/dump
echo "dumping Mongo Database $1"
if [ "all" = "$1" ]; then
$MONGODUMP_BIN -u 'adminDBA' -p 'password' --port 9005 -o $UNIQ_DIR/dump
else
$MONGODUMP_BIN -u 'adminDBA' -p 'password' --port 9005 -d $1 -o $UNIQ_DIR/dump
fi
KEY="database-$BACKUPFILE_DATE.tgz"
echo "Archiving Mongo database to $BACKUP_DEST/$1/$KEY"
DEST_DIR=$BACKUP_DEST/$1
_do_store_archive $1 $UNIQ_DIR $DEST_DIR $KEY
rm -rf $UNIQ_DIR
}
# check to see if individual databases have been specified, otherwise backup the whole server
# to "all"
if [ "" = "$MONGO_DBS" ]; then
MONGO_DB="all"
_do_backup $MONGO_DB
else
for MONGO_DB in $MONGO_DBS; do
_do_backup $MONGO_DB
done
fi
You can also use more versatily syntax as follows:
###Backup mongo###
#!/bin/bash
HOST=${HOST:-"$(hostname -s)"}
BACKUP_HOST=${BACKUP_HOST:-"mongo@tbp-mts-backup01"}
BACKUP_DIR=${BACKUP_DIR:-"/backup/db/mongodb"}
MONGO=$(which mongo)
MONGODUMP=$(which mongodump)
TAR=$(which tar)
SSH=$(which ssh)
MONGO_USER=${MONGO_USER:-"adminDBA"}
MONGO_PASSWORD=${MONGO_PASSWORD:-""}
MONGO_HOST=${MONGO_HOST:-"localhost"}
MONGO_PORT=${MONGO_PORT:-"27017"}
DATE="$(date +%F_%H-%M)"
STAGING_DIR=${STAGING_DIR:-"$(pwd)/dump"}
DUMP_DIR="${STAGING_DIR}/${DATE}"
function backup(){
${MONGODUMP} ${USER_OPTS} --host ${MONGO_HOST} --port ${MONGO_PORT} -o ${DUMP_DIR}
}
function pack_db() {
local of
of="$@_backup_${DATE}.tar.bzip2"
${TAR} -cj -C ${DUMP_DIR} -f ${STAGING_DIR}/${of} $@
}
function pack() {
local databases
databases=$(cd ${DUMP_DIR} && ls -1)
for db in $databases ; do
pack_db $db
done
}
function upload(){
files=$(ls -1 ${STAGING_DIR}/*.bzip2)
rsync -a $files $BACKUP_HOST:$BACKUP_DIR/$HOST/
}
USER_OPTS="-u ${MONGO_USER} -p ${MONGO_PASSWORD}"
backup
pack
upload
crontab
[root@server scripts]# crontab -l 0 0 * * 1-5 /bin/sh /app/mongo/hunterServer/scripts/backupMongo.sh >> /apphome/backup/mongo/backup.log [root@server scripts]#
Restore
To restore such backup, we have to use the mongorestore command, but before that, be sure to tar and compress the folder:
- Compress the backup folder
- Move the tar archive wherever you need
[root@servername]# tar -zcvf hunter_dev.tar.gz hunter_dev/ hunter_dev/ hunter_dev/system.indexes.bson hunter_dev/preTransCredit.bson hunter_dev/preTransCredit.metadata.json hunter_dev/transCredit.bson hunter_dev/transCredit.metadata.json hunter_dev/billGame.bson hunter_dev/billGame.metadata.json hunter_dev/billHit.bson hunter_dev/billHit.metadata.json hunter_dev/billJpContri.bson hunter_dev/billJpContri.metadata.jso
This command will produce a tar.gz file which you can migrate more easily. The restore is done via the mongorestore utility as follows:
Particular DB
[root@servername]# mongorestore
\ --port 9005
\ --db sales
\ --drop /app/oracle/mongobackups/03-20-18/sales/
connected to: 127.0.0.1:9005
2018-03-20T17:43:50.660+0100 /app/oracle/mongobackups/03-20-18/sales/accounts.bson
2018-03-20T17:43:50.660+0100 going into namespace [sales.accounts]
2018-03-20T17:43:50.660+0100 dropping
1 objects found
2018-03-20T17:43:50.682+0100
Creating index: { key: { _id: 1 }, name: "_id_", ns: "sales.accounts" }
[root@lpara sales]
You can specify different authentication database as well:
mongorestore \ -u 'adminDBA' \ -p 'password' \ --authenticationDatabase admin \ --port 9005 \ --db hunter_dev \ --drop /app/mongo/backups/05-22-19/hunter_dev
Relocate DB
You can also redirect a database restore, like: restore the data in Database A → B, as folows:
[root@tbp-mts-uatstdkr01 bulk_copied]# docker exec -i b2dcec2eba9d sh -c 'mongorestore --username root --password="password" --authenticationDatabase admin -d=B --gzip --drop /data/db/dump/A'; 2020-04-06T12:41:46.599+0000 the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead 2020-04-06T12:41:46.603+0000 building a list of collections to restore from /data/db/dump/incident dir 2020-04-06T12:41:46.641+0000 reading metadata for studio.audit.oldtrails from /data/db/dump/incident/audit.oldtrails.metadata.json.gz 2020-04-06T12:41:46.661+0000 reading metadata for studio.rounds from /data/db/dump/incident/rounds.metadata.json.gz 2020-04-06T12:41:46.682+0000 reading metadata for studio.audit.trails from /data/db/dump/incident/audit.trails.metadata.json.gz 2020-04-06T12:41:46.684+0000 reading metadata for studio.rounds.requests from /data/db/dump/incident/rounds.requests.metadata.json.gz 2020-04-06T12:41:46.703+0000 restoring studio.audit.oldtrails from /data/db/dump/incident/audit.oldtrails.bson.gz 2020-04-06T12:41:46.707+0000 restoring studio.rounds from /data/db/dump/incident/rounds.bson.gz 2020-04-06T12:41:46.924+0000 restoring studio.rounds.requests from /data/db/dump/incident/rounds.requests.bson.gz 2020-04-06T12:41:46.925+0000 restoring studio.audit.trails from /data/db/dump/incident/audit.trails.bson.gz
Particular collation
You can also restore a particular collation(table) IF YOU POINT TO A FOLDER WITHOUT A NESTED FOLDERS :)
mongorestore \ -u 'julien' \ -p 'password' \ -authenticationDatabase test \ -c collectionName \ --drop /home/backups/07-03-19/test
Last one means that “/home/backups/07-03-19/test” contains ONLY that collation.
In case the backup has been taken with the “–gzip” option it should be restored with the same:
root@b2dcec2eba9d:/data/db/dump/incident# mongorestore --username root --password="password" -d incident --authenticationDatabase admin --gzip --drop /data/db/dump/incident 2020-01-20T10:14:54.470+0000 the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead 2020-01-20T10:14:54.470+0000 building a list of collections to restore from /data/db/dump/incident dir 2020-01-20T10:14:54.563+0000 reading metadata for incident.files.chunks from /data/db/dump/incident/files.chunks.metadata.json.gz 2020-01-20T10:14:54.585+0000 reading metadata for incident.audit.trails from /data/db/dump/incident/audit.trails.metadata.json.gz *************************************************************************************** 2020-01-20T11:39:57.415+0000 [#######################.] incident.files.chunks 208GB/208GB (99.9%) 2020-01-20T11:40:00.411+0000 [#######################.] incident.files.chunks 208GB/208GB (100.0%) 2020-01-20T11:40:02.398+0000 [########################] incident.files.chunks 208GB/208GB (100.0%) 2020-01-20T11:40:02.398+0000 restoring indexes for collection incident.files.chunks from metadata 2020-01-20T11:40:02.449+0000 finished restoring incident.files.chunks (898117 documents) 2020-01-20T11:40:02.449+0000 done
Please bare in mind, that the database accessability will be VERY LIMITED as the entire server will be locked.