Showing posts with label MongoDB. Show all posts
Showing posts with label MongoDB. Show all posts

Thursday, August 29, 2024

MongoDB - Performance Tuning in Linux for DBAs

There are some important elements must be considered carefully for tuning database performance.

1. File Systems

  • Using XFS as it generally performs better with MongoDB. 
  • With the WiredTiger storage engine, using XFS is strongly recommended for data bearing nodes to avoid performance issues that may occur when using EXT4 with WiredTiger.
  • Set 'noatime' on MongoDB data volumes in /etc/fstab
2. Swappiness
  • “Swappiness” is a Linux kernel setting that influences the behavior of the Virtual Memory manager. The vm.swappiness setting ranges from 0 to 100: the higher the value, the more strongly it prefers swapping memory pages to disk over dropping pages from RAM
  • Set vm.swappiness to 1, not 0
  • Allocating RAM as much as possible (storage.wiredTiger.engineConfig.cacheSizeGB)
  • Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages.
  • However, database workloads often perform poorly with THP enabled, because they tend to have sparse rather than contiguous memory access patterns. When running MongoDB on Linux, THP should be disabled for best performance
  • Avoid overloading the connection resources of a mongod or mongos instance by adjusting the connection pool size to suit your use case.
  • Specify connection pool settings in these locations:
    • The MongoDB URI
    • Your application's MongoClient instance
    • Your application framework's configuration files
  • Some parameters control connections in /etc/mongod.conf file:
    • net.maxIncomingConnections
    • setting parameters with setParameter: option
  • For improved performance, consider separating your database's data, journal, and logs onto different storage devices, based on your application's access and write pattern. 
  • Mount the components as separate filesystems and use symbolic links to map each component's path to the device storing it.
  • For the WiredTiger storage engine, you can also store the indexes on a different storage device by configuring storage.wiredTiger.engineConfig.directoryForIndexes parameter.

Wednesday, August 28, 2024

MongoDB - /etc/mongod.conf sample file

[root@mongodb-01 ~]# cat /etc/mongod.conf 
# mongod.conf

# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# Where and how to store data.
storage:
  dbPath: /u01/mongoDB/helloWorld
  journal:
    enabled: true
  engine: wiredTiger
  wiredTiger:
    engineConfig:
      journalCompressor: snappy
      cacheSizeGB: 1.5            // Adjust base on physical resource
    collectionConfig:
      blockCompressor: snappy

# how the process runs - Controls how MongoDB is managed as a system process.
processManagement:
  fork: true                              # Enable MongoDB to run as a background daemon.
  pidFilePath: /var/run/mongodb/mongod.pid     # Location of the PID file when running as a service.
  timeZoneInfo: /usr/share/zoneinfo

Wednesday, August 14, 2024

MongoDB - Percona Backup for MongoDB

Percona Backup for MongoDB (PBM) is an open source and distributed solution for consistent backups and restores of MongoDB sharded clusters and replica sets to a specific point in time.

Architecture
Percona Backup for MongoDB consists of the following components:
  • pbm-agent is a process running on every mongod node within the cluster or within a replica set that performs backup and restore operations.
  • pbm CLI is a command-line utility that instructs pbm-agents to perform an operation.
  • PBM Control collections are special collections in MongoDB that store the configuration data and backup states. Both pbm CLI and pbm-agent use PBM Control collections to check backup status in MongoDB and communicate with each other.
  • Remote backup storage is where Percona Backup for MongoDB saves backups. It can be either an S3 compatible storage or a filesystem-type storage.
The following diagram illustrates how Percona Backup for MongoDB components communicate with MongoDB.




1. Installation
  • Install percona-release
    • $ sudo yum install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm
  • Enable the repository
    • $ sudo percona-release enable pbm release
  • Install Percona Backup for MongoDB packages
    • $ sudo yum install percona-backup-mongodb
[root@mongodb-01 ~]# ls -l /etc | grep pbm
-rw-r-----   1 root         root           3626 May 15 17:10 pbm-conf-reference.yml
-rw-r-----   1 mongod       mongod           91 Aug 14 10:56 pbm-storage.conf

2. Configuration

đź‘€Create user for backup on Primary Node

use admin

db.getSiblingDB("admin").createRole({ "role": "pbmAnyAction",
      "privileges": [
         { "resource": { "anyResource": true },
           "actions": [ "anyAction" ]
         }
      ],
      "roles": []
   });
   
db.getSiblingDB("admin").createUser({user: "p4backup",
       "pwd": "Password789",
       "roles" : [
          { "db" : "admin", "role" : "readWrite", "collection": "" },
          { "db" : "admin", "role" : "backup" },
          { "db" : "admin", "role" : "clusterMonitor" },
          { "db" : "admin", "role" : "restore" },
          { "db" : "admin", "role" : "pbmAnyAction" }
       ]
    });  

Tuesday, August 06, 2024

MongoDB - Replication

Preparation
1. Install mongodb on all nodes
2. Configuring /etc/hosts on nodes
[root@mongodb-xxx ~]# cat /etc/hosts 
192.168.56.87  mongodb-01.taolaoxibup.com               mongodb-01
192.168.56.88  mongodb-02.taolaoxibup.com               mongodb-02
192.168.56.89  mongodb-slave.taolaoxibup.com           mongodb-slave
192.168.56.90  mongodb-arbiter.taolaoxibup.com         mongodb-arbiter    // an arbiter
 
3. Configuring /etc/mongod.conf on all nodes
[root@mongodb-xxx ~]# vi /etc/mongod.conf 
...

# network interfaces
net:
  port: 27017
  bindIp: mongodb-01  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.            // Update right value for each mongos instances

replication:
  replSetName: "rs0"
...

[root@mongodb-xxx ~]# systemctl restart mongod.service 

Configuration

1. On Primary Node
- Initiate the replica set
  • rs.initiate()
  • rs.status()
  • rs.conf()
- Add Members to a Replica Set
  • rs.add({host: "mongodb-02:27017"})
  • rs.add({host: "mongodb-slave:27017"})
- Verify configuration
  • rs.status()
  • rs.conf()
  • rs.hello()
  • db.isMaster()
2. On Secondary node - Verify configuration
  • rs.status()
  • rs.conf()
  • rs.hello()
  • db.isMaster()
3. Add Members to a Replica Set
  • Start new mongod instance
    • mongod --port 27018 --dbpath /u01/mongoDB/saigon --replSet rs0 --bind_ip 0.0.0.0
    • OR nohup mongod --port 27018 --dbpath /u01/mongoDB/saigon --replSet replOMNI --bind_ip_all > 27018_$HOSTNAME.log 2>&1 &
  • On Primary node, add new member to replica set
    • rs.add("mongodb-01:27018")
  • Verify configuration
    • rs.status()
    • rs.conf()
    • db.isMaster()