Thursday, August 29, 2024

MongoDB - Performance Tuning in Linux for DBAs

There are some important elements must be considered carefully for tuning database performance.

1. File Systems

  • Using XFS as it generally performs better with MongoDB. 
  • With the WiredTiger storage engine, using XFS is strongly recommended for data bearing nodes to avoid performance issues that may occur when using EXT4 with WiredTiger.
  • Set 'noatime' on MongoDB data volumes in /etc/fstab
2. Swappiness
  • “Swappiness” is a Linux kernel setting that influences the behavior of the Virtual Memory manager. The vm.swappiness setting ranges from 0 to 100: the higher the value, the more strongly it prefers swapping memory pages to disk over dropping pages from RAM
  • Set vm.swappiness to 1, not 0
  • Allocating RAM as much as possible (storage.wiredTiger.engineConfig.cacheSizeGB)
  • Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages.
  • However, database workloads often perform poorly with THP enabled, because they tend to have sparse rather than contiguous memory access patterns. When running MongoDB on Linux, THP should be disabled for best performance
  • Avoid overloading the connection resources of a mongod or mongos instance by adjusting the connection pool size to suit your use case.
  • Specify connection pool settings in these locations:
    • The MongoDB URI
    • Your application's MongoClient instance
    • Your application framework's configuration files
  • Some parameters control connections in /etc/mongod.conf file:
    • net.maxIncomingConnections
    • setting parameters with setParameter: option
  • For improved performance, consider separating your database's data, journal, and logs onto different storage devices, based on your application's access and write pattern. 
  • Mount the components as separate filesystems and use symbolic links to map each component's path to the device storing it.
  • For the WiredTiger storage engine, you can also store the indexes on a different storage device by configuring storage.wiredTiger.engineConfig.directoryForIndexes parameter.

WiredTiger can compress collection data using one of the following compression library:
  • snappy (default): Provides a lower compression rate than zlib or zstd but has a lower CPU cost than either.
  • zlib: Provides better compression rate than snappy but has a higher CPU cost than both snappy and zstd.
  • zstd: Provides better compression rate than both snappy and zlib and has a lower CPU cost than zlib.

  • Collection data in the WiredTiger internal cache is uncompressed and uses a different representation from the on-disk format.
  • Data in the filesystem cache is the same as the on-disk format, including benefits of any compression for data files. The filesystem cache is used by the operating system to reduce disk I/O. With the filesystem cache, MongoDB automatically uses all free memory that is not used by the WiredTiger cache or by other processes.
So, we should:
  • Never deploy multiple mongods into the same server.
  • Set storage.wiredTiger.engineConfig.cacheSizeGB equal 50% of physical of RAM.
  • The database profiler collects detailed information about Database Commands executed against a running mongod instance. This includes CRUD operations as well as configuration and administration commands.
  • The profiler writes all the data it collects to a system.profile collection, a capped collection in each profiled database.
  • When enabled, profiling has an effect on database performance and disk use
  • Write concern describes the level of acknowledgment requested from MongoDB for write operations to a standalone mongod, replica sets, or sharded clusters. In sharded clusters, mongos instances will pass the write concern on to the shards.
  • Replica sets and sharded clusters support setting a global default write concern. Operations which do not specify an explicit write concern inherit the global default write concern settings
  • Write Concern for Replica Sets describe the number of data-bearing members (i.e. the primary and secondaries, but not arbiters) that must acknowledge a write operation before the operation returns as successful. A member can only acknowledge a write operation after it has received and applied the write successfully.
  • The dirty_ratio  is the percentage of total system memory that can hold dirty pages. The default on most Linux hosts is between 20-30%. When you exceed the limit, the dirty pages are committed to the disk, creating a small pause. 
  • To avoid the hard pause, there is a second ratio: dirty_background_ratio (default 10-15%) which tells the kernel to start flushing dirty pages to disk in the background without any pause.
  • Can set this by adding the following lines to the /etc/sysctl.conf
    • sysctl -a | egrep "vm.dirty.*_ratio"
    • vm.dirty_ratio = 15
    • vm.dirty_background_ratio = 5
  • Edit the MongoDB configuration file and add the below setParameter option:
    • setParameter:
      •   storageEngineConcurrentReadTransactions: 128
      •   storageEngineConcurrentWriteTransactions: 128
  • Restart mongod service
12. Monitor commands and tools
  • db.dbstats()
  • db.collection.stats()
  • db.getReplicationInfo()
  • db.serverStatus()
    • db.serverStatus().wiredTiger                  //{cache,concurrentTransactions,transaction,session}
    • db.serverStatus().opcounters
    • db.serverStatus().metrics
    • db.serverStatus().locks
    • db.serverStatus().globalLock
    • db.serverStatus().network
    • db.serverStatus().connections
    • db.serverStatus().extra_info
  • db.currentOp()
  • rs.status()
  • rs.printSlaveReplicationInfo()
  • rs.printReplicationInfo()
  • sh.status()
  • mongotop
  • mongostat
Ref:

at
Labels: