EDB - Barman - Part II: Backup via rsync/SSH

In Part I, we have already configured backup via rsync/SSH. In this section, we will backup database with this method.

Note: Barman implements incremental backup through a global/server option called reuse_backup, that transparently manages the barman backup command. It accepts three values:

• off: standard full backup (default)
• link: incremental backup, by reusing the last backup for a server and creating a hard link of the unchanged files (for backup space and time reduction)
• copy: incremental backup, by reusing the last backup for a server and creating a copy of the unchanged files (just for backup time reduction)

1. Validate configuration

[barman@barman-server ~]$ barman list-servers
employees4ssh - Backup employees database via rsync/SSH 

[barman@barman-server ~]$ barman status employees4ssh

[barman@barman-server ~]$ barman check employees4ssh
Server employees4ssh:
        PostgreSQL: OK
        superuser or standard user with backup privileges: OK
        wal_level: OK
        directories: OK
        retention policy settings: OK
        backup maximum age: OK (no last_backup_maximum_age provided)
        backup minimum size: OK (0 B)
        wal maximum age: OK (no last_wal_maximum_age provided)
        wal size: OK (0 B)
        compression settings: OK
        failed backups: OK (there are 0 failed backups)
        minimum redundancy requirements: OK (have 0 backups, expected at least 0)
        ssh: OK (PostgreSQL server)
        systemid coherence: OK
        archive_mode: OK
        archive_command: OK
        continuous archiving: OK
        archiver errors: OK

[barman@barman-server ~]$ barman show-server employees4ssh
Server employees4ssh:
        active: True
        archive_command: barman-wal-archive barman-server employees4ssh %p
        archive_mode: on
        backup_directory: /barman/backup/employees4ssh
        backup_method: rsync
        basebackups_directory: /barman/backup/employees4ssh/base
        conninfo: host=edb-saigon.taolaoxibup.com user=barman dbname=employees
        current_lsn: 0/8DBA0208
        current_size: 756601559
        current_xlog: 00000001000000000000008D
        data_directory: /u01/edb/as16/data
        description: Backup employees database via rsync/SSH
        errors_directory: /barman/backup/employees4ssh/errors
        last_archived_time: 2024-06-12 14:18:51.796703+07:00
        last_archived_wal: 00000001000000000000008C
        name: employees4ssh
        parallel_jobs: 4
        ssh_command: ssh enterprisedb@edb-saigon.taolaoxibup.com
        streaming_backup_name: barman_streaming_backup
        streaming_conninfo: host=edb-saigon.taolaoxibup.com user=barman dbname=employees
        streaming_wals_directory: /barman/backup/employees4ssh/streaming
        synchronous_standby_names: [''] 
        wal_level: replica
        wal_retention_policy: main
        wal_streaming_conninfo: None
        wals_directory: /barman/backup/employees4ssh/wals
        xlog_segment_size: 16777216

EDB - Barman - Part I: Installation and Configuration

 1. System information

Barman server

• IP : 192.168.56.101
• Barman version: 3.10.0 Barman by EnterpriseDB (www.enterprisedb.com)
• The pg_basebackup,pg_receivewal/pg_receivexlog, pg_verifybackup binaries are installed with the PostgreSQL client/server packages
• [root@barman-server ~]# dnf install barman barman-cli edb-as16-server

Database server

• IP: 192.168.56.102
• Database version: PostgreSQL 16.3 (EnterpriseDB Advanced Server 16.3.0) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.4.1 20231218 (Red Hat 11.4.1-3), 64-bit
• Package barman-cli:
• Using barman-wal-archive instead of rsync/SSH reduces the risk of data corruption of the shipped WAL file on the Barman server. When using rsync/SSH as archive_command a WAL file, there is no mechanism that guarantees that the content of the file is flushed and fsync-ed to disk on destination.
• [root@edb-saigon ~]# dnf install barman-cli
• After configuring "employees-ssh" on barman-server, we can check:
• [enterprisedb@edb-saigon ~]$ barman-wal-archive --test barman-server employees4ssh /barman/backup/employees4ssh/incoming
  Ready to accept WAL files for the server employees4ssh 

• Database parameters:
• archive_mode = on
• archive_command = ' test ! -f /u01/edb/as16/data/archivelog/%f && cp %p /u01/edb/as16/data/archivelog/%f && rsync -a %p barman@192.168.56.101:/barman/backup/employees4ssh/incoming/%f'
• Or archive_command = 'barman-wal-archive barman-server employees4ssh %p'
• wal_level = replica (optional)
• max_replication_slots = 4
• max_wal_senders = 4
• Database users for backup by barman:
• barman user
• [enterprisedb@edb-saigon ~]$ createuser -s -P barman
• streaming_barman user
• [enterprisedb@edb-saigon ~]$ createuser -P --replication streaming_barman
• pg_hba.conf 
• host    all                  all             0.0.0.0/0           trust
• host    replication     all             0.0.0.0/0            trust

EDB - Installing EDB Postgres Advanced Server 16

1. Setup repository
[root@edb-saigon ~]# curl -1sLf 'https://downloads.enterprisedb.com/XYZ.../enterprise/setup.rpm.sh' | sudo -E bash
... ... ...
... ... ...
   OK: Updating the dnf cache to fetch the new repository metadata ...
   OK: The repository has been installed successfully - You're ready to rock!

Note:
Install EPAS v16, we need lz4 package. So we must to configure Oracle Linux Yum Repo.

[root@edb-nhatrang ~]# vi /etc/yum.repos.d/oracle-linux-9u4.repo

[ol9_baseos_latest]
name=Oracle Linux 9 BaseOS Latest  ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL9/baseos/latest/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol9_appstream_latest]
name=Oracle Linux 9 AppStream ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL9/appstream/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol9_developer_latest]
name=Oracle Linux 9 Developer ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL9/developer/EPEL/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[root@edb-nhatrang ~]# wget https://yum.oracle.com/RPM-GPG-KEY-oracle-ol9 -O /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle

[root@edb-nhatrang ~]# dnf clean all
[root@edb-nhatrang ~]# dnf repolist

2. Install the software packages

[root@edb-saigon ~]# dnf -y install edb-as16-server

[root@edb-saigon ~]# id enterprisedb
uid=986(enterprisedb) gid=983(enterprisedb) groups=983(enterprisedb)

[root@edb-saigon ~]# passwd enterprisedb

RHEL9 - Set static IP address

Requirement: Assign static IP address on enp0s3 interface:

·       IP: 192.168.56.56

·       netmask: 255.255.255.0

·       gateway: 192.168.56.1

·       dns: 8.8.8.8

1. Check information

[root@barman ~]# systemctl status NetworkManager.service

[root@barman ~]# NetworkManager --print-config

[root@barman ~]# nmcli device

 DEVICE  TYPE      STATE                   CONNECTION
 enp0s8  ethernet  connected               enp0s8   
 enp0s3  ethernet  connected               enp0s3   
 lo      loopback  connected (externally)  lo    
 

 [root@barman ~]# nmcli connection show

 NAME    UUID                                  TYPE      DEVICE
 enp0s8  a2be3b45-4395-35ab-ba08-e6c7ba92d236  ethernet  enp0s8
 enp0s3  247f61a8-b997-3f30-88ea-688882942b52  ethernet  enp0s3
 lo      b259c109-60d7-4682-8016-192ebb4de8ae  loopback  lo    

  

[root@barman ~]# ls -l /etc/NetworkManager/system-connections/

 total 8
 -rw-------. 1 root root 229 Jun  6 10:37 enp0s3.nmconnection
 -rw-------. 1 root root 229 Jun  6 10:37 enp0s8.nmconnection

 

[root@barman ~]# ip addr show enp0s3

[root@barman ~]# cat /etc/NetworkManager/system-connections/enp0s3.nmconnection

  

2. Network Configuration

[root@barman ~]# nmcli connection modify 'enp0s3' ifname enp0s3 ipv4.method manual ipv4.addresses 192.168.56.56/24 gw4 192.168.56.1 ipv4.dns 8.8.8.8

[root@barman ~]# nmcli connection down 'enp0s3'

[root@barman ~]# nmcli connection up 'enp0s3'

 

RHEL8 - Create Local HTTP Yum/DNF Repository

 A. Yum Server

1.      Install package httpd, createrepo

2.      Mount iso image for rhel8, edb4rhel8

[root@yum-server conf.d]# df –h

…

/dev/loop1              13G   13G     0 100% /edb

/dev/loop2              14G   14G     0 100% /rhel8

3.      Copy all files from mount disks to repository

[root@yum-server ~]# cp -r /rhel8/* /source/www/html/repo

[root@yum-server ~]# cp -r /edb/* /source/www/html/repo/EDB

4.      Create Repository Metadata

[root@yum-server ~]# createrepo /source/www/html/repo/BaseOS

[root@yum-server ~]# createrepo /source/www/html/repo/AppStream

[root@yum-server ~]# createrepo /source/www/html/repo/EDB

 

[root@yum-server ~]# ls -l /source/www/html/repo

total 400

dr-xr-xr-x 4 root root   4096 Jun  5 09:14 AppStream

dr-xr-xr-x 4 root root   4096 Jun  5 09:13 BaseOS

drwxr-xr-x 4 root root 331776 Jun  5 09:24 EDB

dr-xr-xr-x 3 root root   4096 Jun  5 08:59 EFI

-r--r--r-- 1 root root   8154 Jun  5 09:05 EULA

-r--r--r-- 1 root root   1455 Jun  5 09:05 extra_files.json

-r--r--r-- 1 root root  18092 Jun  5 09:05 GPL

dr-xr-xr-x 3 root root   4096 Jun  5 09:05 images

dr-xr-xr-x 2 root root   4096 Jun  5 09:05 isolinux

-r--r--r-- 1 root root    104 Jun  5 09:05 media.repo

drwxr-xr-x 2 root root   4096 Jun  5 09:30 repodata

-r--r--r-- 1 root root   1669 Jun  5 09:05 RPM-GPG-KEY-redhat-beta

-r--r--r-- 1 root root   5135 Jun  5 09:05 RPM-GPG-KEY-redhat-release

-r--r--r-- 1 root root   1796 Jun  5 09:05 TRANS.TBL

5.      Configuring Local Repository

[root@yum-server ~]# ls -l /etc/yum.repos.d/

…

-rw-r--r-- 1 root root  504 Jun  5 09:23 rhel8.repo

-r--r--r-- 1 root root 5135 Jun  4 13:40 RPM-GPG-KEY-RHEL8 (copy from RPM-GPG-KEY-redhat-release and rename)

 

[root@yum-server ~]# vi /etc/yum.repos.d/rhel8.repo

[rhel8u10_baseos]

name=Local - Red Hat Enterprise Linux 8.10 - BaseOS

baseurl=file:///source/www/html/repo/BaseOS/

gpgkey=file:///etc/yum.repos.d/RPM-GPG-KEY-RHEL8

gpgcheck=1

enabled=1

 

[rhel8u10_appstream]

name=Local - Red Hat Enterprise Linux 8.10 - Appstream

baseurl=file:///source/www/html/repo/AppStream/

gpgkey=file:///etc/yum.repos.d/RPM-GPG-KEY-RHEL8

gpgcheck=1

enabled=1

 

[edb4rhel8]

name=Local - Enterprise Database for RHEL8/OL8

baseurl=file:///source/www/html/repo/EDB/

gpgcheck=0

enabled=1