Friday, August 23, 2024

13.5.0 Enterprise Manager Cloud Control - Holistic Patch for Oracle Management Service - July 2024

Holistic Patching in Enterprise Manager is a method of applying all stack patch bundle for a given CPU released by Oracle for WebLogic and its components. Applying all SPB,CPU patches on OMS is a downtime patching activity and requires maintenance window. 

NoteThere is no dependency between OMS RU Patch to Holistic Patch.

A. Current System Information

1. Database version

[oracle@monitor-db ~]$ opatch lspatches

36233263;Database Release Update : 19.23.0.0.240416 (36233263)

29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399) 

OPatch succeeded.

2. OMS version

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ omspatcher lspatches

oracle.sysman.db.oms.plugin/13.5.1.0.0            Plugin              36335368            36329033            Oracle Enterprise Manager for Oracle Database 13c Release 5 Plug-in Update 22 (13.5.1.22) for Oracle Management Service

oracle.sysman.cfw.oms.plugin/13.5.1.0.0           Plugin              36335368            36329231            Oracle Enterprise Manager for Cloud Framework 13c Release 5 Plug-in Update 22 (13.5.1.22) for Oracle Management Service

oracle.sysman.xa.oms.plugin/13.5.1.0.0            Plugin              36335368            36329152            Oracle Enterprise Manager for Exadata 13c Release 5 Plug-in Update 22 (13.5.1.22) for Oracle Management Service

oracle.sysman.top.oms/13.5.0.0.0                  Core                36335368            36329009            Oracle Enterprise Manager 13c Release 5 Platform Update 22 (13.5.0.22) for Oracle Management Service

oracle.sysman.si.oms.plugin/13.5.1.0.0            Plugin              36335368            36329196            Oracle Enterprise Manager for Systems Infrastructure 13c Release 5 Plug-in Update 22 (13.5.1.22) for Oracle Management Service

oracle.sysman.emas.oms.plugin/13.5.1.0.0          Plugin              36335368            36329020            Oracle Enterprise Manager for Fusion Middleware 13c Release 5 Plug-in Update 22 (13.5.1.22) for Oracle Management Service

3. Agent version

[oracle@monitor-app ~]$ . agent_env

[oracle@monitor-app ~]$ agentpatcher lspatches

oracle.sysman.si.agent.plugin/13.5.1.0.0          Plugin              36335371            36335417            Oracle Enterprise Manager for Systems Infrastructure 13c Release 5 Plug-in Update 22 (13.5.1.22) for Oracle Management Agent

oracle.sysman.top.agent/13.5.0.0.0                Core                36335371            36335374            Oracle Enterprise Manager 13c Release 5 Platform Update 22 (13.5.0.22) for Oracle Management Agent

4. OMSPatcher version

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ omspatcher version

OMSPatcher Version: 13.9.5.20.0

OPlan Version: 12.2.0.1.16

OsysModel build: Tue Apr 28 18:16:31 PDT 2020 

OMSPatcher succeeded. 

B. Holistic Patch for Oracle Management Service - July 2024

Ref: OEM 13c: How To Apply/Rollback Holistic Patch (SPB, CPU, and JDK) On OEM 13.5 (Doc ID 3002450.1)  

1. Upgrade OMSPatcher version as per Holistic patch README - 13.9.5.21.0

Ref: 13.5: How to Upgrade Enterprise Manager 13.5 Cloud Control OMSPatcher Utility to the Latest Version (Doc ID 2809842.1)

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ cd $ORACLE_HOME

[oracle@monitor-app middleware]$ mv OMSPatcher OMSPatcher.22Aug

[oracle@monitor-app ~]$ cd /source/monitor/patch/

[oracle@monitor-app patch]$ unzip p19999993_135000_Generic.zip -d $ORACLE_HOME

[oracle@monitor-app middleware]$ omspatcher version

OMSPatcher Version: 13.9.5.21.0

OPlan Version: 12.2.0.1.16

OsysModel build: Tue Apr 28 18:16:31 PDT 2020 

OMSPatcher succeeded.


2. Apply one-off Patch 36634909

Start Oracle WebLogic Administration server ONLY

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ emctl stop oms –all

[oracle@monitor-app ~]$ emctl start oms -admin_only

[oracle@monitor-app ~]$ emctl status oms -details

Oracle Enterprise Manager Cloud Control 13c Release 5 

Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.

Console Server Host        : monitor-app.taolaobidao.com

HTTP Console Port          : 7788

HTTPS Console Port         : 7803

HTTP Upload Port           : 4889

HTTPS Upload Port          : 4903

EM Instance Home           : /u01/app/oracle/gc_inst/em/EMGC_OMS1

OMS Log Directory Location : /u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/log

OMS is not configured with SLB or virtual hostname

Agent Upload is locked.

OMS Console is locked.

Active CA ID: 1

Console URL: https://monitor-app.taolaobidao.com:7803/em

Upload URL: https://monitor-app.taolaobidao.com:4903/empbs/upload

 

WLS Domain Information

Domain Name            : GCDomain

Admin Server Host      : monitor-app.taolaobidao.com

Admin Server HTTPS Port: 7102

Admin Server is RUNNING

 

Oracle Management Server Information

Managed Server Instance Name: EMGC_OMS1

Oracle Management Server Instance Host: monitor-app.taolaobidao.com

WebTier is Down

Oracle Management Server is Down

JVMD Engine is Down

 

Creating a Property File

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ $ORACLE_HOME/OMSPatcher/wlskeys/createkeys.sh -oh /u01/app/oracle/middleware -location /home/oracle/encrypted

The oraclehome passed is valid.

The wls url is t3s://monitor-app.taolaobidao.com:7102

 

Please enter weblogic admin server username : weblogic 

Please enter weblogic admin server password :

The cfgFile is /home/oracle/encrypted/config

The keyFile is /home/oracle/encrypted/key

 

Initializing WebLogic Scripting Tool (WLST) ... 

Welcome to WebLogic Server Administration Scripting Shell 

Type help() for help on available commands

 

Thu Aug 22 14:11:46 ICT 2024

Storing WLS User configuration

Admin url: t3s://monitor-app.taolaobidao.com:7102

Admin Config File: /home/oracle/encrypted/config

Key File : /home/oracle/encrypted/key

Please enter weblogic admin server username :

Enter Admin user's password:

Thu Aug 22 14:11:46 ICT 2024

Connecting to AdminServer

Connecting to t3s://monitor-app.taolaobidao.com:7102 with userid weblogic ...

<Aug 22, 2024 2:11:47 PM ICT> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.>

<Aug 22, 2024 2:11:47 PM ICT> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.>

<Aug 22, 2024 2:11:47 PM ICT> <Info> <Security> <BEA-090908> <Using the default WebLogic SSL Hostname Verifier implementation.>

Successfully connected to Admin Server "EMGC_ADMINSERVER" that belongs to domain "GCDomain".

 

Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key...

The username and password that were used for this WebLogic Server connection are stored in /home/oracle/encrypted/config and /home/oracle/encrypted/key.

Disconnected from weblogic server: EMGC_ADMINSERVER

******************  **********************88

0

User configuration file created: /home/oracle/encrypted/config

User key file created: /home/oracle/encrypted/key

'createkeys' succeeded.


[oracle@monitor-app ~]$ ls -l /home/oracle/encrypted/

total 8

-rw-r----- 1 oracle oinstall 233 Aug 22 14:11 config

-rw-r----- 1 oracle oinstall  80 Aug 22 14:11 key

 

[oracle@monitor-app ~]$ vi /home/oracle/encrypted/oem_property_file

AdminServerURL=t3s://monitor-app.taolaobidao.com:7102

AdminConfigFile=/home/oracle/encrypted/config

AdminKeyFile=/home/oracle/encrypted/key

 

[oracle@monitor-app ~]$ ls -l /home/oracle/encrypted/

total 12

-rw-r----- 1 oracle oinstall 233 Aug 22 14:11 config

-rw-r----- 1 oracle oinstall  80 Aug 22 14:11 key

-rw-r--r-- 1 oracle oinstall 139 Aug 22 14:16 oem_property_file

 

Apply patch 36634909

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ cd /source/monitor/patch/36634909/

[oracle@monitor-app 36634909]$ omspatcher apply -analyze -property_file /home/oracle/encrypted/oem_property_file

[oracle@monitor-app 36634909]$ omspatcher apply -property_file /home/oracle/encrypted/oem_property_file

OMSPatcher Automation Tool

Copyright (c) 2017, Oracle Corporation.  All rights reserved.

  

OMSPatcher version : 13.9.5.21.0

OUI version        : 13.9.4.0.0

Running from       : /u01/app/oracle/middleware

Log file location  : /u01/app/oracle/middleware/cfgtoollogs/omspatcher/opatch2024-08-22_14-35-52PM_1.log 

OMSPatcher log file: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/36634909/omspatcher_2024-08-22_14-35-52PM_apply.log 

 

Enter DB user name : sys

Enter 'sys' password :

Checking if current repository database is a supported version

Current repository database version is supported 

 

Prereq "checkComponents" for patch 36634909 passed.

Configuration Validation: Success

Running apply prerequisite checks for sub-patch(es) "36634909" and Oracle Home "/u01/app/oracle/middleware"...

Sub-patch(es) "36634909" are successfully analyzed for Oracle Home "/u01/app/oracle/middleware"

 

To continue, OMSPatcher will do the following:

[Patch and deploy artifacts]   : 

Do you want to proceed? [y|n]

y

User Responded with: Y 

Applying sub-patch(es) "36634909"

Please monitor log file: /u01/app/oracle/middleware/cfgtoollogs/opatch/opatch2024-08-22_14-36-08PM_1.log 

Complete Summary

================ 

All log file names referenced below can be accessed from the directory "/u01/app/oracle/middleware/cfgtoollogs/omspatcher/2024-08-22_14-35-52PM_SystemPatch_36634909_1" 

Patching summary:

----------------- 

Binaries of the following sub-patch(es) have been applied successfully:

 

                        Featureset   Sub-patches                                     Log file

                        ----------   -----------                                     --------

  oracle.sysman.top.oms_13.5.0.0.0      36634909   36634909_opatch2024-08-22_14-36-08PM_1.log

Log file location: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/36634909/omspatcher_2024-08-22_14-35-52PM_apply.log

 OMSPatcher succeeded.

 

[oracle@monitor-app ~]$ omspatcher lspatches

oracle.sysman.top.oms/13.5.0.0.0                  Core                36634909            36634909           

 

3. Apply Holistic Patch 36929569

Note: ONLY the Oracle WebLogic Administration server is UP and RUNNING

[oracle@monitor-app ~]$ . oms_env

[oracle@monitor-app ~]$ omspatcher apply /source/monitor/patch/36929569 -spb_patch -analyze

JDK update log location :: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/jdk_update_Thu_Aug_22_14_45_03_2024.log

SPB JDK version :: 1.8.0.421,  Env JDK version :: 1.8.0.261

The Env version is lower as compared to JDK included in patch :: JDK will be updated

OPatch update log :: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/opatch_update_Thu_Aug_22_14_45_08_2024.log

 

The Env Opatch version :: 13.9.4.2.15

The SPB Opatch version :: 13.9.4.2.16

The Environment Opatch version is low as per OPatch included in Patch :: OPatch will be updated

 

OMSPatcher log file :: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/omspatcher_analyze_Thu_Aug_22_14_45_10_2024.log

SUCCESS :: The minimum Release Update (RU) check passed for applying the given holistic patch. Minimum OMS RU Version is 13.5.0.21

Running Command :: /u01/app/oracle/middleware/OPatch/opatch lspatches -oh /u01/app/oracle/middleware

Running Command :: /u01/app/oracle/middleware/OPatch/opatch prereq CheckForNoOpPatches -oh /u01/app/oracle/middleware -phbasefile /u01/app/oracle/middleware/linux64_patchlist.txt -silent

Running Command :: /u01/app/oracle/middleware/OPatch/opatch prereq CheckConflictAgainstOH -oh /u01/app/oracle/middleware -phbasefile /u01/app/oracle/middleware/linux64_patchlist.txt -silent

 

OMSPatcher succeeded.

 

[oracle@monitor-app ~]$ omspatcher apply /source/monitor/patch/36929569 -spb_patch

JDK update log location :: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/jdk_update_Thu_Aug_22_14_55_13_2024.log

 

SPB JDK version :: 1.8.0.421,  Env JDK version :: 1.8.0.261

 

The Env version is lower as compared to JDK included in patch :: JDK will be updated

Updating the JDK now...

Stopping the OMS...

Oracle Enterprise Manager Cloud Control 13c Release 5 

Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.

Stopping Oracle Management Server...

Oracle Management Server Already Stopped

Oracle Management Server is Down

JVMD Engine is Down

 

Back_up Java folder :: /u01/app/oracle/middleware/oracle_common/jdk_1724313330

 

New Java version is...

java version "1.8.0_421"

Java(TM) SE Runtime Environment (build 1.8.0_421-b25)

Java HotSpot(TM) 64-Bit Server VM (build 25.421-b25, mixed mode)

 

Updated Java Successfully

 

OPatch update log :: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/opatch_update_Thu_Aug_22_14_55_30_2024.log

 

The Env Opatch version :: 13.9.4.2.15

The SPB Opatch version :: 13.9.4.2.16

 

The Environment Opatch version is low as per OPatch included in Patch :: OPatch will be updated

Updating the Opatch now.......

Updated Opatch Successfully.......

OMSPatcher Automation Tool

Copyright (c) 2017, Oracle Corporation.  All rights reserved.

 

OMSPatcher version : 13.9.5.21.0

OUI version        : 13.9.4.0.0

Running from       : /u01/app/oracle/middleware

Log file location  : /u01/app/oracle/middleware/cfgtoollogs/omspatcher/opatch2024-08-22_14-56-18PM_1.log

OMSPatcher log file: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/omspatcher_2024-08-22_14-56-18PM_apply.log

Please enter OMS weblogic admin server URL(t3s://monitor-app.taolaobidao.com:7102):>

Please enter OMS weblogic admin server username(weblogic):> weblogic

Please enter OMS weblogic admin server password:>

 

Performing prerequisites.....

The following sub-patches are already applied to the OMS system:

 31657681 34153238 35430934

The following sub-patches will be applied to the OMS system:

36700543 34809489 35965629 1221422 33093748 36770738 32720458 36919198 36316422 35024228 34065178 36769312 34831198 36553894 36741053 36805124 36929608 

Loaded SPB properties successfully 

Stopping the oms.... 

Oracle Enterprise Manager Cloud Control 13c Release 5 

Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.

Stopping Oracle Management Server...

WebTier Successfully Stopped

Oracle Management Server Successfully Stopped

AdminServer Successfully Stopped

Oracle Management Server is Down

JVMD Engine is Down

Running the command :: /u01/app/oracle/middleware/OPatch/opatch napply -oh /u01/app/oracle/middleware -phbasefile /u01/app/oracle/middleware/linux64_patchlist.txt -silent

Log file to be checked to get the list of applied patches : /u01/app/oracle/middleware/cfgtoollogs/opatch/opatch2024-08-22_15-06-02PM_1.log

Command ran successfully 

Starting the oms.... 

Oracle Enterprise Manager Cloud Control 13c Release 5 

Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.

Starting Oracle Management Server...

WebTier Successfully Started

Oracle Management Server Successfully Started

Oracle Management Server is Up

JVMD Engine is Up

 

Log file location: /u01/app/oracle/middleware/cfgtoollogs/omspatcher/SPB_PATCH/omspatcher_2024-08-22_14-56-18PM_apply.log 

OMSPatcher succeeded.


[oracle@monitor-app ~]$ emctl status oms

Oracle Enterprise Manager Cloud Control 13c Release 5 

Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.

WebTier is Up

Oracle Management Server is Up

JVMD Engine is Up

 

[oracle@monitor-app ~]$ emctl status oms -details

Oracle Enterprise Manager Cloud Control 13c Release 5 

Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.

Console Server Host        : monitor-app.taolaobidao.com

HTTP Console Port          : 7788

HTTPS Console Port         : 7803

HTTP Upload Port           : 4889

HTTPS Upload Port          : 4903

EM Instance Home           : /u01/app/oracle/gc_inst/em/EMGC_OMS1

OMS Log Directory Location : /u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/log

OMS is not configured with SLB or virtual hostname

Agent Upload is locked.

OMS Console is locked.

Active CA ID: 1

Console URL: https://monitor-app.taolaobidao.com:7803/em

Upload URL: https://monitor-app.taolaobidao.com:4903/empbs/upload

 

WLS Domain Information

Domain Name            : GCDomain

Admin Server Host      : monitor-app.taolaobidao.com

Admin Server HTTPS Port: 7102

Admin Server is RUNNING

 

Oracle Management Server Information

Managed Server Instance Name: EMGC_OMS1

Oracle Management Server Instance Host: monitor-app.taolaobidao.com

WebTier is Up

Oracle Management Server is Up

JVMD Engine is Up

 

[oracle@monitor-app ~]$ omspatcher lspatches | more

[oracle@monitor-app ~]$ opatch lsinventory | more

Ref:

  • OEM 13c: Holistic Patch Frequently Asked Questions (Doc ID 3002413.1)
  • OEM 13c: How To Apply/Rollback Holistic Patch (SPB, CPU, and JDK) On OEM 13.5 (Doc ID 3002450.1)