Table of Contents

Grid Infra Patch on Node 1

We will assume 2 nodes configuration, so let's start going.

Once the patch has been unzip and the correct version of OPatch applied, we can start by analysing, if the patch is compatible:

Prepare

Unzip the patches

[oracle@lparaca OPatch]$ cd /home/oracle/grid/
[oracle@lparaca grid]$ ls -lart
total 2634532
-rw-r--r--  1 oracle oinstall 1014527110 Jan 13 07:05 p21419221_121020_Linux-x86                               -64_2of10.zip
-rw-r--r--  1 oracle oinstall 1673519571 Jan 13 07:08 p21419221_121020_Linux-x86                               -64_1of10.zip
-rw-r--r--  1 oracle oinstall    9705040 Jan 13 08:17 VNC-Viewer-6.17.1113-Windo                               ws.exe
drwxr-xr-x  4 oracle oinstall        165 Jan 17 17:36 .
drwxr-xr-x  2 oracle oinstall          6 Jan 17 17:36 grid
drwxr-xr-x  3 oracle oinstall        103 Jan 17 17:37 patch
drwx------. 7 oracle oinstall        192 Jan 18 08:33 ..
[oracle@lparaca grid]$ cd patch/
[oracle@lparaca patch]$ cd 27010872/
[oracle@lparaca 27010872]$ ls -lart
total 92
drwxr-xr-x  5 oracle oinstall    62 Jan  5 15:50 26925218
drwxr-xr-x  4 oracle oinstall    48 Jan  5 15:50 26983807
drwxr-xr-x  5 oracle oinstall    62 Jan  5 15:52 26910974
drwxrwxr-x 16 oracle oinstall   267 Jan  5 15:54 26925311
drwxr-xr-x  2 oracle oinstall  4096 Jan  5 15:54 automation
-rw-r--r--  1 oracle oinstall     0 Jan  5 15:55 README.txt
drwxr-xr-x  7 oracle oinstall   143 Jan  5 15:55 .
-rw-rw-r--  1 oracle oinstall  4991 Jan  5 16:31 bundle.xml
-rw-rw-r--  1 oracle oinstall 78371 Jan 16 09:00 README.html
drwxr-xr-x  3 oracle oinstall   103 Jan 17 17:37 ..

Check the OPatch

[oracle@lparaca 27010872]$ which opatch
/usr/bin/which: no opatch in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/                               sbin:/home/oracle/.local/bin:/home/oracle/bin:/u01/app/oracle/grid/bin)
[oracle@lparaca 27010872]$ ls -lart /pwd
ls: cannot access /pwd: No such file or directory
[oracle@lparaca 27010872]$ pwd
/home/oracle/grid/patch/27010872
[oracle@lparaca 27010872]$ ls -lart
total 92
drwxr-xr-x  5 oracle oinstall    62 Jan  5 15:50 26925218
drwxr-xr-x  4 oracle oinstall    48 Jan  5 15:50 26983807
drwxr-xr-x  5 oracle oinstall    62 Jan  5 15:52 26910974
drwxrwxr-x 16 oracle oinstall   267 Jan  5 15:54 26925311
drwxr-xr-x  2 oracle oinstall  4096 Jan  5 15:54 automation
-rw-r--r--  1 oracle oinstall     0 Jan  5 15:55 README.txt
drwxr-xr-x  7 oracle oinstall   143 Jan  5 15:55 .
-rw-rw-r--  1 oracle oinstall  4991 Jan  5 16:31 bundle.xml
-rw-rw-r--  1 oracle oinstall 78371 Jan 16 09:00 README.html
drwxr-xr-x  3 oracle oinstall   103 Jan 17 17:37 ..
[oracle@lparaca 27010872]$ /u01/app/oracle/grid/OPatch/opatchauto apply /home/oracle/grid/patch/27010872/ -analyze -ocmrf /home/oracle/ocmcheck.ocm


Check Consistencies

Here I though that opatchauto can be run as oracle user, boy was I wrong:

OPatchauto session is initiated at Thu Jan 18 08:52:05 2018
OPATCHAUTO-72046: Wallet option is not provided.
OPATCHAUTO-72046: Wallet option is required during patching as non-root user.
OPATCHAUTO-72046: Please provide a wallet option or execute as root user.

OPatchauto session completed at Thu Jan 18 08:52:06 2018
Time taken to complete the session 0 minute, 1 second

opatchauto bootstrapping failed with error code 46.
[oracle@lparaca 27010872]$ /u01/app/oracle/grid/OPatch/opatchauto apply /home/oracle/grid/patch/27010872/ -analyze

OPatchauto session is initiated at Thu Jan 18 08:52:21 2018
OPATCHAUTO-72046: Wallet option is not provided.
OPATCHAUTO-72046: Wallet option is required during patching as non-root user.
OPATCHAUTO-72046: Please provide a wallet option or execute as root user.

OPatchauto session completed at Thu Jan 18 08:52:21 2018
Time taken to complete the session 0 minute, 0 second

opatchauto bootstrapping failed with error code 46.
[oracle@lparaca 27010872]$ exit
logout

Changed to root. For some reason Oracle requires 9 times the space of the Patch in the Oracle home, so the first check failed:

[root@lparaca ~]# cd /home/oracle/
.bash_history      .bashrc            .oracle/           .vnc/
.bash_logout       grid/              .oracle_jre_usage/ .Xauthority
.bash_profile      .kshrc             .ssh/
[root@lparaca ~]# cd /home/oracle/grid/patch/
[root@lparaca patch]# cd 27010872/
[root@lparaca 27010872]#  /u01/app/oracle/grid/OPatch/opatchauto apply /home/oracle/grid/patch/27010872/ -analyze

OPatchauto session is initiated at Thu Jan 18 08:52:46 2018

System initialization log file is /u01/app/oracle/grid/cfgtoollogs/opatchautodb/systemconfig2018-01-18_08-52-50AM.log.

Session log file is /u01/app/oracle/grid/cfgtoollogs/opatchauto/opatchauto2018-01-18_08-52-59AM.log
The id for this session is 1RSZ

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/grid
Patch applicability verified successfully on home /u01/app/oracle/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Analysis for applying patches has failed:

Host:lparaca
CRS Home:/u01/app/oracle/grid
Analysis for patches has failed.

==Following patches FAILED in analysis for apply:

Patch: /home/oracle/grid/patch/27010872/26910974
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_08-53-07AM_1.log
Reason: Failed during Analysis: CheckSystemSpace Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:
Required amount of space(11244.146MB) is not available.]
Failed during Analysis: CheckNApplyReport Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:

Prerequisite check "CheckSystemSpace" failed.]


Patch: /home/oracle/grid/patch/27010872/26925218
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_08-53-07AM_1.log
Reason: Failed during Analysis: CheckSystemSpace Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:
Required amount of space(11244.146MB) is not available.]
Failed during Analysis: CheckNApplyReport Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:

Prerequisite check "CheckSystemSpace" failed.]


Patch: /home/oracle/grid/patch/27010872/26925311
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_08-53-07AM_1.log
Reason: Failed during Analysis: CheckSystemSpace Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:
Required amount of space(11244.146MB) is not available.]
Failed during Analysis: CheckNApplyReport Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:

Prerequisite check "CheckSystemSpace" failed.]


Patch: /home/oracle/grid/patch/27010872/26983807
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_08-53-07AM_1.log
Reason: Failed during Analysis: CheckSystemSpace Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:
Required amount of space(11244.146MB) is not available.]
Failed during Analysis: CheckNApplyReport Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:

Prerequisite check "CheckSystemSpace" failed.]



OPATCHAUTO-72053: Analysis for the patches failed.
OPATCHAUTO-72053: Command execution failed.
OPATCHAUTO-72053: Please check the summary for more details.

Second try to analyze it

After I have added space, I tried to check again for consistency between the patch and the grid home:

[root@lparaca /]#  /u01/app/oracle/grid/OPatch/opatchauto apply /home/oracle/grid/patch/27010872/ -analyze

OPatchauto session is initiated at Thu Jan 18 09:02:21 2018

System initialization log file is /u01/app/oracle/grid/cfgtoollogs/opatchautodb/systemconfig2018-01-18_09-02-23AM.log.

Session log file is /u01/app/oracle/grid/cfgtoollogs/opatchauto/opatchauto2018-01-18_09-02-30AM.log
The id for this session is VASA

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/grid

Patch applicability verified successfully on home /u01/app/oracle/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Analysis for applying patches has completed successfully:

Host:lparaca
CRS Home:/u01/app/oracle/grid


==Following patches were SUCCESSFULLY analyzed to be applied:

Patch: /home/oracle/grid/patch/27010872/26910974
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-02-37AM_1.log

Patch: /home/oracle/grid/patch/27010872/26925218
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-02-37AM_1.log

Patch: /home/oracle/grid/patch/27010872/26925311
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-02-37AM_1.log

Patch: /home/oracle/grid/patch/27010872/26983807
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-02-37AM_1.log

Patching

The patching of the first node took me around 1 hour, during which the cluster was put in ROLLING patch mode and all resources were migrated to the 2nd node:

Export correct OPatch and check the services


[root@lparaca /]#
[root@lparaca /]# export PATH=$PATH:/u01/app/oracle/grid/OPatch
[root@lparaca /]# which opatch
/u01/app/oracle/grid/OPatch/opatch
[root@lparaca /]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/oracle/product
[root@lparaca /]#
[root@lparaca /]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.DATA.dg    ora....up.type ONLINE    ONLINE    lparaca
ora....ER.lsnr ora....er.type ONLINE    ONLINE    lparaca
ora....N1.lsnr ora....er.type ONLINE    ONLINE    lparaca
ora....N2.lsnr ora....er.type ONLINE    ONLINE    lparaca
ora....N3.lsnr ora....er.type ONLINE    ONLINE    lparaca
ora.MGMTLSNR   ora....nr.type ONLINE    ONLINE    lparaca
ora.asm        ora.asm.type   ONLINE    ONLINE    lparaca
ora.cvu        ora.cvu.type   ONLINE    ONLINE    lparacb
ora....SM1.asm application    ONLINE    ONLINE    lparaca
ora....CA.lsnr application    ONLINE    ONLINE    lparaca
ora....aca.ons application    ONLINE    ONLINE    lparaca
ora....aca.vip ora....t1.type ONLINE    ONLINE    lparaca
ora....SM2.asm application    ONLINE    ONLINE    lparacb
ora....CB.lsnr application    ONLINE    ONLINE    lparacb
ora....acb.ons application    ONLINE    ONLINE    lparacb
ora....acb.vip ora....t1.type ONLINE    ONLINE    lparacb
ora.mgmtdb     ora....db.type ONLINE    ONLINE    lparaca
ora....network ora....rk.type ONLINE    ONLINE    lparaca
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    lparacb
ora.ons        ora.ons.type   ONLINE    ONLINE    lparaca
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    lparaca
ora.scan2.vip  ora....ip.type ONLINE    ONLINE    lparaca
ora.scan3.vip  ora....ip.type ONLINE    ONLINE    lparaca

Patching

[root@lparaca /]# opatchauto apply /home/oracle/grid/patch/27010872/ -oh /u01/app/oracle/grid/

OPatchauto session is initiated at Thu Jan 18 09:16:14 2018

System initialization log file is /u01/app/oracle/grid/cfgtoollogs/opatchautodb/systemconfig2018-01-18_09-16-17AM.log.

Session log file is /u01/app/oracle/grid/cfgtoollogs/opatchauto/opatchauto2018-01-18_09-16-25AM.log
The id for this session is 48G6

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/grid
Patch applicability verified successfully on home /u01/app/oracle/grid


Bringing down CRS service on home /u01/app/oracle/grid
Prepatch operation log file location: /u01/app/oracle/grid/cfgtoollogs/crsconfig/crspatch_lparaca_2018-01-18_09-19-42AM.log
CRS service brought down successfully on home /u01/app/oracle/grid


Start applying binary patch on home /u01/app/oracle/grid


Binary patch applied successfully on home /u01/app/oracle/grid


Starting CRS service on home /u01/app/oracle/grid
Postpatch operation log file location: /u01/app/oracle/grid/cfgtoollogs/crsconfig/crspatch_lparaca_2018-01-18_10-27-48AM.log
CRS service started successfully on home /u01/app/oracle/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:lparaca
CRS Home:/u01/app/oracle/grid
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/grid/patch/27010872/26910974
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-23-13AM_1.log

Patch: /home/oracle/grid/patch/27010872/26925218
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-23-13AM_1.log

Patch: /home/oracle/grid/patch/27010872/26925311
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-23-13AM_1.log

Patch: /home/oracle/grid/patch/27010872/26983807
Log: /u01/app/oracle/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-01-18_09-23-13AM_1.log



OPatchauto session completed at Thu Jan 18 10:35:34 2018
Time taken to complete the session 79 minutes, 20 seconds

Check if the patch has been applied

[oracle@lparaca ~]$ . oraenv
ORACLE_SID = [oracle] ? +ASM1
The Oracle base has been set to /u01/app/oracle/product
[oracle@lparaca ~]$ cd /u01/app/oracle/grid
[oracle@lparaca grid]$ cd OPatch
[oracle@lparaca OPatch]$ ./opatch lsinventory
Oracle Interim Patch Installer version 12.2.0.1.11
Copyright (c) 2018, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/grid
Central Inventory : /u01/app/oracle/oraInventory
   from           : /u01/app/oracle/grid/oraInst.loc
OPatch version    : 12.2.0.1.11
OUI version       : 12.1.0.2.0
Log file location : /u01/app/oracle/grid/cfgtoollogs/opatch/opatch2018-01-18_10-48-24AM_1.log

Lsinventory Output file location : /u01/app/oracle/grid/cfgtoollogs/opatch/lsinv/lsinventory2018-01-18_10-48-24AM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: lparaca
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1):

Oracle Grid Infrastructure 12c                                       12.1.0.2.0
There are 1 products installed in this Oracle Home.


Interim patches (4) :

Patch  26983807     : applied on Thu Jan 18 10:27:43 EST 2018
Unique Patch ID:  21704746
Patch description:  "WLM Patch Set Update: 12.1.0.2.180116 (26983807)"
   Created on 8 Nov 2017, 07:59:12 hrs PST8PDT
   Bugs fixed:
     20075154, 20641027, 22271856, 20548410, 19016964, 19582630

Patch  26925311     : applied on Thu Jan 18 10:27:03 EST 2018
Unique Patch ID:  21850549
Patch description:  "Database Patch Set Update : 12.1.0.2.180116 (26925311)"
   Created on 6 Dec 2017, 01:18:05 hrs PST8PDT
Sub-patch  26713565; "Database Patch Set Update : 12.1.0.2.171017 (26713565)"
Sub-patch  26609783; "Database Patch Set Update : 12.1.0.2.170814 (26609783)"
Sub-patch  25755742; "Database Patch Set Update : 12.1.0.2.170718 (25755742)"
Sub-patch  25171037; "Database Patch Set Update : 12.1.0.2.170418 (25171037)"
Sub-patch  24732082; "Database Patch Set Update : 12.1.0.2.170117 (24732082)"
Sub-patch  24006101; "Database Patch Set Update : 12.1.0.2.161018 (24006101)"
Sub-patch  23054246; "Database Patch Set Update : 12.1.0.2.160719 (23054246)"
Sub-patch  22291127; "Database Patch Set Update : 12.1.0.2.160419 (22291127)"
Sub-patch  21948354; "Database Patch Set Update : 12.1.0.2.160119 (21948354)"
Sub-patch  21359755; "Database Patch Set Update : 12.1.0.2.5 (21359755)"
Sub-patch  20831110; "Database Patch Set Update : 12.1.0.2.4 (20831110)"
Sub-patch  20299023; "Database Patch Set Update : 12.1.0.2.3 (20299023)"
Sub-patch  19769480; "Database Patch Set Update : 12.1.0.2.2 (19769480)"
   Bugs fixed:
     19309466, 24570598, 25475853, 21172913, 19902195, 18250893, 17655240
     19383839, 21266085, 19028800, 19035573, 16756406, 19366375, 18456643
     26546664, 24523374, 25034396, 19289642, 18845653, 19915271, 21291274
     18007682, 20172151, 18417036, 23713236, 23521523, 24796092, 20475845
     22148226, 22528741, 19243521, 19658708, 21153266, 24652769, 26088426
     19326908, 19597583, 17414008, 20897759, 19174430, 22046677, 22243719
     20938170, 24825843, 21960504, 24509056, 19054077, 22657942, 20688221
     20428621, 21899588, 21387964, 13542050, 19723336, 19835133, 17532734
     19333670, 21842017, 19285025, 21373473, 23260854, 19687159, 14643995
     21623164, 20977794, 20734332, 19012119, 19869255, 17551063, 18681056
     21977392, 23324000, 24461826, 19676012, 20588502, 25427662, 22068305
     23315889, 19520602, 23053606, 19841800, 19439759, 20245930, 19303936
     19001359, 21476308, 26546754, 22916353, 19393542, 23533524, 21099555
     24835538, 22353346, 25429959, 19141838, 19644859, 21106027, 21915719
     26444887, 23088803, 19908836, 21421886, 22529728, 19358317, 19134173
     19524158, 20447445, 23548817, 25861398, 20803014, 23025340, 21188584
     19335438, 19390567, 19058490, 19207117, 18799993, 26569225, 20835241
     24662775, 19769480, 19475971, 21097043, 21225209, 20677396, 19284031
     19450314, 19016730, 18967382, 20919320, 22075064, 20347562, 20348653
     22551446, 19896336, 24812585, 22721409, 20048359, 21896069, 18440095
     22496904, 19524384, 16439813, 18354830, 20596234, 20440930, 22022760
     20936905, 19171086, 23197103, 17867700, 19791273, 24718260, 21514877
     26111842, 18990023, 21241829, 19591608, 22707244, 18419520, 22492533

Check the patching mode of the Grid

You can also check the state of the Grid Infra with the following command:

[oracle@lparaca OPatch]$ crsctl query crs softwareversion
Oracle Clusterware version on node [lparaca] is [12.1.0.2.0]
[oracle@lparaca OPatch]$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [12.1.0.2.0]
[oracle@lparaca OPatch]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [ROLLING PATCH]. The cluster active patch level is [0].
[oracle@lparaca OPatch]$