Overview

With 12c, restore and recover of the voting disks and the OCR is quite simple since Oracle remembers the exact physical place of the OCR and voting disks on the ASM.

This scenario assumes:

The ASM Spfile is on separate diskgroup than the voting disks(Check “Restore ASM” if they are on the same)

The voting disk is on NORMAL redundancy and has 3 disks only.

In a Nutshell

In nutshell, the procedure is:

Corrupt the data. (P.S. don't do that :D This is only to show you what happens IF it is corrupted)

Stop CRS on ALL Nodes

Start CRS exclusively on one node

Restore OCR

Restore voting disks

Corrupt the Voting Disks

Corruption is done via the “dd” command:

Corrupted Disk

You can check the corruption by checking the header of the corrupted disk and a normal disk:

Normal Disk

You can compare this result with:

Shutdown CRS

CRS should be brought down to all instances in the cluster, feel free to use the -f option in case of errors:

Start CRS stack Exclusively

Start the CRS on ONLY ONE node exclusively and without CRS:

Restore the OCR

OCR is restore as follows:

Check the voting disks

Drop and Recreate Diskgroup

The diskgroup should be re-created OR the voting disks should be relocated to other diskgroup, whichever you choose. Since we have to re-establish our connection to the corrupted disk, I preferred to re-create the disk group:

Delete the voting disks from the corrupter diskgroup

Re-create the ASM Diskgroup

Restore of the Voting Disks

Restore of the voting disks is also simple:

Shutdown CRS

Start CRS

Start CRS again normally on all nodes:

Restore ASM

Restore the ASM Instance

Since the ASM parameter file should reside in ASM, it can happen that the ASM should be restored as well. However in such case , both the ASM Spfile and the password file should be restored as follows:

Startup with pfile

Re-create the spfile

ASM Password File

The ASM Password file can really be pain in the….arm. In order to solve teh ASM password there are couple problems which you should address: (e.g. MOS: 2139591.1) In a nutshell, you should:

Create new user

Grant him SYSDBA and SYSASM

Modify the hash password ← This step was a real nightmare, you will see later why.

Create new user

The creation of a new user and new password file is fairly easy:

First be sure that all password files are deleted

Create a new password file either via:

or

Create a new user with SYSDBA and SYSASM Roles

Extract the hash password

This step was pain in the….arm because I was missing the PATH to the credential domain:

So to solve that, I assumed that the path displayed in the trace file of the 2nd node is correct and just run with it.

Tracefile (2nd Node)

Extract the cdentials

Change the password using the hash

Restart the CRS on the 2nd node

Check