Overview
Now, disk in ASM diskgroup CAN get corrupted and depends on the way it was corrupted, MAYBE it can be fixed. From my experience there are couple ways how this can happen:
- OS Overlapped with the ASM disk
- Manual Corruption
- Oracle Bug
So let's see what we can do:
Oracle Bug
Now the disk can get corrupted for a lot of reasons and that's why it is always good to have NORMAL or HIGH redundancy. In either case, let's see what happens if we can fix the corruption:
Check the header of the disk
[root@oracrs ~]# /u01/app/oracle/product/11.3.0/grid/bin/kfod status=TRUE asm_diskstring='/dev/oradev' disk=ALL
--------------------------------------------------------------------------------
Disk Size Header Path User Group
================================================================================
1: 10236 Mb MEMBER /dev/oradev/ora-data1 oracle dba
2: 25595 Mb CANDIDATE /dev/oradev/ora-data10 oracle dba
3: 10236 Mb MEMBER /dev/oradev/ora-data2 oracle dba
4: 10236 Mb MEMBER /dev/oradev/ora-data3 oracle dba
5: 10236 Mb MEMBER /dev/oradev/ora-data4 oracle dba
6: 10236 Mb MEMBER /dev/oradev/ora-data5 oracle dba
7: 10236 Mb CANDIDATE /dev/oradev/ora-data6 oracle dba
8: 51199 Mb MEMBER /dev/oradev/ora-data7 oracle dba
9: 25595 Mb CANDIDATE /dev/oradev/ora-data8 oracle dba
10: 25595 Mb MEMBER /dev/oradev/ora-data9 oracle dba
11: 10236 Mb MEMBER /dev/oradev/ora-fra1 oracle dba
12: 10236 Mb MEMBER /dev/oradev/ora-fra2 oracle dba
13: 10236 Mb MEMBER /dev/oradev/ora-fra3 oracle dba
14: 10236 Mb MEMBER /dev/oradev/ora-fra4 oracle dba
15: 10236 Mb MEMBER /dev/oradev/ora-fra5 oracle dba
16: 10236 Mb CANDIDATE /dev/oradev/ora-fra6 oracle dba <- Corrupted, it shows is a candidate
17: 5114 Mb MEMBER /dev/oradev/ora-fra7 oracle dba
18: 25595 Mb MEMBER /dev/oradev/ora-fra8 oracle dba
19: 25595 Mb CANDIDATE /dev/oradev/ora-fra9 oracle dba
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
================================================================================
+ASM /u01/app/oracle/product/11.3.0/grid
Disk number “fra6” is a corrupted one, even though it shows it as a candidate. This can be seen from:
Corrupted header
[root@oracrs ~]# /u01/app/oracle/product/11.3.0/grid/bin/kfed read /dev/oradev/ora-fra6 aun=0 blkn=0
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 0 ; 0x001: 0x00
kfbh.type: 0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt: 0 ; 0x003: 0x00
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 0 ; 0x008: file=0
kfbh.check: 0 ; 0x00c: 0x00000000
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
2B2530EBB400 00000000 00000000 00000000 00000000 [................]
Repeat 31 times
2B2530EBB600 4542414C 454E4F4C 00000001 00000000 [LABELONE........]
2B2530EBB610 5133D14F 00000020 324D564C 31303020 [O.3Q ...LVM2 001]
2B2530EBB620 3336334E 314A4643 49585056 63396C5A [N363CFJ1VPXIZl9c]
2B2530EBB630 576B484A 306C6455 4C4B6F52 58567A33 [JHkWUdl0RoKL3zVX]
2B2530EBB640 7FCB3400 00000002 00030000 00000000 [.4..............]
2B2530EBB650 00000000 00000000 00000000 00000000 [................]
2B2530EBB660 00000000 00000000 00001000 00000000 [................]
2B2530EBB670 0002F000 00000000 00000000 00000000 [................]
2B2530EBB680 00000000 00000000 00000000 00000000 [................]
Repeat 215 times
That can be repaired as follows:
Repair not 0 AU corruption in ASM Disk
[root@oracrs ~]# /u01/app/oracle/product/11.3.0/grid/bin/kfed repair /dev/oradev/ora-fra6 aun=0 blkn=0 [root@oracrs ~]# /u01/app/oracle/product/11.3.0/grid/bin/kfed read /dev/oradev/ora-fra6 aun=0 blkn=0 kfbh.endian: 1 ; 0x000: 0x01 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD kfbh.datfmt: 1 ; 0x003: 0x01 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 2147483653 ; 0x008: disk=5 kfbh.check: 2061566458 ; 0x00c: 0x7ae101fa kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8 kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000 kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
In the summary, now this disk appears as a member:
ASM Disk Summary
[root@oracrs ~]# /u01/app/oracle/product/11.3.0/grid/bin/kfod status=TRUE asm_diskstring='/dev/oradev' disk=ALL
--------------------------------------------------------------------------------
Disk Size Header Path User Group
================================================================================
1: 10236 Mb MEMBER /dev/oradev/ora-data1 oracle dba
2: 25595 Mb CANDIDATE /dev/oradev/ora-data10 oracle dba
3: 10236 Mb MEMBER /dev/oradev/ora-data2 oracle dba
4: 10236 Mb MEMBER /dev/oradev/ora-data3 oracle dba
5: 10236 Mb MEMBER /dev/oradev/ora-data4 oracle dba
6: 10236 Mb MEMBER /dev/oradev/ora-data5 oracle dba
7: 10236 Mb CANDIDATE /dev/oradev/ora-data6 oracle dba
8: 51199 Mb MEMBER /dev/oradev/ora-data7 oracle dba
9: 25595 Mb CANDIDATE /dev/oradev/ora-data8 oracle dba
10: 25595 Mb MEMBER /dev/oradev/ora-data9 oracle dba
11: 10236 Mb MEMBER /dev/oradev/ora-fra1 oracle dba
12: 10236 Mb MEMBER /dev/oradev/ora-fra2 oracle dba
13: 10236 Mb MEMBER /dev/oradev/ora-fra3 oracle dba
14: 10236 Mb MEMBER /dev/oradev/ora-fra4 oracle dba
15: 10236 Mb MEMBER /dev/oradev/ora-fra5 oracle dba
16: 10236 Mb MEMBER /dev/oradev/ora-fra6 oracle dba <- Now a member
17: 5114 Mb MEMBER /dev/oradev/ora-fra7 oracle dba
18: 25595 Mb MEMBER /dev/oradev/ora-fra8 oracle dba
19: 25595 Mb CANDIDATE /dev/oradev/ora-fra9 oracle dba
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
================================================================================
+ASM /u01/app/oracle/product/11.3.0/grid