Oracle RAC Blog

While it seems simple to add and drop the disk online in an ASM environment, often, we encounter very many errors based on the cluster software, OS and the way the SAN is visible to cluster. Before going into the detail, the error and work around let me briefly touch base on the general ASM operation specifically when altering the disk group for those who are new to ASM (automatic storage management).

We can use the ALTER DISKGROUP statement to alter a ASMdisk group configuration. As we know, we can add, resize, or drop disks while the database remains online. Whenever possible, multiple operations in a single ALTER DISKGROUP statement are recommended.

ASM automatically rebalances when the configuration of a disk group changes. By default, the ALTER DISKGROUP statement does not wait until the operation is complete before returning. Query the V$ASM_OPERATION view to monitor the status of this operation.

We can use the REBALANCE WAIT clause if you want the ALTER DISKGROUP statement processing to wait until the rebalance operation is complete before returning. This is especially useful in scripts. The statement also accepts a REBALANCE NOWAIT clause that invokes the default behavior of conducting the rebalance operation asynchronously in the background.

Having familiarized with the syntaxes and concepts, lets see the below command and error. As a part of migrating the ASM to a new SAN storage and would like to add the new disks to the existing the ASM disk group and plan to drop the old disks. The operation can be done online in oracle 11G R1 onwards; we plan to do this offline as the environment is 10204.

In the ASM instance:
==============

alter diskgroup RAWDG01 add disk ‘/dev/did/rdsk/d21s6′;

ERROR at line 1:

ORA-15032: not all alterations performed

ORA-15031: disk specification ‘/dev/did/rdsk/d21s6′ matches no disks

ORA-15014: location ‘/dev/did/rdsk/d21s6′ is not in the discovery set

As per the metalink note above, the recommendation is to use ‘dd’ command at OS level to cleanse the prior contents or use FORCE.

Since we are using the new disks, there is no need to use dd and the following force command did not work for us.

With FORCE option:
====================

alter diskgroup RAWDG01 add disk ‘/dev/did/rdsk/d21s6′ force

ORA-15032: not all alterations performed

ORA-15031: disk specification ‘/dev/did/rdsk/d22s6′ matches no disks

ORA-15014: location ‘/dev/did/rdsk/d22s6′ is not in the discovery set

SQL> CREATE DISKGROUP RAWDG03 disk ‘/dev/did/rdsk/d22s6′ force;

CREATE DISKGROUP RAWDG03 disk ‘/dev/did/rdsk/d22s6′ force

*

ERROR at line 1:

ORA-15018: diskgroup cannot be created

ORA-15031: disk specification ‘/dev/did/rdsk/d22s6′ matches no disks

ORA-15014: location ‘/dev/did/rdsk/d22s6′ is not in the discovery set
After going through the hundreds of pages of ASM manuals, we come across a relevant parameter asm_dsikstring that is empty in our environment.

SQL> show parameter asm_diskstring;

NAME                                 TYPE        VALUE

———————————— ———– ——————————

asm_diskstring                       string

SQL> alter system set asm_diskstring = ‘/dev/rdsk*’;

alter system set asm_diskstring = ‘/dev/rdsk*’

*

ERROR at line 1:

ORA-02097: parameter cannot be modified because specified value is invalid

ORA-15014: location ‘/dev/rdsk/c4t60060160829012004CBE135E05ADDE11d0s0′ is not

in the discovery set

ORA-15025: could not open disk ‘/dev/rdsk’

ORA-15056: additional error message

SVR4 Error: 13: Permission denied

Additional information: 42

Additional information: 109884796

Additional information: 107607288

SQL>  alter system set asm_diskstring = ‘/dev/rdsk/*s0′ SID=’*';

System altered.

SQL> show parameter asm

NAME                                 TYPE        VALUE

———————————— ———– ——————————

asm_diskgroups                       string      RAWDG01, RAWDG02

asm_diskstring                       string      /dev/rdsk/*s0

asm_power_limit                      integer     1

SQL> alter diskgroup RAWDG01 add disk ‘/dev/did/rdsk/d21s6′;

Diskgroup altered.

SQL> select substr(name,1,10) name,substr(path,1,20) path, REDUNDANCY, TOTAL_MB,  free_mb from V$ASM_DISK

2  /

NAME       PATH                 REDUNDA   TOTAL_MB    FREE_MB

———- ——————– ——- ———- ———-

/dev/rdsk/c4t6006016 UNKNOWN        511          0

/dev/rdsk/c4t6006016 UNKNOWN        511          0

/dev/rdsk/c4t600144F UNKNOWN        511          0

/dev/rdsk/c4t600144F UNKNOWN        511          0

/dev/rdsk/c4t600144F UNKNOWN     204699          0

/dev/rdsk/c4t600144F UNKNOWN     102286          0

/dev/rdsk/c4t600144F UNKNOWN        519          0

/dev/did/rdsk/d23s6  UNKNOWN        519          0

/dev/did/rdsk/d22s6  UNKNOWN     102286          0

/dev/did/rdsk/d19s6  UNKNOWN        511          0

/dev/did/rdsk/d20s6  UNKNOWN        511          0

NAME       PATH                 REDUNDA   TOTAL_MB    FREE_MB

———- ——————– ——- ———- ———-

RAWDG02_00 /dev/rdsk/c4t6006016 UNKNOWN     204788     141340

RAWDG01_00 /dev/rdsk/c4t6006016 UNKNOWN     102394      11609

RAWDG01_00 /dev/did/rdsk/d21s6  UNKNOWN     204699     194393

In conclusion, we are able to add the new disks to the existing disk group after setting up the asm_distring on both the nodes. The fact that prior disk groups are being added during installation and configuration of ASM indicates that, perhaps we do not need to set this parameter when doing via cluster install or dbca. However, it is imperative from the current experience that asm_diskstring must be specified for managing the ASM disk groups.

Hope this helps to folks working on the ASM/RAC environment.