Friday, October 23, 2015

How To Setup ASM & ASMLIB On Native Linux Multipath Mapper disks

This note apply to Oracle Database using ASM and linux S.O.

After installed and configured the Multipath, follow bellow:

2. Check the disks :
ls -l /dev/mapper
brw-rw---- 1 root disk 253, 46 Jul 27 17:19 Prod_Orcl01
brw-rw---- 1 root disk 253, 67 Jul 27 17:59 Prod_Orcl02
brw-rw---- 1 root disk 253, 36 Jul 27 17:19 Prod_Orcl03
brw-rw---- 1 root disk 253, 58 Jul 27 18:00 Prod_Orcl04

3. Create the ASMLIB disks on mapper partitions as follow:
/etc/init.d/oracleasm createdisk DSKORA1  /dev/mapper/Prod_Orcl01
/etc/init.d/oracleasm createdisk DSKORA2  /dev/mapper/Prod_Orcl02
After create the disks, the ASM put a mark on the disks to know which are your own.

4. If this is a RAC configuration, then from each node execute:
/etc/init.d/oracleasm scandisks
/etc/init.d/oracleasm listdisks

5. Configure ASMLIB to use multipath  (from each node on RAC environments):
By any path the ASMLIB can found the disks, but, the best path is using the multipath :
Modify in /etc/sysconfig/oracleasm :
ORACLEASM_SCANORDER=”dm”
ORACLEASM_SCANEXCLUDE=”sd”
note: The Oracle ASMLib configuration file is located at /etc/sysconfig/oracleasm. It is a link to file /etc/sysconfig/oracleasm-_dev_oracleasm.
Restart ASMLIB (from each node on RAC environments):
/etc/init.d/oracleasm stop
/etc/init.d/oracleasm start

6. Verify if the configuration is correct:
6.1 During the disk discovery, ASMLIB uses file /proc/partitions, see :
# cat /proc/partitions
   8     0  877264896 sda
   8     1     104391 sda1
   8     2  877157032 sda2
   8    16  209715200 sdb
   8    32  382730240 sdc
   8    48  379596800 sdd
   8    64    2097152 sde
   8    80    2097152 sdf
   8    96 1169776640 sdg
 253     6  209715200 dm-6
 253     7  382730240 dm-7
 253     8  379596800 dm-8
 253     9    2097152 dm-9
 253    10    2097152 dm-10



6.2 The ASMLIB mount disks at /dev/oracleasm/disks, see:


# ls -la /dev/oracleasm/disks


brw-rw---- 1 grid asmadmin 253,  6 Ago 16 16:33 DSKORA1
brw-rw---- 1 grid asmadmin 253,  7 Ago 16 16:33 DSKORA2





6.3 Check if major and minor of disks “dm” not “sd”, is the same in /proc/partitions  and /dev/oracleasm/disks , see “253″ and “6″ bellow:


/proc/partitions 253     6  209715200 dm-6


/dev/oracleasm/disks
brw-rw---- 1 grid asmadmin 253,  6 Ago 16 16:33 DSKORA1



You can check using querydisk too :


# /etc/init.d/oracleasm querydisk -d DSKORA1
Disk "DSKORA1" is valid ASM disk on device [253, 6]












Posted by



at





No comments:
  


































          

        

          

        

Friday, October 2, 2015


          

        









ASMLib and EMC PowerPath Multipathing














Problem - ASMLib picks the first disk device it finds for a particular ASM disk and uses that device. This usually means that ASM is NOT using the multipath emcpower device.











ASMLib picks the first disk device it finds for a particular ASM disk and uses that device. This usually means that ASM is NOT using the multipath emcpower device. 
Let's pick an ASMLib disk on my server - in this case I choose a disk that has been named  ORA_FPF_90GB_EMCP022 .
root@fpfkxtdb02 ~# /etc/init.d/oracleasm  querydisk ORA_FPF_90GB_EMCP022 
Disk "ORA_FPF_90GB_EMCP022" is a valid ASM disk 
There are lots of possible paths to the disk. ASM picks the first one it finds and ignores the rest. What paths are there for our disk?? 
root@fpfkxtdb02 ~# /etc/init.d/oracleasm querydisk -p ORA_FPF_90GB_EMCP022 
Disk "ORA_FPF_90GB_EMCP022" is a valid ASM disk 
/dev/sdn1: LABEL="ORA_FPF_90GB_EMCP022" TYPE="oracleasm" 
/dev/sdav1: LABEL="ORA_FPF_90GB_EMCP022" TYPE="oracleasm" 
/dev/sdcd1: LABEL="ORA_FPF_90GB_EMCP022" TYPE="oracleasm" 
/dev/sddk1: LABEL="ORA_FPF_90GB_EMCP022" TYPE="oracleasm" 
/dev/emcpowerm1: LABEL="ORA_FPF_90GB_EMCP022" TYPE="oracleasm" 
Lots of paths.







Lets see what device path is actually being used for our ASMLib disk: 
root@fpfkxtdb02 ~# /etc/init.d/oracleasm querydisk -d ORA_FPF_90GB_EMCP022 
Disk "ORA_FPF_90GB_EMCP022" is a valid ASM disk on device [8, 209] 
Device [8, 209]  is the first disk in the path list - /dev/sdn1 - NOT the emcpower device. 
root@fpfkxtdb02 ~# ls -l /dev/sdn1 
brw-r----- 1 root disk 8, 209 Oct 24 12:15 /dev/sdn1 
We want ASMLib to be using THIS device: 
root@fpfkxtdb02 ~# ls -l /dev/emcpowerm1 
brw-r----- 1 root disk 120, 193 Oct 24 12:15 /dev/emcpowerm1 
HOW TO FIX IT SO THAT THE EMCPOWER DEVICE IS THE DEVICE THAT ASMLIB USES




Edit ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE!

root@fpfkxtdb02 ~# vim /etc/sysconfig/oracleasm 
Change

# ORACLEASM_SCANORDER: Matching patterns to order disk scanning 
ORACLEASM_SCANORDER="" 
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan 
ORACLEASM_SCANEXCLUDE="" 
TO

# ORACLEASM_SCANORDER: Matching patterns to order disk scanning 
ORACLEASM_SCANORDER="emcpower" 
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan 
ORACLEASM_SCANEXCLUDE="sd dm" 

Unload, reload ASM and scandisks
root@fpfkxtdb02 ~# oracleasm exit 
Unmounting ASMlib driver filesystem: /dev/oracleasm 
Unloading module "oracleasm": oracleasm 
root@fpfkxtdb02 ~# oracleasm init 
Loading module "oracleasm": oracleasm 
Mounting ASMlib driver filesystem: /dev/oracleasm 
root@fpfkxtdb02 ~# /etc/init.d/oracleasm scandisks 
Scanning the system for Oracle ASMLib disks:               [  OK  ] 

Check that ASM is now using the emcpower device:

root@fpfkxtdb02 ~# /etc/init.d/oracleasm querydisk -d ORA_FPF_90GB_EMCP022 
Disk "ORA_FPF_90GB_EMCP022" is a valid ASM disk on device [120, 193] 







Done







Source: http://www.oracletech.com.au/2012/02/asmlib-and-emc-powerpath-multipathing.html












Posted by



at





No comments:
  










































Mapping ASM disks to Physical Devices














Sometimes you may need to map ASM Disks to its physical devices. 





If they are based on ASMLib you will see their ASM name, ie: ORCL:VOL1 when querying v$asm_disk

When running oracleasm querydisk VOL1 you will get in addition the major - minor numbers, that can be used to match the physical device, ie:



[root@orcldb2 ~]# /etc/init.d/oracleasm querydisk VOL1
Disk "VOL1" is a valid ASM disk on device [8, 97]

[root@orcldb2 ~]# ls -l /dev | grep 8, | grep 97
brw-rw----   1 root disk     8,      81 Nov  4 13:02 sdg1

This script can do the job for a group of ASM Disks:


---------- start  here ------------
#!/bin/ksh
for i in `/etc/init.d/oracleasm listdisks`
do
v_asmdisk=`/etc/init.d/oracleasm querydisk $i | awk  '{print $2}'`
v_minor=`/etc/init.d/oracleasm querydisk $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk '{print $1}'`
v_major=`/etc/init.d/oracleasm querydisk $i | awk -F[ '{print $2}'| awk -F] '{print $1}' | awk '{print $2}'`
v_device=`ls -la /dev | grep $v_minor | grep $v_major | awk '{print $10}'`
echo "ASM disk $v_asmdisk based on /dev/$v_device  [$v_minor $v_major]"
done
---------- finish here ------------


The output looks like this:


ASM disk "VOL1" based on /dev/sdg1  [8, 97]
ASM disk "VOL10" based on /dev/sdp1  [8, 241]
ASM disk "VOL2" based on /dev/sdh1  [8, 113]
ASM disk "VOL3" based on /dev/sdk1  [8, 161]
ASM disk "VOL4" based on /dev/sdi1  [8, 129]
ASM disk "VOL5" based on /dev/sdl1  [8, 177]
ASM disk "VOL6" based on /dev/sdj1  [8, 145]
ASM disk "VOL7" based on /dev/sdn1  [8, 209]
ASM disk "VOL8" based on /dev/sdo1  [8, 225]
ASM disk "VOL9" based on /dev/sdm1  [8, 193]


If you are using multi-path, you will need an additional step to map the physical device to the multi-path device, for instance if using EMC Powerpath if you want to map sdf1


[root@orclp ~]# /etc/init.d/oracleasm querydisk vol1
Disk "VOL1" is a valid ASM disk on device [8, 81]

[root@orclp ~]# ls -l /dev | grep 8,| grep 81
brw-rw----   1 root disk     8,      81 Oct 29 20:42 sdf1

[root@orclp ~]# powermt display dev=all
...
...
Pseudo name=emcpowerg
Symmetrix ID=000290101698
Logical device ID=0214
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
### HW Path                 I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sdf       FA  7bB   active  alive      0      0
   2 qla2xxx                   sdq       FA 10bB   active  alive      0      0
...
...





The last step is to check the partition assigned to the emcpower device, ie:


[root@orclp ~]# ls -l /dev/emcpowerg*
brw-------  1 root root 120, 96 Oct 29 20:41 /dev/emcpowerg
brw-------  1 root root 120, 97 Nov 15 13:08 /dev/emcpowerg1







Source: https://blogs.oracle.com/AlejandroVargas/entry/mapping_asm_disks_to_physical














Posted by



at





No comments:
  


































        

      









Subscribe to:
Posts (Atom)