Linux not picking up new partition correctly on emc pseudo device

clariionemclinux

We have a database server running oracle rac. We were recently running out of space on the main LUN that it is attached to. I created a new 100GB LUN and concatenated this onto the existing LUN creating a new MetaLUN. After some messing I managed to get linux to recognise the new space. I then created a new partition in on the pseudo device, to use the new space. Previously when I have done this on other system the next step is to create an ASM disk on the new partition and add this disk to the oracle disk group. This however fails. I am aware of various issues with ASM and powerpath, but I don't think this is the issue here. As on while investigating the issue I discovered that one of the underlying logical device is not reflecting the size change. See below;

Powermt displays all of the underlying logical units

[root@XXXXX~]# powermt display dev=emcpowerd  
Pseudo name=emcpowerd  
CLARiiON ID=CKM00091500009 [VFRAC2]  
Logical device ID=6006016030312200787502866C65DE11 [LUN 30]  
state=alive; policy=CLAROpt; priority=0; queued-IOs=0  

Owner: default=SP A, current=SP A       Array failover mode: 1  
`==============================================================================`
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---  
`###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors`
`==============================================================================`
   3 qla2xxx                   sde       SP A0     active  alive      0      0  
   3 qla2xxx                   sdj       SP B0     active  alive      0      0  
   4 qla2xxx                   sdo       SP A1     active  alive      0      0  
   4 qla2xxx                   sdt       SP B1     active  alive      0      0  

**Fdisk on the pseudo device shows correct space.**

[root@XXXXX ~]# fdisk -l /dev/emcpowerd  

Disk /dev/emcpowerd: 429.4 GB, 429496729600 bytes  
255 heads, 63 sectors/track, 52216 cylinders  
Units = cylinders of 16065 * 512 = 8225280 bytes  

         Device Boot      Start         End      Blocks   Id  System  
/dev/emcpowerd1               1       39162   314568733+  83  Linux  
/dev/emcpowerd2           39163       52216   104856255   83  Linux  

**fdisk on one of the logical units is wrong**

[root@XXXXX~]# fdisk -l /dev/sde  

Disk /dev/sde: 322.1 GB, 322122547200 bytes    
255 heads, 63 sectors/track, 39162 cylinders  
Units = cylinders of 16065 * 512 = 8225280 bytes  

   Device Boot      Start         End      Blocks   Id  System  
/dev/sde1               1       39162   314568733+  83  Linux  
/dev/sde2           39163       52216   104856255   83  Linux  

**fdisk on the rest of the units is fine**

[root@XXXXX ~]# fdisk -l /dev/sdj  
Disk /dev/sdj: 429.4 GB, 429496729600 bytes  
255 heads, 63 sectors/track, 52216 cylinders  
Units = cylinders of 16065 * 512 = 8225280 bytes  
   Device Boot      Start         End      Blocks   Id  System  
/dev/sdj1               1       39162   314568733+  83  Linux  
/dev/sdj2           39163       52216   104856255   83  Linux

Also when I created the the partition linux did not create the any entries in the /dev directory for the second partition so I created these manually

[root@XXXXX dev]# mknod sde2 b 8 66
[root@XXXXX dev]# ls -al sd[ejot]?  
brw-r----- 1 root disk  8,  65 Dec 29 14:20 sde1  
brw-r--r-- 1 root disk  8,  66 Apr  8 20:31 sde2  
brw-r----- 1 root disk  8, 145 Dec 29 14:19 sdj1  
brw-r--r-- 1 root disk  8, 146 Apr  8 20:33 sdj2  
brw-r----- 1 root disk  8, 225 Apr  6 23:12 sdo1  
brw-r--r-- 1 root disk  8, 226 Apr  8 20:33 sdo2  
brw-r----- 1 root disk 65,  49 Dec 29 14:19 sdt1  
brw-r--r-- 1 root disk 65,  50 Apr  8 20:33 sdt2

This is a production server that we cannot easily reboot.

Any ideas would be much appreciated.

J

Best Answer

Apart from partprobe, try using the blockdev utility to reread partition table of the device:

blockdev --rereadpt /dev/sde

Then, the problem may be that the LUN itself hasn't been properly updated.

You can try issuing a rescan command against the Fibre Channel or SCSI host through the /sys filesystem.

Some time ago, I've written this scsi_rescan_bus.sh script for coping with our EMC Clariion devices:

#!/bin/sh
host_number="$1"
echo "1" > /sys/class/fc_host/host${host_number}/issue_lip
sleep 10
echo "- - -" > /sys/class/scsi_host/host${host_number}/scan

I'm not entirely sure that it would still work with the modern kernels and devices. Always test this on a dedicated test environment before trying this in production!

There are numerous gotchas, so make sure you read these relevant threads:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1454807

And the official Red Hat documentation ("Online Storage Reconfiguration Guide"): http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/index.html

Related Topic