aprince1

updated

17 years ago

A

aprince1

7 Posts

0

9297

April 8th, 2010 10:00

MPxIO and Clariion CX4-480

Hi

We have encountered a pathing issue with Solaris MPxIO and a clariion CX4-480. Here are some details regarding our config:

CX4-480
Initiator type:         CLARiiON Open
HBA type:              Host
Array CommPath:    Enabled
Failover Mode:        1
Unit Serial:            Array

Connection

Initiator    SP    Type    Logged    Registered               
1             B-0    Fibre    Yes    Yes   
2             A-3    Fibre    Yes    Yes   
3             A-0    Fibre    Yes    Yes   
4             B-3    Fibre    Yes    Yes

Luns

Name                      ID    Capacity       Type    Hostid
hostname /oraredo    47    40.000 GB    FC           0
hostname /oracrm     80    50.000 GB    FC           1
hostname /orabiz      81    50.000 GB    FC           2
hostname /orasoft     92    100.000 GB    SATAII    3
hostname /oraarch    97    150.000 GB    FC          4
hostname /orafra       96    200.000 GB    FC         5

Format output from solaris

Searching for disks...

The current rpm value 0 is invalid, adjusting it to 3600

The current rpm value 0 is invalid, adjusting it to 3600

done

c1t5006016B3CE00FACd0: configured with capacity of 40.00GB
c1t500601603CE00FACd0: configured with capacity of 39.98GB
c2t500601633CE00FACd0: configured with capacity of 39.98GB
c2t500601683CE00FACd0: configured with capacity of 40.00GB
c3t60060160294122002A2ADDB296EADE11d0: configured with capacity of 150.00GB
c3t6006016029412200C28485A496EADE11d0: configured with capacity of 199.99GB
c3t60060160294122003CD924D692EADE11d0: configured with capacity of 50.00GB
c3t6006016029412200D219342394EADE11d0: configured with capacity of 100.00GB
c3t60060160294122003DD924D692EADE11d0: configured with capacity of 50.00GB


AVAILABLE DISK SELECTIONS:
       0. c1t5006016B3CE00FACd0
          /pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w5006016b3ce00fac,0
       1. c1t500601603CE00FACd0
          /pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w500601603ce00fac,0
       2. c2t500601633CE00FACd0
          /pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601633ce00fac,0
       3. c2t500601683CE00FACd0
          /pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601683ce00fac,0
       4. c3t5000C5000A6B4BF7d0
          /scsi_vhci/disk@g5000c5000a6b4bf7
       5. c3t5000C5000A6B4D97d0
          /scsi_vhci/disk@g5000c5000a6b4d97
       6. c3t60060160294122002A2ADDB296EADE11d0
          /scsi_vhci/ssd@g60060160294122002a2addb296eade11
       7. c3t6006016029412200C28485A496EADE11d0
          /scsi_vhci/ssd@g6006016029412200c28485a496eade11
       8. c3t60060160294122003CD924D692EADE11d0
          /scsi_vhci/ssd@g60060160294122003cd924d692eade11
       9. c3t6006016029412200D219342394EADE11d0
          /scsi_vhci/ssd@g6006016029412200d219342394eade11
      10. c3t60060160294122003DD924D692EADE11d0
          /scsi_vhci/ssd@g60060160294122003dd924d692eade11

luxadm
Found Fibre Channel device(s):

  Node WWN:50060160bce00fac  Device Type:Disk device

    Logical Path:/dev/rdsk/c1t5006016B3CE00FACd0s2

    Physical Path:

     /devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w5006016b3ce00fac,0:c,raw

    Logical Path:/dev/rdsk/c1t500601603CE00FACd0s2

    Physical Path:

     /devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w500601603ce00fac,0:c,raw

    Logical Path:/dev/rdsk/c2t500601633CE00FACd0s2

    Physical Path:

     /devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601633ce00fac,0:c,raw

    Logical Path:/dev/rdsk/c2t500601683CE00FACd0s2

    Physical Path:

     /devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601683ce00fac,0:c,raw

  Node WWN:50060160bce00fac  Device Type:Disk device
    Logical Path:/dev/rdsk/c3t60060160294122002A2ADDB296EADE11d0s2
    Physical Path:
     /devices/scsi_vhci/ssd@g60060160294122002a2addb296eade11:c,raw

  Node WWN:50060160bce00fac  Device Type:Disk device
    Logical Path:/dev/rdsk/c3t6006016029412200C28485A496EADE11d0s2
    Physical Path:
     /devices/scsi_vhci/ssd@g6006016029412200c28485a496eade11:c,raw

  Node WWN:50060160bce00fac  Device Type:Disk device
    Logical Path:/dev/rdsk/c3t60060160294122003CD924D692EADE11d0s2
    Physical Path:
     /devices/scsi_vhci/ssd@g60060160294122003cd924d692eade11:c,raw

  Node WWN:50060160bce00fac  Device Type:Disk device
    Logical Path:/dev/rdsk/c3t6006016029412200D219342394EADE11d0s2
    Physical Path:
     /devices/scsi_vhci/ssd@g6006016029412200d219342394eade11:c,raw

  Node WWN:50060160bce00fac  Device Type:Disk device
    Logical Path:/dev/rdsk/c3t60060160294122003DD924D692EADE11d0s2
    Physical Path:
     /devices/scsi_vhci/ssd@g60060160294122003dd924d692eade11:c,raw

As can be seen for the lun presented as hostid 0 from the array mpxio has not created the logical path and format detects 2 different sizes for the lun (suspect this is something to do with the active/passive connection)

I have been searching for any similar issues but can't seem to find a fix for this that doesn't involve a reconfig reboot. Can someone advise if there is an issue with the lun presented as hostid 0 (is this a reserved lun of some sort?)

Thanks

Adam

  • 2130

    1

    Posted February 29th, 2012 06:00

    I've found a solution for CLARiiON/VNX to correctly configure new paths for Solaris MPxIO and new paths for host LUN id 0.  First, run cfgadm -al | grep '::5006', this should show the connection points for CLARiiON/VNX storage devices (since their WWPNs all begin with 5006); note for Symmetrix it's 5000.  Make a note of the newly added connection points.  For each of these, run cfgadm -c unconfigure c#::WWPN, e.g. cfgadm -c unconfigure c9::5006016944600811.  Once the newly added connection points are unconfigured, run devfsadm -C to cleanup the device table.  Now for the newly added connection points, run cfgadm -c configure c#::WWPN.  Finally, once the newly added connection points are configured, run devfsadm.  If you now run mpathadm list lu, the device for host LUN id 0 should now have the correct number of paths.

  • kelleg

    6 Operator

    •

    4537 Posts

    1331

    0

    Posted April 8th, 2010 13:00

    In the Host Connectivity Guide on page 147 is talks about setting up using MPxIO

    Home >Support > Technical Documentation and Advisories >Host Connectivity/HBAs > Installation/Configuration

    see the EMC Host Connectivity Guide for Sun  Solaris  -- A24

    http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Technical_Documentation/300-000-607.pdf

    glen

  • kelleg

    6 Operator

    •

    4537 Posts

    1331

    0

    Posted April 8th, 2010 14:00

    Not being a server guy, I was once told by a Solaris support person that after all configuration changes involving Solaris that it's a good practice to do something called reboot/reconfig.

    Sort of like with NT - reboot for all issues

    glen

  • aprince1

    7 Posts

    1331

    0

    Posted April 8th, 2010 14:00

    Hi Glen, thanks for the link, I have already read this document and confirmed our solaris config is correct. We also opened a support call with Sun who confirmed this as well.

    Other luns presented from the array are pathed correctly via mpxio i.e a psuedo lun is created and 4 paths are visible from `mpathadm list lu` but not in format. The issue is with this individual lun, we see all 4 paths in format rather than the pseudo path which hasn't been created and format also shows the luns with different sizes (see the cylinders in the format output and the geometry error)

    Has anyone come across this error before?

  • aprince1

    7 Posts

    1331

    0

    Posted April 8th, 2010 15:00

    Glen

    I am predominantly a solaris admin and have minimal experience with Clariion's which is my reason for coming on these forums. mpxio is enabled on the solaris server with stmsboot -e (or varying options depending on how you want to configure mpxio) as part of that process a reboot is required to implement the changes.

    After the reboot the luns were assigned to the server, mpxio automatically creates the pseudo lun for 5 out of the 6 luns presented. It is just the first one we have issues with (hostid 0) and we are struggling to identify whether this is a config problem on the server or with the array / lun. Hopefully this is something we can resolve without having to reboot all the time

  • d2dowling

    30 Posts

    1331

    0

    Posted April 8th, 2010 16:00

    A couple of comments & questions:

    Was lun 47, host-id 0 previously attached to another server or was it 'freshly' bound for this system?
    I doubt if it is causing your ptoblem, but using naviseccli would allow you to present lun 47 to the server with a different host-id.
    What version of Solaris is in use? Failovermode 4 may work better.

    Excerpt from emc99467:

    For Solaris MPxIO or STMS, failovermode settings are as follows:
    • FLARE 24 or earlier: failovermode MUST be set to 1
    • FLARE 26 or later:
    - Solaris 9 with SAN 4.4.7 thru 4.4.11: failovermode MUST be set to 1
    - Solaris 9 with SAN 4.4.12 or later: failovermode may be set to 1 for
    active/passive behavior, or
    - Solaris 9 with SAN 4.4.12 or later: failovermode may be set to 4 for ALUA
    behavior
    - Solaris 10 U2 or earlier: failovermode MUST be set to 1
    - Solaris 10 U3 or later: failovermode may be set to 1 for active/passive
    behavior, or
    - Solaris 10 U3 or later: failovermode may be set to 4 for ALUA behavior
    Note: Use of failovermode = 4 (ALUA) with MPxIO/STMS cannot be supported
    with clustering environments requiring SCSI2 Reservations such as 2 node Sun
    Clusters.

  • aprince1

    7 Posts

    1331

    0

    Posted April 8th, 2010 18:00

    This server is running Solaris 10u8 and the array is at flare 28.6 I believe. I'm not entirely sure whether the lun has been used before, i'll get clarification on that. How would that impact the pathing of the lun if it had been used previously ?

    I'll look into the suggestion of failover mode 4 but it seems odd that all the other luns are fine with mode 1.

    Has anyone come across this specific issue before with solaris and mpxio? Have any of the suggested changes above worked.

    Thanks for all the comments so far, much appreciated.

  • kelleg

    6 Operator

    •

    4537 Posts

    1331

    0

    Posted April 9th, 2010 10:00

    I've attached the SUN E-Lab PDF - there are a lot of notes about connecting using MPxIO - there might be something here that will help - look for the section on Clariion and find your model/OS version and match to the clariion model.

    glen

    1 Attachment

  • StuartA461

    1 Rookie

    •

    63 Posts

    1331

    0

    Posted April 9th, 2010 11:00

    Why don't you use PowerPath?  Is that an option?

       Stuart

  • SKT2

    2 Intern

    •

    1340 Posts

    1331

    0

    Posted April 10th, 2010 03:00

    Is your multipath.conf tuned to the best practice for EMC clar array?Such as vendor  specific "devices" section for example