UNSOLVED

lrnajjones

updated

16 years ago

L

lrnajjones

1 Rookie

•

16 Posts

0

16338

February 10th, 2011 12:00

MD3200i Problem

I have a MD3200i with two R610 servers and two PC6224 Network Switches.

The problem I am having the two servers operate VERY slowly over ethernet.  There are four subnets for each card.

192.168.130.X/24

192.168.131.X/24

192.168.132.X/24

192.168.133.X/24

Both VMWare servers are connected using the Broadcom 5709 network adapters (not configured thru BIOS).  When I have high reads/writes, vmware says that the connect is lost, then immediately reset itself.  I have talk with VMWare and they said it's a problem with the MD3200i's.  They ran a program called "esxtop", then press d, and I had a extemly high iSCSI command delay. 

Anybody know what I could be doing wrong?

  • Dev Mgr

    6 Operator

    •

    9308 Posts

    •

    3 Points

    768

    0

    Posted February 10th, 2011 18:00

    Can you paste in the output of:

    From the server (SSH):

    - esxcfg-vswitch -l

    - esxcfg-vmknic -l

    - esxcli swiscsi nic list -d vmhba## (where ## is the instance number of the software iSCSI initiator (assuming that you're running the software iSCSI initiator)

    From the 6224 switches:

    - show running-config

  • lrnajjones

    1 Rookie

    •

    16 Posts

    768

    0

    Posted February 10th, 2011 19:00

    ~ # esxcfg-vswitch -l
    Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
    vSwitch0         128         12          128               1500    vmnic4

      PortGroup Name        VLAN ID  Used Ports  Uplinks
      Commission Network    0        9           vmnic4
      Management Network    0        1           vmnic4

    Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
    vSwitch2         128         9           128               1500    vmnic0,vmnic1,vmnic2,vmnic3

      PortGroup Name        VLAN ID  Used Ports  Uplinks
      iSCSI4                0        1           vmnic3
      iSCSI3                0        1           vmnic2
      iSCSI2                0        1           vmnic1
      iSCSI1                0        1           vmnic0

    ~ # esxcfg-vmknic -l
    Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type
    vmk0       Management Network  IPv4      172.20.20.72                            255.255.254.0   172.20.21.255   f0:4d:a2:06:58:1c 1500    65535     true    STATIC
    vmk0       Management Network  IPv6      fe80::f24d:a2ff:fe06:581c               64                              f0:4d:a2:06:58:1c 1500    65535     true    STATIC, PREFERRED
    vmk2       iSCSI1              IPv4      192.168.130.13                          255.255.255.0   192.168.130.255 00:50:56:7d:b5:0d 1500    65535     true    STATIC
    vmk2       iSCSI1              IPv6      fe80::250:56ff:fe7d:b50d                64                              00:50:56:7d:b5:0d 1500    65535     true    STATIC, PREFERRED
    vmk3       iSCSI2              IPv4      192.168.131.13                          255.255.255.0   192.168.131.255 00:50:56:7e:0f:9b 1500    65535     true    STATIC
    vmk3       iSCSI2              IPv6      fe80::250:56ff:fe7e:f9b                 64                              00:50:56:7e:0f:9b 1500    65535     true    STATIC, PREFERRED
    vmk4       iSCSI3              IPv4      192.168.132.13                          255.255.255.0   192.168.132.255 00:50:56:70:03:ae 1500    65535     true    STATIC
    vmk4       iSCSI3              IPv6      fe80::250:56ff:fe70:3ae                 64                              00:50:56:70:03:ae 1500    65535     true    STATIC, PREFERRED
    vmk5       iSCSI4              IPv4      192.168.133.13                          255.255.255.0   192.168.133.255 00:50:56:79:08:a9 1500    65535     true    STATIC
    vmk5       iSCSI4              IPv6      fe80::250:56ff:fe79:8a9                 64                              00:50:56:79:08:a9 1500    65535     true    STATIC, PREFERRED
    ~ #

    vmware2-switch>enable

    vmware2-switch#show running-config
    !Current Configuration:
    !System Description "Powerconnect 6224, 3.2.0.7, VxWorks 6.5"
    !System Software Version 3.2.0.7
    !Cut-through mode is configured as disabled
    !
    configure
    vlan database
    vlan 100,200-203
    vlan routing 200 1
    vlan routing 201 2
    vlan routing 202 3
    vlan routing 203 4
    exit
    hostname "vmware2-switch"
    sntp unicast client enable
    sntp server 172.20.20.12
    clock summer-time recurring USA zone "CDT"
    clock timezone -6 minutes 0 zone "CST"
    stack
    member 1 1
    member 2 1

    --More-- or (q)uit
                        

    exit
    ip address 172.20.20.74 255.255.254.0
    ip default-gateway 172.20.20.1
    ip name-server 172.20.20.12
    ip name-server 172.20.20.9
    logging file debug
    logging buffered debug
    ip routing
    interface vlan 200
    name "iSCSI0"
    routing
    ip address 192.168.130.1 255.255.255.0
    exit
    interface vlan 201
    name "iSCSI1"
    routing
    ip address 192.168.131.1 255.255.255.0
    exit
    interface vlan 202
    name "iSCSI2"
    routing

    --More-- or (q)uit
                        

    ip address 192.168.132.1 255.255.255.0
    exit
    interface vlan 203
    name "iSCSI3"
    routing
    ip address 192.168.133.1 255.255.255.0
    exit
    username "admin" password ************* level 15 encrypted
    !
    interface ethernet 1/g5
    description 'MD3200i Array 2 RC0 P0'
    spanning-tree disable
    switchport access vlan 200
    exit
    !
    interface ethernet 1/g6
    description 'MD3200i Array 2 RC0 P1'
    spanning-tree disable
    switchport access vlan 201
    exit
    !
    interface ethernet 1/g7
    description 'MD3200i Array 2 RC0 P2'
    spanning-tree disable
    switchport access vlan 202
    exit
    !
    interface ethernet 1/g8
    description 'MD3200i Array 2 RC0 P3'
    spanning-tree disable
    switchport access vlan 203
    exit
    !
    interface ethernet 1/g11
    description 'VMSERVER2 iSCSI0'
    spanning-tree disable
    switchport access vlan 200
    exit
    !
    interface ethernet 1/g12
    description 'VMSERVER2 iSCSI1'
    spanning-tree disable
    switchport access vlan 201
    exit
    !
    interface ethernet 1/g15
    description 'Symantec BackupExec Server'
    spanning-tree disable
    switchport access vlan 202
    exit
    !
    interface ethernet 1/g19
    description 'APC-ATS Switch'
    exit
    !
    interface ethernet 1/g24
    description 'Uplink'
    switchport mode general
    switchport general allowed vlan add 100 tagged
    exit
    !
    interface ethernet 2/g5
    description 'MD3200i Array 2 RC1 P0'
    spanning-tree disable
    switchport access vlan 200
    exit
    !
    interface ethernet 2/g6
    description 'MD3200i Array 2 RC1 P1'
    spanning-tree disable
    switchport access vlan 201
    exit
    !
    interface ethernet 2/g7
    description 'MD3200i Array 2 RC1 P2'
    spanning-tree disable
    switchport access vlan 202
    exit
    !
    interface ethernet 2/g8
    description 'MD3200i Array 2 RC1 P3'
    spanning-tree disable
    switchport access vlan 203
    exit
    !
    interface ethernet 2/g11
    description 'VMSERVER2 iSCSI2'
    spanning-tree disable
    switchport access vlan 202
    exit
    !
    interface ethernet 2/g12
    description 'VMSERVER2 iSCSI3'
    spanning-tree disable
    switchport access vlan 203
    exit
    !
    interface ethernet 2/g19
    switchport access vlan 100
    exit
    snmp-server community public ro
    snmp-server host 172.20.20.2 public informs
    exit

    vmware2-switch#exit

    vmware2-switch>quit

     

  • Dev Mgr

    6 Operator

    •

    9308 Posts

    •

    3 Points

    768

    0

    Posted February 11th, 2011 10:00

    Are you trying to use the Broadcom hardware iSCSI offload? You didn't include the "esxcli swiscsi nic list -d vmhba##" output for the software iscsi initiator instance.

     

    Per these 2 links, it seems that some testing shows that hardware iSCSI (on Broadcom 1Gbit NICs) without jumbo frames is significantly slower than software iSCSI with jumbo frames:

    - http://www.delltechcenter.com/thread/4100544/vSphere+4.1+upgrade

    - http://www.vmadmin.co.uk/vmware/35-esxserver/252-esxihwswiscsijumbo

     

    I'd suggest the following changes:

    On the switches:

    - for each iSCSI port, use the following settings: "mtu 9216", "spanning-tree portfast", "storm-control broadcast", "storm-control multicast"

    On the servers:

    - rebuild your iSCSI setup using the software initiator with jumbo frames. You can use http://www.delltechcenter.com/page/VMware+ESX+4.0+and+PowerVault+MD3000i to find the commands to do this.

     

    After that, you'll want to see how your setup ends up running.