UNSOLVED

a.aleksiev

updated

11 years ago

0

126265

February 26th, 2016 13:00

PowerEdge 2950 LOM (Broadcom NetXtreme II BCM5708) Firmware support for Wake-on-LAN

Hi,

I'm helping in ESXi 5.1 U3 maintenance on two PE2950 with Broadcom NetXtreme II BCM5708 1000Base-T cards.

When upgrading the FWs for different components I mentioned that the new firmware (bc 4.6.0 UMP 1.1.9) is not supporting WOL for the ESXi kernel:

~ # esxcli network nic get --nic-name=vmnic1
Advertised Auto Negotiation: true
Advertised Link Modes: 10baseT/Half, 10baseT/Full, 100baseT/Half, 100baseT/Full, 1000baseT/Full
Auto Negotiation: true
Cable Type: Twisted Pair
Current Message Level: 0
Driver Info:
Bus Info: 0000:09:00.0
Driver: bnx2
Firmware Version: bc 4.6.0 UMP 1.1.9
Version: 2.2.5k.v50.1
Link Detected: true
Link Status: Up
Name: vmnic1
PHYAddress: 1
Pause Autonegotiate: true
Pause RX: false
Pause TX: false
Supported Ports: TP
Supports Auto Negotiation: true
Supports Pause: true
Supports Wakeon: false
Transceiver: internal
Wakeon: None
~ #

The old FW, on the other identical server, is with full support:

~ # esxcli network nic get --nic-name=vmnic1
Advertised Auto Negotiation: true
Advertised Link Modes: 10baseT/Half, 10baseT/Full, 100baseT/Half, 100baseT/Full, 1000baseT/Full
Auto Negotiation: true
Cable Type: Twisted Pair
Current Message Level: -1
Driver Info:
Bus Info: 0000:09:00.0
Driver: bnx2
Firmware Version: bc 2.9.1
Version: 2.0.15g.v50.11-7vmw
Link Detected: true
Link Status: Up
Name: vmnic1
PHYAddress: 1
Pause Autonegotiate: true
Pause RX: false
Pause TX: false
Supported Ports: TP
Supports Auto Negotiation: true
Supports Pause: true
Supports Wakeon: true
Transceiver: internal
Wakeon: MagicPacket(tm)
~ #

Initially both cards were with the same driver version (2.0.15g.v50.11-7vmw) - even when I updated the card with new FW to the last ESXi 5.1 driver (2.2.5k.v50.1) the WOL feature was not available.

I was wondering if somebody can direct me to the right combination of FW and drivers in order to have the WOL available - it can be even older, but working versions.

Thank you in advance,

Alex

  • DELL-Josh Cr

    Community Manager

    •

    9790 Posts

    •

    45217 Points

    1880

    0

    Posted February 26th, 2016 16:00

    Hi,

    If wake on lan is disabled in the NIC settings it will show false for supports wake on lan. You can enable WOL in the Broadcom NICs Boot Agent. During POST you may see a banner message for the Broadcom NICs. Press Ctrl-S, and it will let you enable the WOL option through that menu. If the banner is not there just press ctrl+s during the early part of post.

  • 1892

    0

    Posted February 28th, 2016 13:00

    Hi Josh,

    Thank you for the prompt reply!

    I forgot to mention (sorry about that) that the BIOS and the NIC boot Agent setups were the first two things I verified. There were nothing related to the NIC configuration in the BIOS, as I expected when saw the Broadcom boot agent prompt.

    This is a screenshot from the DRAC



    As you can see, the feature is enabled on the agent level as well.

    The only difference with the working server is the Firmware, as I specified in my initial post.

    Do you have any idea how to proceed?

    Greets,

    Alex

  • DELL-Josh Cr

    Community Manager

    •

    9790 Posts

    •

    45217 Points

    1892

    0

    Posted February 29th, 2016 09:00

    Was the firmware from us or directly from Broadcom? Do you have a link to the update that you used?

  • 1892

    0

    Posted February 29th, 2016 13:00

    That's the only one I found is from 12 Jan 2012:

    www.dell.com/.../DriversDetails

    Greets,

    Alex

  • DELL-Josh Cr

    Community Manager

    •

    9790 Posts

    •

    45217 Points

    1892

    0

    Posted February 29th, 2016 15:00

    That looks like the latest 5708 firmware aside from the more generic versions that do not explicitly state 5708 support. Will it let you set WOL with ethtool? ethtool -s vmnic0 wol g

  • 1892

    0

    Posted February 29th, 2016 19:00

    I gave it a try with two different tools - same negative result:

    ~ # esxcli network nic list
    Name PCI Device Driver Link Speed Duplex MAC Address MTU Description
    ------ ------------- ------ ---- ----- ------ ----------------- ---- -------------------------------------------
    vmnic0 0000:005:00.0 bnx2 Up 1000 Full 00:18:8b:44:4f:a9 1500 Broadcom Corporation QLogic 5708 1000Base-T
    vmnic1 0000:009:00.0 bnx2 Up 1000 Full 00:18:8b:44:4f:ab 1500 Broadcom Corporation QLogic 5708 1000Base-T
    ~ # esxcli network nic set -w g -n vmnic0
    Not Support wol.
    ~ # ethtool -s vmnic0 wol g
    Cannot set new wake-on-lan settings: Invalid argument
    not setting wol
    ~ #

    I'm pretty sure that the same Firmware should fix this problem, but I can't find it for download.

    In the same moment I'm really curious why the new one is not supporting something well working before.

    Do you have any other suggestions?

  • 1892

    0

    Posted February 29th, 2016 21:00

    I am also looking for this solution

  • DELL-Josh Cr

    Community Manager

    •

    9790 Posts

    •

    45217 Points

    1892

    0

    Posted March 1st, 2016 09:00

    The old firmware versions can be found here. downloads.dell.com/.../poweredge-2950.html

  • 1892

    0

    Posted March 1st, 2016 15:00

    Hi Josh,

    I created a bundle from those Firmware files and prepared a bootable USB (Linux ISO) to facilitate the upgrade:

    I was using manual mode from the media check start option.
    The auto-update will go up to the highest available firmare version:

    sh-4.1# cd /opt/dell/toolkit/systems/drm_files/
    sh-4.1# ./apply_bundles.sh

    Force scripts to update to lower version with '-f' option:

    sh-4.1# cd /opt/dell/toolkit/systems/drm_files/repository/NetXtreme\ II\ BCM5780/
    sh-4.1# sh NETW_FRMW_LX_R214709.BIN -q -f | tee /bundleapplicationlogs/apply_components.log
    sh-4.1# reboot

    next time just specify manually the next version BIN file:

    sh-4.1# sh NETW_FRMW_LX_R246418.BIN -q | tee /bundleapplicationlogs/apply_components.log

    As you can see, the lowest version is "4.6.8" - shown in the CLI as "bc 4.6.0":

    NETW_FRMW_LX_R319248.BIN
    ~ # esxcli network nic get --nic-name=vmnic1
    Advertised Auto Negotiation: true
    Advertised Link Modes: 10baseT/Half, 10baseT/Full, 100baseT/Half, 100baseT/Full, 1000baseT/Full
    <... omitted ...>
    Driver Info:
    Bus Info: 0000:09:00.0
    Driver: bnx2
    Firmware Version: 6.4.5 bc 5.0.7 UMP 1.1.9 <--- This is the highest version
    Version: 2.2.5k.v50.1
    Link Detected: true
    <... omitted ...>
    Supports Pause: true
    Supports Wakeon: false <---
    Transceiver: internal
    Wakeon: None
    ~ #















    NETW_FRMW_LX_R214709.BIN
    ~ # esxcli network nic get --nic-name=vmnic1
    Advertised Auto Negotiation: true
    Advertised Link Modes: 10baseT/Half, 10baseT/Full, 100baseT/Half, 100baseT/Full, 1000baseT/Full
    <... omitted ...>
    Driver Info:
    Bus Info: 0000:09:00.0
    Driver: bnx2
    Firmware Version: 4.6.8 bc 4.6.0 UMP 1.1.9 <--- This is the lowest version
    Version: 2.2.5k.v50.1
    Link Detected: true
    Link Status: Up
    <... omitted ...>
    Supports Pause: true
    Supports Wakeon: false <---
    Transceiver: internal
    Wakeon: None
    ~ #
















    NETW_FRMW_LX_R239524.BIN
    ~ # esxcli network nic get --nic-name=vmnic1
    Advertised Auto Negotiation: true
    Advertised Link Modes: 10baseT/Half, 10baseT/Full, 100baseT/Half, 100baseT/Full, 1000baseT/Full
    <... omitted ...>
    Driver Info:
    Bus Info: 0000:09:00.0
    Driver: bnx2
    Firmware Version: 5.0.9 bc 5.0.4 UMP 1.1.9 <---
    Version: 2.2.5k.v50.1
    Link Detected: true
    <... omitted ...>
    Supports Pause: true
    Supports Wakeon: false <---
    Transceiver: internal
    Wakeon: None
    ~ #















    ...

    How can I find the "bc 2.9.1"  firmware, used on the other PE2950?

    Greets,

    Alex

  • 457

    0

    Posted March 1st, 2016 16:00

    It seems like I found the reason ans it's not in the Firmware at all.
    I'm going to share this here in order to serve somebody else in the future...

    WOL is supported in Broadcom NetXtreme II BCM5708 devices with silicon revisions of B2 or later.

    To determine the silicon revision of one Broadcom NetXtreme II device, we can use the 'lspci' command on a linux box, where “10” = revision B0, “11” = revision B1, and “12” = revision B2.
    For ESXi the 'lspci' is with very limited output, but we can use 'esxcli hardware pci list' to discover the details.

    A verification on the PE2950 with working WOL is showing revision B2 (0x12):

    ~ # lspci | grep 5708
    00:05:00.0 Network controller: Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T [vmnic0]
    00:09:00.0 Network controller: Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T [vmnic1]
    ~ # esxcli hardware pci list | sed -n "/0[5,9]:00.0/,/^$/p"
    000:005:00.0
    Address: 000:005:00.0
    Segment: 0x0000
    Bus: 0x05
    Slot: 0x00
    Function: 0x00
    VMkernel Name: vmnic0
    Vendor Name: Broadcom Corporation
    Device Name: Broadcom NetXtreme II BCM5708 1000Base-T
    Configured Owner: Unknown
    Current Owner: VMkernel
    Vendor ID: 0x14e4
    Device ID: 0x164c
    SubVendor ID: 0x1028
    SubDevice ID: 0x01b2
    Device Class: 0x0200
    Device Class Name: Ethernet controller
    Programming Interface: 0x00
    Revision ID: 0x12
    Interrupt Line: 0x0b
    IRQ: 11
    Interrupt Vector: 0xc0
    PCI Pin: 0xb0
    Spawned Bus: 0x00
    Flags: 0x0201
    Module ID: 19
    Module Name: bnx2
    Chassis: 0
    Physical Slot: 0
    Slot Description: Embedded NIC 1
    Passthru Capable: false
    Parent Device: PCI 0:4:0:0
    Dependent Device: PCI 0:4:0:0
    Reset Method: Bridge reset
    FPT Sharable: false

    000:009:00.0
    Address: 000:009:00.0
    Segment: 0x0000
    Bus: 0x09
    Slot: 0x00
    Function: 0x00
    VMkernel Name: vmnic1
    Vendor Name: Broadcom Corporation
    Device Name: Broadcom NetXtreme II BCM5708 1000Base-T
    Configured Owner: Unknown
    Current Owner: VMkernel
    Vendor ID: 0x14e4
    Device ID: 0x164c
    SubVendor ID: 0x1028
    SubDevice ID: 0x01b2
    Device Class: 0x0200
    Device Class Name: Ethernet controller
    Programming Interface: 0x00
    Revision ID: 0x12
    Interrupt Line: 0x0b
    IRQ: 11
    Interrupt Vector: 0xb8
    PCI Pin: 0x32
    Spawned Bus: 0x00
    Flags: 0x0201
    Module ID: 19
    Module Name: bnx2
    Chassis: 0
    Physical Slot: 0
    Slot Description: Embedded NIC 2
    Passthru Capable: false
    Parent Device: PCI 0:8:0:0
    Dependent Device: PCI 0:8:0:0
    Reset Method: Bridge reset
    FPT Sharable: false

    ~ #

    A verification on the PE2950 with NO working WOL is showing revision B1 (0x11):

    ~ # lspci | grep 5708
    00:05:00.0 Network controller: Broadcom Corporation QLogic 5708 1000Base-T [vmnic0]
    00:09:00.0 Network controller: Broadcom Corporation QLogic 5708 1000Base-T [vmnic1]
    ~ # esxcli hardware pci list | sed -n "/0[5,9]:00.0/,/^$/p"
    000:005:00.0
    Address: 000:005:00.0
    Segment: 0x0000
    Bus: 0x05
    Slot: 0x00
    Function: 0x00
    VMkernel Name: vmnic0
    Vendor Name: Broadcom Corporation
    Device Name: QLogic 5708 1000Base-T
    Configured Owner: Unknown
    Current Owner: VMkernel
    Vendor ID: 0x14e4
    Device ID: 0x164c
    SubVendor ID: 0x1028
    SubDevice ID: 0x01b2
    Device Class: 0x0200
    Device Class Name: Ethernet controller
    Programming Interface: 0x00
    Revision ID: 0x11
    Interrupt Line: 0x05
    IRQ: 5
    Interrupt Vector: 0xc0
    PCI Pin: 0x00
    Spawned Bus: 0x00
    Flags: 0x0201
    Module ID: 19
    Module Name: bnx2
    Chassis: 0
    Physical Slot: 0
    Slot Description: Embedded NIC 1
    Passthru Capable: false
    Parent Device: PCI 0:4:0:0
    Dependent Device: PCI 0:4:0:0
    Reset Method: Bridge reset
    FPT Sharable: false

    000:009:00.0
    Address: 000:009:00.0
    Segment: 0x0000
    Bus: 0x09
    Slot: 0x00
    Function: 0x00
    VMkernel Name: vmnic1
    Vendor Name: Broadcom Corporation
    Device Name: QLogic 5708 1000Base-T
    Configured Owner: Unknown
    Current Owner: VMkernel
    Vendor ID: 0x14e4
    Device ID: 0x164c
    SubVendor ID: 0x1028
    SubDevice ID: 0x01b2
    Device Class: 0x0200
    Device Class Name: Ethernet controller
    Programming Interface: 0x00
    Revision ID: 0x11
    Interrupt Line: 0x05
    IRQ: 5
    Interrupt Vector: 0xb8
    PCI Pin: 0x00
    Spawned Bus: 0x00
    Flags: 0x0201
    Module ID: 19
    Module Name: bnx2
    Chassis: 0
    Physical Slot: 0
    Slot Description: Embedded NIC 2
    Passthru Capable: false
    Parent Device: PCI 0:8:0:0
    Dependent Device: PCI 0:8:0:0
    Reset Method: Bridge reset
    FPT Sharable: false

    ~ #

    Since the revision B1 is not supporting WOL, there are no way to see it working on this server.
    However, in meanwhile I configured successfully the remote power management via IPMI ;-)

    Thank you Josh for your prompt replies and kind cooperation!

    Cheers,

    Alex