Origin3k

updated

2 years ago

O

Origin3k

6 Operator

•

2424 Posts

•

12408 Points

2

899

December 18th, 2024 21:01

OME 4.3 reboot Server even if you select "no reboot" patches!!!!

Hello,

i have upgraded to 4.3 and updated also all the plugins.  If i try to upgrade the iDRAC in the servers by using the filter Reboot required: "no" and execute all of the selected Hosts(R750/R740 ESXi Server) rebootet to my surprise.  Also the Windows is now different because there is no drop down any more about Shutdown and instead there is just "no reboot".

I bring down 2 dozen ESXi Hosts included and vSAN Cluster.  That was not funny!!

  • jimmy_muraleedharan

    1 Rookie

    •

    13 Posts

    •

    6 Points

    0

    1

    Posted January 21st, 2025 12:22

    Hello, 

    The firmware update issue discussed here in OpenManage Enterprise v4.3 is addressed with 4.3.1 release. Sharing the links for 4.3.1 release below.

    OME 4.3.1 is now available!

    Note: The firmware issue that was seen in OME 4.3 has been fixed in OME 4.3.1.

    OpenManage Enterprise version 4.3.1 replaces OpenManage Enterprise version 4.3. If you are currently using OpenManage Enterprise version 4.3, we strongly recommend upgrading to version 4.3.1 immediately. Please reach out to all of your OME 4.3 customers with this information.

    Link for new installs: https://www.dell.com/support/home/drivers/driversdetails?driverid=835VY

     

    To upgrade offline from OME 4.1, OME 4.2 and OME 4.3, please use this link: https://dl.dell.com/openmanage_enterprise/4.3.1/full/omenterprise_4.3.1.tar.gz

  • DELL-Young E

    Moderator

    •

    5421 Posts

    •

    37 Points

    0

    0

    Posted December 19th, 2024 02:15

    Hello,
    4.3 was just released 
     
    Can I ask for exact firmware updates you did? 
     
    from what version to what version

     

    Respectfully,

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • Origin3k

    6 Operator

    •

    2424 Posts

    •

    12408 Points

    0

    0

    Posted December 19th, 2024 08:05

    We upgrade to iDRAC 7.10.90.00  from the previous version which i think was 7.10.70.00.   We use OME "core" since a very long time and in general.

    This installation does not use OMEVV.

    Based on a quick review FW for NICs are also installed even if they are not selected because of the used "filter".  I am also sure that "Clear Joblog" was selected because thats part of your checklist. We would avoid that there a pending actions.

    I will spend some time over the holidays to find out whats going wrong.

    Regards,
    Joerg

  • Origin3k

    6 Operator

    •

    2424 Posts

    •

    12408 Points

    0

    0

    Posted December 19th, 2024 08:45

    As you can see clear jobqueue was selected and executed.  I assume that OME also deliver the NIC Updates as well and from the iDRAC perspective for that a reboot is required.  Because of the changed Windows GUI and the dropdown which wasnt there for sure there where Code changes.

    Target System: esx-node-34.acme.local
    Messages:
    Running
    Verifying if the device Service Tag is valid.
    The device Service Tag is valid.
    Starting communication with the device.
    Performing the requested operation
    Delete Job Queue has been attempted
    Checking LC Service State
    Checking Remote Services availability
    Remote LC Services are ready.
    Reset-iDRAC and/or Clear Job Completed
    Update allowed in target, System Lockdown status = DISABLED
    Checking Remote LC Services availability
    Remote LC Services are ready now
    performIDRACUpdate(): Attempting to initiate/stage iDRAC payload instance: DCIM:INSTALLED#iDRAC.Embedded.1-1#IDRACinfo
    Initiated Status check for Job ID : JID_345343316829
    Job state for JID_345343316829 is NEW
    Job state for JID_345343316829 is DOWNLOADING
    Job state for JID_345343316829 is COMPLETED
    Checking Remote LC Services availability
    Remote LC Services are ready now
    Attempting to initiate/stage payload instance: DCIM:INSTALLED#701__NIC.Integrated.1-1-1
    Initiated Status check for Job ID : JID_345349485610
    Job state for JID_345349485610 is DOWNLOADING
    Job state for JID_345349485610 is DOWNLOADED
    Job state for JID_345349485610 is SCHEDULED
    Initiating host reboot action as one or more payload(s) marked as reboot required.
    Job state for JID_345349485610 is RUNNING
    Job state for JID_345349485610 is COMPLETED
    Completed

  • DELL-Young E

    Moderator

    •

    5421 Posts

    •

    37 Points

    0

    0

    Posted December 21st, 2024 06:25

    Hello, I can see what you have done wrong but it's a bit tricky to show you with words because it's a process.

    ttempting to initiate/stage payload instance: DCIM:INSTALLED#701__NIC.Integrated.1-1-1

    So you selected not just iDRAC but NIC as well, and I'm not blaming you it's easy to make this mistake. You're supposed to choose only idrac but you chose multiple and it's clearly indicated in the logs.

    If you use filter on firmware selection it does not reset options you selected prior. For example, you select downgrade option and mark one firmware for upgrade.
    then use filter for non-reboot option and assume previous firmware gets deselected, it is not true. It still ticked, but hidden. If you submit iDRAC it still going to grab NIC you ticked prior. need to be careful with the way you use firmware compliance report.
    If you understand, it's good. If you don't, I highly recommend that you raise an official ticket so that the support can remote-control you.

    Respectfully,

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • Origin3k

    6 Operator

    •

    2424 Posts

    •

    12408 Points

    0

    0

    Posted December 26th, 2024 12:40

    I dont have selected the NICs.  As stated we use the "NO Reboot" filter to get a View which components are saved to update and these where selected.

    About the "downgrade uption" i need to check that but based on LCC the NIC versions are liftet up..so no forced downgrade.

    We dont use the tool for first time. For sure we can do something wrong. 

    The OME GUI have changed with the 4.3 as you can see on my screenshot.

    Regards,

    Joerg

    Regards,

    Joerg

    (edited)

  • Jprlopes

    1 Rookie

    •

    1 Message

    •

    2 Points

    0

    0

    Posted January 7th, 2025 12:17

    Its happening to me the same issue.

    Start after the last OME update....

    But not only bypassing the Reboot / no Reboot choice. If you select only a server on a group, he will install the updates on all servers on the group!

    Please fix this ASAP!

  • Franck Ehret

    1 Rookie

    •

    11 Posts

    0

    0

    Posted January 8th, 2025 07:15

    Hi, there,

    The exact same happened to me on Monday : I wanted to update iDRAC only so I selected the "no reboot" in my compliance report and triggered the update.

    Result :

    - Most of my servers in multiple clusters/locations were SHUT DOWN because at some point in time, Dell OME appliance did the forced shutdown on the servers, including the vSphere host were it was located

    - "Luckily, half of our main vSphere cluster was still on when OME was powered off, so OME was not able to shut them all and vSphere HA could start rebooting VMs

    - vCenter , Active Directory and Exchange were ungracefully shut down, resulting of main systems being unavailable, including filers that relies on AD.

    THOUSANDS of users were impacted for a couple of hours.

    HOW can I trust this software now if the most basic stuff doesn't work as expected?

    Claim to account manager is following, I'm ashamed and mad at the same time

    PS: happened in the past, I almost post a reply there: https://www.dell.com/community/en/conversations/dell-openmanage-enterprise/performing-bulk-idrac-updates-without-system-reboot/64c105fcf4ccf8a8decf5f43

    (edited)

  • DELL-Marco B

    Moderator

    •

    3980 Posts

    0

    0

    Posted January 8th, 2025 21:18

    Hello,

    I didn't find any known issue related to this OME release.

    Please can you provide some details? Any logs from OME about forced shutdown?

    Which version of iDRAC firmware?

    Thanks

    DELL- Marco B

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • Franck Ehret

    1 Rookie

    •

    11 Posts

    0

    0

    Posted January 9th, 2025 07:44

    Hi @DELL-Marco B 

    I just opened a service request <Private data removed from public view. DELL-Admin> and I uploaded audit logs but of course we can provide any other logs, just let me know how I can generate/transmit them to you.

    K.R.

    Franck

    (edited)