UNSOLVED

Bojik

updated

7 years ago

B

Bojik

18 Posts

0

2899

May 1st, 2020 10:00

RAID storage initilizing

My array is stuck with initilizing, I have tried new controllers but still stuck.

I ran 'su raidtool" and got this reply;

Technical Support.
Driver Status: *Admin Intervention Requested*
RAID LUN 0 Ok.
raid status unrecoverable.
15 Drives (0,2,4,6,8,10,12,1,3,5,7,9,11,13,14)
RAID 6 (64KB sectPerSU)
Capacity 9,986,026,831,872 bytes
Available Drives List: 15

Is my RAID gone?

Any way to recover?

  • dwilliam62

    6 Operator

    •

    1516 Posts

    •

    6374 Points

    1844

    0

    Posted May 1st, 2020 11:00

    Hello, 

     I just sent a reply asking for this output 

    You're RAIDset is not gone. As the LUN itself is "OK" 

     Do you have a good backup? 

     You have a couple of options. If you are in a region where Dell supports a Time and Material case you can open a case and they can properly triage it. 

     If you either have a good backup or are willing to risk data loss.  You can try to clear this LUN condition and the array could finish booting and everything will come back online.  If that happens you will then need to check the filesystem/data integrity of all your volumes.  I.e. using fsck, or chkdsik/scandisk for Windows. 

     The command would at the CLI would be  CLI>clearlostdata 

      Regards, 

    Don 

     

  • Bojik

    18 Posts

    1090

    0

    Posted May 1st, 2020 12:00

    Ok, on the reset. I just type in reset in the CLI and it will return to default right?

     

  • Bojik

    18 Posts

    1830

    0

    Posted May 1st, 2020 12:00

    On the chassis there are a few lights as well.  

    The chassis lights are address to left, power top right then a check mark and another symbel.  

    Both controller lights are all green or yellow.

  • Bojik

    18 Posts

    1831

    0

    Posted May 1st, 2020 12:00

    I did and appoligize for now having two posts for same overall issue (I'll stick to this post moving forward).  I think I answered you other question above.

  • Bojik

    18 Posts

    1839

    0

    Posted May 1st, 2020 12:00

    I have not replaced any drives but the place I bought the array from installed SSD drives that have been running fine for over a year.

  • dwilliam62

    6 Operator

    •

    1516 Posts

    •

    6374 Points

    1832

    0

    Posted May 1st, 2020 12:00

    Hello, 

    The yellow light on the controller simply means it's the passive CM.  Green/Green is active. And RED means the controller has a fault. 

     Your issue is not with the controllers. 

    Don 

     

  • Bojik

    18 Posts

    1842

    0

    Posted May 1st, 2020 12:00

    after clearlostdata it gets stuck in this loop:

    ####################################################
    # #
    # Dell (tm), Inc. Storage Array #
    # Copyright 2001-2009 #
    # #
    # Controller Information: #
    # Part=70-0300 Rev=A06 SN=RMS0943675G1CF4 ECO=C00 #
    # #
    #########################################################
    Bootloader Version 2.3.5 (SWINT Rev:1)
    Compiled on Mon Apr 30 17:08:16 EDT 2012
    (type h for help)
    Enter Ctrl-P for boot prompt

    Executing bootcmd0 [dload sd primary/eqlstor.gz]
    0X80231000/11912 0X85f08100/31424212 entrypt 80231000

    Executing bootcmd1 [run]
    cpu_online_map=ffff, userapp_cpu_map ffff
    psb_os_active_mask=0, psb_os_mask=0
    boot1_info: userapp_cpu_map=ffff, psb_os_cpu_map=0
    cpu_online_map = 0xffff
    Jumping to the application... 0x80231000
    ------------------------------------------------------------
    Preparing ffff bitmask of cpus to run
    No network device to cleanup
    count = 16, total = 16
    All slave cpus (16) ack'ed userapp init
    count = 4, total = 4
    All slave cpus (4) ack'ed message ring init

    Dell Inc. Storage Array

    Copyright 2001-2016 Dell Inc.
    SP:13.81:mips_pss_init.c:363:INFO:28.2.107:Control module in slot 1 with serial number RMS0943675G1CF4 is designated as active.
    SP:1588371414.28:ppool_nvram.c:396:ERROR:15.4.1:NVRAM contains valid data. This is a PANIC RECOVERY due to a panic on a NetBSD processor.
    SP:1588371414.28:ppool_nvram.c:188:ERROR:15.4.5:Saved CPU registers, CPU 0
    at 0000000000000000 v0 ffffffff80b73d00 v1 0000000000000000
    a0 ffffffff80b8d3e0 a1 ffffffff80b81200 a2 ffffffff80b74380 a3 ffffffff80b73d00
    t0 0000000000000840 t1 ffffffff815c5770 t2 0000000000000001 t3 ffffffff80ba0000
    t4 000000004000c001 t5 0000000000000007 t6 0000000000000007 t7 0000000000000000
    s0 ffffffff80b8d3e0 s1 ffffffff8efd3c00 s2 0000000000000000 s3 0000000000000001
    s4 0000000000000000 s5 0000000051eb851f s6 0000000000000064 s7 ffffffff8e175c80
    t8 ffffffff8344c00c t9 ffffffff80414538 k0 0000000000384bdf k1 0000000000000001
    gp ffffffff8110a6b0 sp 00000000c87c7820 s8 00000000000000a2 ra ffffffff809ef8f0
    SP:1588371414.28:ppool_nvram.c:206:ERROR:15.4.6:Saved CP0 registers, CPU 0
    sr 4000ff01 badva c0332e98 epc 8092eed0 errorepc bfc00000
    cause 00000008 errctl 00000000 cacheeri 00000000 cacheerd 00000000
    buserr 0000000000000000 cacheerrdpa 0000000000000000
    SP:1588371414.28:ppool_nvram.c:213:ERROR:15.4.7:Saved function call stack, CPU 0
    80586ab4 809ef8f0 80904abc 8091614c 80916b7c 80930d5c 809359b4 80935e78
    808a22d4 8088f660 8088f9cc 8088fd00 8099b458 8099b770 80592434 00000000
    SP:1588371414.28:eqllog_mbuf_Q.c:1100:ERROR:2.4.0:Panic recovery from CPU0 with reason 'kernel fatalassertion "!IOM_IS_INVALID_METADATA_VERSION(version)" failed: file "../../../NetBSD/src/sys/EQL/ioman/lv2/lv2_lv_log_read.cc", line 2112'.
    MFS set up
    Building databases...
    SP:1588371421.98:emm.c:1333:INFO:28.2.6:Enclosure serial number: SHU0937898Y9856.
    [ENTRY][BIST][EQL][TWDOG][WFP][210][CM1D][MESR][SCAN][BOOT2][PROC][BOARD][SMP][IOBUS][FLASH][HT][ETH0][ETH1][ETH2]

     

    I tried to call Dell and since I bought the array from a third party and am not the current owner in their records they will not help me.  And the third party does not have a record from over a year ago of who the original owner was.  GRRRRRRRRR

  • dwilliam62

    6 Operator

    •

    1516 Posts

    •

    6374 Points

    1092

    0

    Posted May 1st, 2020 12:00

    Hello, 

     Sorry but this is about as far I can take this here.  You can try to open up a T&M case or reset it and restore the data. 

     Regards, 

    Don 

     

  • Bojik

    18 Posts

    1829

    0

    Posted May 1st, 2020 12:00

    I pulled the drives halfway out and restarted the chassis and ran clearlostdata:

    CLI> clearlostdata
    % Error - could not retrieve email notification configuration

    WARNING: You are executing on the Passive Control Module.
    This is normally not the right thing to do, because most of
    the data this command gathers will not be available here.

    Do you want to proceed anyway (y/n) [n]: n
    Aborted.
    Invalid request, driver is not in admin state, no lost data to clear.
    CLI>

    PS Series Storage Arrays
    Unauthorized Access Prohibited

    login: grpadmin
    Password:


    Welcome to Group Manager

    Copyright 2001-2016 Dell Inc.

     

    The storage array is still initializing. Limited commands will be available unt
    il the initialization is complete. Please try again later.

    CLI> 3256:1:netmgtd: 1-May-2020 18:38:24.540001:rcc_util.c:1024:INFO::25.2.9:CLI: Login to account grpadmin succeeded, using local authentication. User privilege is group-admin.
    3464:3335:SP: 1-May-2020 18:38:36.493335:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.
    4167:4033:SP: 1-May-2020 18:39:17.754033:emm.c:2363:ERROR::28.4.47:Critical health conditions exist.
    Correct immediately before they affect array operation.
    Link to the enclosure management processor failed.
    There are 1 outstanding health conditions. Correct these conditions before they affect array operation.
    5401:5259:SP: 1-May-2020 18:40:29.851163:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.
    6103:5957:SP: 1-May-2020 18:41:11.101861:emm.c:2363:ERROR::28.4.47:Critical health conditions exist.
    Correct immediately before they affect array operation.
    Link to the enclosure management processor failed.
    There are 1 outstanding health conditions. Correct these conditions before they affect array operation.
    7338:7183:SP: 1-May-2020 18:42:23.213087:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.

    CLI> 8058:7898:SP: 1-May-2020 18:43:04.463802:emm.c:2363:ERROR::28.4.47:Critical health conditions exist.
    Correct immediately before they affect array operation.
    Link to the enclosure management processor failed.
    There are 1 outstanding health conditions. Correct these conditions before they affect array operation.

     

  • dwilliam62

    6 Operator

    •

    1516 Posts

    •

    6374 Points

    1836

    0

    Posted May 1st, 2020 12:00

    Hello, 

     Thank you.   Did you see my reply to your other post?

    Don