UNSOLVED

Jodi-2015

updated

20 years ago

J2

Jodi-2015

17 Posts

0

1693

November 27th, 2006 06:00

daemons hanging

Good Morning,

Our current setup is a Networker server running 7.3.2 on a Solaris 8 platform with storage node running 7.3.2 on a Solaris 9 platform. Almost all clients are running the same version. We have a few cluster clients where the physical node has been upgraded, but the virtual node still reports the old version.

Several times our server has just hung. At first there were GSS authentication errors reported. We disabled GSS and the same thing happened this weekend. Three backups were running and just stopped. There were no errors reported, below is the section of the daemon.log:

11/26/06 02:38:07 nsrd: stacker3:index:drnotes1 done saving to pool 'IndxSt2Weekly' (A05038) 502 MB
11/26/06 02:38:27 nsrd: stacker3:index:bhnsm02 saving to pool 'Weekly' (A06435L2)
11/26/06 02:43:03 savegrp: job (53779) host: f68vrec savepoint: /u01 had WARNING indication(s) at completion.
pools supported: IndxSt2Weekly;
11/26/06 03:50:09 savegrp: Aborting inactive job (53513) lnotes3:/plaza2
11/26/06 03:50:10 savegrp: command 'save -s stacker3 -g fullweekly -LL -f - -m lnotes3 -l full -q -W 78 -N /plaza2 /plaza2 ' for client lnotes3 exited with return code 1.
11/26/06 03:50:10 savegrp: job (53513) host: lnotes3 savepoint: /plaza2 had WARNING indication(s) at completion.
11/26/06 03:50:10 savegrp: lnotes3:/plaza2 failed.
* lnotes3:/plaza2 (interrupted), exiting
* lnotes3:/plaza2 aborted due to inactivity
11/26/06 03:50:10 savegrp: lnotes3:/plaza2 will retry 1 more time(s)
11/26/06 04:03:05 nsrlcpd #1: Jukebox `lto1' is exiting. The jukebox is no longer managed by nsrlcpd.

Then there was nothing in the daemon log and nsrwatch got hung as well. I could not even do an nsr_shutdown. I had to kill all processes one at a time. Then when I restarted the daemons, there was an error saying the volumes have reached the low water mark. A second restart of the daemons cleared that.

Any ideas?
  • ble1

    6 Operator

    •

    14354 Posts

    •

    56186 Points

    477

    0

    Posted November 27th, 2006 07:00

    First time nsrd or even nsrexecd went down or into banana state... but no idea why of course.
  • Jodi-2015

    17 Posts

    477

    0

    Posted November 27th, 2006 08:00

    GSS authentication is a new method of authenticating, it uses ssl.

    Here is how we disabled it:

    # nsradmin -p nsrexec
    NetWorker administration program.
    Use the "help" command for help, "visual" for full-screen mode.
    nsradmin> . type: NSRLA (note: there is a blank space between . and type)
    Current query set
    nsradmin> show auth methods
    nsradmin> print
    auth methods: "0.0.0.0/0,nsrauth/oldauth";
    nsradmin> update auth methods: "0.0.0.0/0,oldauth"
    auth methods: "0.0.0.0/0,oldauth";
    Update? yes
    updated resource id 3.0.156.101.198.4.188.67.137.69.101.64(137)
    nsradmin> print
    auth methods: "0.0.0.0/0,oldauth";
    nsradmin> quit

    Restart daemons when done.
  • Jodi-2015

    17 Posts

    477

    0

    Posted November 27th, 2006 08:00

    This has happened several times before. The first couple times there were GSS authentication errors reported. After disabling GSS, there were no errors reported.
  • pfrassino

    1 Rookie

    •

    77 Posts

    477

    0

    Posted November 27th, 2006 08:00

    Hi,
    since I'm a novice in v7.3.2, could you clarify what GSS authentication means and how did you disable it ?

    Thanks
    Pierpa
  • ble1

    6 Operator

    •

    14354 Posts

    •

    56186 Points

    477

    0

    Posted November 27th, 2006 10:00

    Funny enough, today I was playing with that on HPUX server and did manage to break index backups (that would be fine after restart). After few tweaks I managed to make it work, but it's really amazing how unstable can be.
  • pfrassino

    1 Rookie

    •

    77 Posts

    477

    0

    Posted November 28th, 2006 00:00

    Jodi,
    the GSS is disabled on the NW server side or on the NW client/storage node side ?
    Suppose to have a client in v7.2.1, a storage node and a server in v7.3.2.
    I have to

    1) do nothing on client side ( v7.2.1 has no authentication method )
    2) modify storage node configuration with noauth
    3) modify server configuration with noauth

    Isn't it ??

    Thanks
    Pierpa
  • Jodi-2015

    17 Posts

    477

    0

    Posted November 28th, 2006 04:00

    That sounds correct. We disabled it on both the server and storage node.
  • Jodi-2015

    17 Posts

    477

    0

    Posted December 1st, 2006 06:00

    I'm not prepared to say our problems are cured, but I believe they may be.

    It looks like our server couldn't keep up. We have upgraded the CPU and memory on the Networker server. Our backups all ran without issue last night.
  • Jodi-2015

    17 Posts

    477

    0

    Posted December 5th, 2006 05:00

    It looks like we're in the clear. After upgrading the hardware all problems have gone away. We have not re-enabled GSS and probably won't. It doesn't seem necessary.