[SunHELP] Update! (was RE: Can login in single-user mode - can't in regular - could use a nudge)

[Brian Dunbar]

FWIW.

The actual problem appears to be the vintela auth software.

Haven't discovered 'why' it blows up, that's going to be a week or so away.
Thought I'd get this out now.

- We found another V240 in the 'CYA stack'. [1]  Installed the OS, recovered
from backup.

- Dragged the chassis back to the lab for follow-up.


THIS time I let the login at console just sit there for a while.  Eventually a
message printed to TTY referencing multi-user service being disabled.  Ah-ha:
a clue.

Back to single-user, disabled multi-user service.  Boot and .. it comes up as
expected.  Enable multi-user and it hangs running the
/etc/rc2.d/Swhatevervintealla script.  prstat shows the CPU is clocked at 50%
for this process.  And it stays like that for just under 30 minutes before I
killed the process.

mv Swhatevervintealla swhatevervintealla
init 6

And the box comes up as it should.

I kicked myself, but not hard.  The clues were all there the day of the
problem, I just didn't put them together due to fatigue.



[1] Management hates our CYA hardware.  I can see their POV, but as long as we
_have_ old hardware in production, and it's not covered by a service contract,
having a supply of hanger queens is a good idea.

~brian


-----Original Message-----
From: Brian Dunbar
Sent: Tuesday, October 25, 2011 2:55 AM
To: 'sunhelp at sunhelp.org'
Subject: Can login in single-user mode - can't in regular - could use a nudge

Hi,

It's late, I'm tired. I've got a production problem and could use a nudge.

V240.
Solaris 10.

DMZ host, running three non-global zones.  Call from service desk "FTP isn't
working".

SSH connection - hangs.
Login through SC, issue 'console -f'.  I can login but am not returned to
shell - it hangs.

Issue break, bring it up in single-user mode.  Disk space looks tight but ok.
Restart, same issue as above.

I've set all three non-global autoboot=false, same problem.

>From another Solaris host I ssh ...

# ssh brian.dunbar.admin at hostname
Server had a GSS-API error; the connection will close (458752/2):
No credentials were supplied, or the credentials were unavailable or
inaccessible
No such file or directory

Use the GssKeyEx option to disable GSS-API key exchange and try again.
Disconnecting: The server had a GSS-API error during GSS-API protected SSHv2
key exchange



So perhaps I do have a disk space problem?

# zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool  33.8G  28.8G  4.97G    85%  ONLINE  -


Which is tight but should be okay.


I'm missing something.   Comments welcome.  Maybe someone could airdrop some
coffee while they're at it.


Brian Dunbar
Systems Administrator - IT
920.751.3364 (Direct)
920.716.2027 (Mobile)