[geeks] asthsma attacks suck

[Goldarg]

Today just sucked:

2am I got paged that a box is dead
9am in the middle of my shower I  get paged by the guy who gets in before 
me asking where a ATX powersupply is.
9:45am on the drive into work I get paged saying the disk is dead not the 
power supply
10 am I get to work and find there are two large craters in the PCB where 
electronics used to be on the drive.
10am-3pm I work to try to find another SCSI drive to use to replace this 
drive, Bad news is this drive hasn't been backed up except for a dump of the
database on it since November, this drive is the only drive in a database 
box that also does cgi based building / destroying on one of our cisco's 
for our DSL customers, There are no backups of the current tools to do so, 
the backups we have are from 2 years ago of the tool before it supported 
PPP mode and using a completely different router.
10:30am Tour group from local highschool comes lead by a person in another 
department to distract our work on the dead system
3pm I realize that the disks I've spent all day futzing around with are 
DIFF SCSI instead of normal SCSI and that's why the damn things wont work, 
not a fried controller on the peecee that blew the disk. ( One of the 
drives I was trying to use for testing came out of a SS5 that we recently 
RMA'd and sent back to Seagate after it crashed about a month ago, hadn't 
looked at it yet but I'm pretty sure the internal drive in the SS5 isn't 
DIFF SCSI)
(we did find an exact match for the drive and its getting over nighted to 
us, hopefully we will see it tomorrow morning where I get to attempt to 
mate the electronics for the new drive with the platters/etc of the old 
drive to get one more read of our data, otherwise its off to some place in 
California that does data recovery)
3:30pm colo customer shows up to try to fix their NT box we colo for them, 
I have to go to the building next door where we keep our server farm and 
sit with him since its monitored access only, he needs a monitor to hook up 
to his NT box so I get one from our 2nd server room where we keep all the 
x86 boxes ( we only allow sun's and the alike in our big server room after 
a PC caught fire once),  While getting the monitor I bump a rack and 
unknown to me at the time the network card falls out of our network 
monitoring box (Home brew setup, it pings and does various types of 
connections and then uses its modem to page us if things are broken)
needless to say it started going nuts as to it the sky was falling and the 
world was coming to an end!
meanwhile I'm baby sitting the colo customer who is having problems with 
their NT box, so I page my boss and only remaining co-worker (in the last 
month and a half we have had 2 out of 3 admins leave, I started 6 weeks ago 
when the first guy left, he then hired the 2nd guy at his new place, my 
boss and I are basically screwed :) ) to come over and see what's wrong 
with the monitoring box.

The colo customer decides that his box cant be fixed without a restore from 
tape, I know nothing about our tape backups as the other guy does them so 
he decides to head back next door to where my boss is ( my page never went 
through but my co-worker came over to see if he could fix the box before we 
ran up a huge paging bill ) so I walk the customer to the door and head 
upstairs to the 2nd server room and find my co-worker. Then I call next 
door and find someone to greet the colo customer  on his return since it 
will be about 15 minutes or so we figure to get the network monitoring box 
happy again and we walk back.

Turns out the guy didn't wait so we came back to a sticky note from him 
saying he'd give us a call,

5pm he calls back and my co-worker goes next door and sets up their NT box 
with their assistance to run a terminal service or something ( I didn't 
take the call, I was busy still with the database box and its fried scsi)

meanwhile all through the day I'm being bugged by the support staff for our 
DSL service and their boss as they have to have their database back and 
they cant effectively troubleshoot without the cisco tools that we had (and 
by a tech who cant troubleshoot even with the tools we have).

I have to say this was the worst day of my job yet.

The good news is I asked for Friday off on monday and I plan to goto the 
Titan Missile Museum in Sahuarita, AZ which isnt too far from where I live 
and I've never had the chance to go. So a 3 day weekend and a hopefully fun 
trip should help :)