DDOcrackhead wrote on Sep 2
nd, 2013 at 1:52pm:
Ok I'm an addict on withdrawal and I know that's part of the problem, yet I have to be mean and say: Why do this fat ass Jerry and the dyke not give out any information whether this will take 1 hour or 1 day?
I can answer for him.
First I'm critical to Turbine when it's needed ( like now, but that's a manglement decision and the poor guys at the bottom are just there to take the flak ).
They can't give an ETA because they have no idea of the ETA.
See my previous post, when I was called at 10 AM on friday I had no clue of the size of the failure.
The Fiber channel card was replaced at 4PM.
Then I spent from 4PM to 10 PM to try to get back the server working.
I raised the fucking Third Line ( going through the second line like lightning after explaining the problem at 10 PM. I left the site, a colleague joined me at midnight, at 3 AM on Saturday.
I went back on the site at 5PM on the Saturday, the system was not online yet, there was a few databases to restore.
That was already one day and a half of downtime and we fucking had no clue of the ETA when the system would be back online as we had 4 databases to recover from a 2 year old backup...
( and even if we could recover we didn't know if the data in the databases wasn't too obsolete )
After restoring all the databases it was 3AM Sunday, the system came online but with two years old data. ( it as a provisioning system for telco equipments, needless to say that we HAD TO recover the missing data one way or another before the customer could provision data again, it took us a month )
So all in all in my case, the downtime started at 10 AM on a Friday and lasted until 5AM on a Sunday... And we had no clue about the ETA or partial system recovery until 1 AM on Sunday.
This outage made history in Local Support, at the Global Support Level and at the Third Line ( Design ) level as one of the best (disaster) recovery ever made on that system, but I can tell you that when I was working on it it didn't feel like one of the best recovery, it was a pure and unbriddled nightmare.