No water, No work

Today was an unexpected day off. When I arrive at work it was busier than usual around the premisses and in the garden of the studio. Lot's of people hanging out, chatting at an early hour that is usually packed with dailies, touchbase meetings, rounds, and what not.

When I got to my desk I noticed it was slightly hotter in side than normal. Got to my email to find a zillion of mails, with alaring titles like network branch down, filer pegged, database server shutdown, Culver City water pressure issue, restrooms out of order, cue taken offline, AC capacity reduced, ...

What is going on? Scrolling all the way down I read that it all began with some construction work going on in Culvercity. Someone had hit a main tube of the watersystem, which burst, causing the waterpressure to drop to a very low level over big parts of the city. First thoughts...fine... no water... that's not too bad. We are not cooking, nor brewing coffee (at least not professionally), we are a hightech company. Electricty loss would be bad.... NOT!

The sudden los in waterpressure made the airconditionning system fail. That had to be reduced in power and eventually be shut down. Still, we can work in an environment that is a few degrees hotter than usual... but wait... what about the cooling of the machine room? No cooling, means that servers, filers, routers and renderracks had to be shut down. First partially, soon pretty much all of them. There number, we are talking about thousands of computers in a confined space, makes that they produce so much heat that if they wouldn't actively be cooled, they would fail because the heat. Talking about a melt down...

They brought in huge watertrucks and a number of portable toilets (the type that is used on events). Everyone was told they could leave till 13.00 When we came back, workstations were still hung, files unaccessible, email down. A few hours later the first systems were brought back online. Limited capacity. Movies that are near their deadlines (we are talking a few weeks) get priority. We got to go home. Our systems wouldn't be fully operational before the next day. No high tech without low tech. But we enjoyed the time off. That's for sure.