That Time an IT Emergency Made Me Sneeze Blood

Due to popular Twitter demand, you all apparently want to hear this tale. Warning, it really isn’t all that gruesome, but should probably serve as a cautionary fable for anyone who has decided to get into the magical world of consulting. I am also under an NDA, so the names have been changed to protect the grotesquely stupid, and I sadly, do not have any photos.

The Situation

This is a client I started working for about a year ago, mostly network stuff. They brought me in to reign in the insanity that is intrinsic to small <redacted> industry IT (hint, all IT sucks). One of the first things I did was whip out a label maker, and label patch cables everywhere I could find them, this saved my butt in putting this all back together later.

This particular small shop had a single rack for their IT assets, tucked into a back store room. This rack had many, many, many issues. I guess it is time for a bullet list.

  • 23 inch rack, nothing in the rack was wider than 19 inches. So, multiple 2 inch spacers on each side, top to bottom.
  • Two post rack, plenty of stuff that really screams four post. At least it was all at the bottom.
  • Cheap, flimsy construction, this thing would wobble even without the 2 inch spacers.
  • Filled to the brim, stuffed.
  • Bolted to the floor, a badly poured concrete slab that had clearly been laid down in winter. Stomping your feet made dust appear.
  • A stiff breeze caused this rack to wobble in all directions.

Between the shaky rack, and the shitty foundation, it doesn’t take an idiot to realize that the bolts holding this thing down were slowly wiggling themselves free. I told them a year ago, this is going to fall, and it is going to suck. They dismissed my warnings. Oh, I should have walked then.

Friday

They call in the AM. “We are completely down, our rack of servers fell over!”

“Yup, lemme grab my drill and my crimpers, I’ll be right in.” I replied.

Coffee to go would have been appropriate, but I had a cup of traditional at home first. My E-rate doesn’t start until I arrive, and I warned them, I fucking warned them.

Also, I knew what I would have to do.

Sure enough, the rack had ripped the bolts straight out from the floor, and collapsed. One Dell something or other is not in good shape, as it took the brunt of the fall. The rest looks like it might be alive.

I tasked one of their underlings with testing cables, anything that cleared gigE/voip on the the Paladin was re-usable, at least for now. I got to work on the rack itself. Fortunately, I only had to make six new cables by the end of this mess. No, underling didn’t know how to do that, and I am not a teacher when shit is hitting the fan right after shit has hit the fan.

So, four big ass 3/4 inch bolt holes in the floor, blasted out to all hell and back like incels think happens to lady bits if they dare have sex with someone not them. Yes, this rack really needed something bigger, 1 1/4 would be a solid minimum, but, I don’t carry concrete bolts in my Network/Systems/Security IT kit. Shit, I don’t have those in my house. But I do own a drill that can eat concrete. Thank you very much DeWalt for making a beast of a monster that I can afford. Also, my years in WISP land left me with a collection of masonry bits. LETS DO THIS.

Relocate the rack a few feet over, and mark out my holes. “NUMBER ONE, ENGAGE!”

This is where the shitty foundation starts to matter. In addition to not typically carrying concrete lags in my standard IT kit, I don’t normally bring a hazmat mask. This concrete slab had clearly been poured in the winter. For those not familiar with construction, masonry, or physics, water freezes, water is a critical component of concrete. When you lay concrete in sub zero temperatures, you get some bad shit, like a lot of dust, uneven level, and an overall shit pour.

I spent the next forty five minutes creating dust storms in my face, drilling out four holes in shit concrete with my barely adequate DeWalt Doomhammer.

I inhaled a small quantity of dust, it sucked. Then I had to make six replacement cables, and trace out shitloads of stuff that had come loose, and test. I was there for just short of three hours. Maybe 2 hours, 40 minutes. We got it done, they didn’t lose an entire day. I’m good, yo.

The Aftermath

I felt like shit, I had clearly inhaled a great deal of dust. But the next morning…… Dear god. Sneezing up blood, repeatedly. That was not fun. I still feel like ass, my nose and throat are clearly irritated beyond belief.

The client has already contested my bill. My emergency rate is always in hour increments, rounded up, no exceptions.  This particular client has a signed contract stating this, so I will get my money. But, that isn’t the point. The three hours of E-rate have no chance of addressing any possible health complications I might encounter because of this mess. Yet, here they are trying to claim they only owe me for two and a half hours, not a solid three. Now, I find myself looking for a legal way to make them responsible for the hell that is my lungs right now.

The Moral

And the moral of this motherfuckah is, ladies make em……. no wait, that is Prince.

Don’t let a company fuck with your health, they will happily do so to get what they want. I am currently updating my contracts to include personal health and danger clauses.

Organizations will not look out for you, you have to make sure you are looking out for yourself. Do not make your health a lower priority than your dedication. It isn’t worth it.