It's one of the unwritten laws of physics: At some time or another, everybody screws up.
But when IT pros make mistakes, they don't mess around. Entire buildings go dark. Web sites disappear. Companies grind to a halt. Because if you're going to mess up, you might as well make it count.
"I always tell my guys, hey, you're gonna do stupid stuff," says Rich Casselberry, director of IT operations at Enterasys, a networking systems vendor. "It's OK to do something stupid if you have the wrong information. But if you do something stupid because you're stupid, that's a problem. The trick is to not flip out, which only makes it worse, or try to hide it. You need to figure out how to keep it from happening again."
[ For more adventures in IT mishaps, check out Stupid user tricks 3: IT admin follies and Stupid QA tricks: Colossal testing oversights ]
We've gathered up some of the more egregious examples from IT pros brave enough to share their screwups with us. Backups gone bad, people with admin privileges who probably shouldn't, what can go south when you unplug the wrong equipment -- in some cases, we've obscured their identities to spare them embarrassment; other geeks, however, are perfectly willing to own up to their youthful mistakes.
Sure, some of these mishaps are amusing in retrospect. But don't laugh too hard. We know you've probably done worse.
True IT confession No. 1: The case of the mysterious invisible backup
Our first tale of misadventure involves a longtime IT pro who doesn't want his real name used, so we'll just call him Hard Luck Harry.
Harry had his share of mishaps when he started out a decade ago at a major networking equipment maker in the Northeast. There was the time he changed an environmental variable that broke everything on his company's financial apps, earning an e-mail from his boss ordering him to "never hack on this system again." Or the time he crashed the company's core ERP system by overwriting /dev/tty. Harry says after he accidentally ripped the company's T1 lines out of the wall with his pager, he was banned from ever reentering the telecom closet.
But the worst one happened after Harry installed an Emerald tape backup system. Did he bother to read the manual? Please. This was child's play. Just load install.exe and let the software do its thing.
It seemed to work perfectly. Four hours later, the first backup completed and everything looked fine.
Fast-forward six months. Harry gets a call late one night at home from one of his work pals. That night's backup tape is completely blank, the friend tells him. Worse, the last four weeks of backups are also blank.
As Harry soon discovered, that particular backup program installs in demo mode by default. Demo mode looked exactly like real mode and even took the same amount of time as an actual backup, but nothing ever got written to tape -- a fact that was noted in the manual, which Harry might have seen had he read it.
Read more about adventures in IT in InfoWorld's Adventures in IT Channel.
Get the independent advice and expertise you need to support a virtual workforce.
The increase in Linux popularity has increased the frequency and sophistication of malware attacks. Read this 2 page white paper now to learn how you can protect your Linux environment with real-time protection that is certified by all major Linux vendors.
Download now »
Ensuring acceptable application delivery will become even more difficult over the next few years. As a result, IT organizations need to ensure that the approach that they take to resolving the current application delivery challenges can scale to support the emerging challenges. This handbook elaborates on the key tasks associated with planning, optimization, management and control and provides decision criteria to help IT organizations choose appropriate solutions.
Download now »
A common misconception is that mid-range storage requirements are dramatically different than that of a larger enterprise. Mid-range storage users may require less capacity, but they have similar functionality and management requirements. This ESG paper examines mid-range storage needs and reviews a new solution that adjusts size while retaining value, performance and functionality.
Download now »
Sign up to receive InfoWorld Resource Alerts
61 Recommendations
