IT has never been easier or harder

October 29, 2012

http://www.infoworld.com/d/data-center/it-has-never-been-easier-or-harder-205644?page=0,1

IT has never been easier -- or harder

Beneath all the virtualization, automation, and plush Web GUIs lurk obscenely complex puzzles to solve

I often hear and read comments about how "easy" IT has become. I suppose if you compare IT of a decade ago versus IT now, there's some truth to that statement.

Nowadays, we can completely automate server instance creation and management, deploy complex application frameworks with a few clicks of a mouse, and troubleshoot server problems from the bottom up without leaving our desks. Not so long ago, much of this would have required actual heavy lifting in the form of racking servers, manual OS installations, visits to the data center, complex cabling and networking configurations, and so forth.

[ Revisit Paul Venezia's classic blog post: How to become a certified IT ninja. | Get expert networking how-to advice from InfoWorld's Networking Deep Dive PDF special report. | For the latest practical data center info and news, check out InfoWorld's Data Center newsletter. ]

But as time goes on, we have continually adapted and refined these methods to ease administration, streamline common tasks, and lighten our load somewhat. Mind you, we haven't reduced complexity -- IT systems are more complex than ever before -- but there's much less manual labor involved and fewer repetitive tasks. We've been working on abstraction of IT fundamentals, moving things up the chain. This requires significantly more planning and effort at the outset, but the rewards and savings are instantly realized and quickly surpass the cost of the initial effort.

Virtualization is of course the driving force behind this progress, now encompassing not only server virtualization but also storage virtualization, application virtualization, and network virtualization via concepts like VMware's VXLAN. It takes much more skill to build these components, but less to drive them on a day-to-day basis. We're no longer constantly building and rebuilding our IT infrastructure, but building it once and leveraging that investment over longer periods of time.

But make no mistake, IT is not "easy."

The mundane operations are simpler and even self-service in many cases, but we're still beholden to the vagaries of our field, the thoroughly unexpected. On one hand, you might point to the AWS outage of a week ago. It was an extremely lengthy disruption, highly visible (especially to non-IT folks), and apparently caused by human error and compounded by a lack of planning. All the automation in the world wouldn't have helped prevent that breakdown, and it might generally make the problem worse. Amazon will be paying for that blunder for a long time.

Then there are peculiar problems like the one Mina Naguib recently encountered. I happened upon this write-up last week and consider it to be a perfect example of a highly skilled admin discovering and rectifying a problem that by all reasonable understanding shouldn't exist. Logical or not, troubleshooting the impossible seems to navigate all the same stages as grief.

1. Denial. A highly intermittent problem that's difficult to reproduce, involving one of the Internet's most stable and reliable protocols. It simply shouldn't be happening.
2. Anger. After exhausting all reasonable troubleshooting steps, the problem persists. Frustration is an understatement.
3. Bargaining. Even the most seasoned IT ninja has fleeting thoughts of trading something to make the problem go away. It usually involves promises of spending more time implementing monitoring and consistency checking if the fix would just appear out of thin air.
4. Depression. This doesn't necessarily affect the problem solver, but I've seen many cases where it almost immediately affects management and others on the periphery of the issue. They're not the ones with their heads down working through the roadblock. They're on the sidelines, losing hope that the problem will ever be fixed. Their sky is falling.
5. Acceptance. This is the one stage where the paths of IT problem solving and grief diverge. There's no acceptance of a problem in IT. It must be fixed -- one way or another. The quality and reliability of that fix may be in question, but there is no point at which you can just give up on a major blocking problem. It's not an option. It must work. The bits must flow.

In the case of Mina's bizarre TCP corruption issue, the problem was caused by a network provider several hops away, with no direct relationship to the affected parties. A router with buggy code or a blown interface was causing this unfathomable behavior, and it was only through remarkable diligence and a carefully tuned eye that it was brought to light.

This wasn't a problem to be fixed with a few mouse clicks or by running through a Web portal. This wasn't a problem a novice could uncover and fix. This was not easy.

The public face of IT might seem to be getting easier, more fluid, faster, and sleeker, but behind that paint and polish, the work of ages continues. The heavy lifting may now be more logical than physical, but the load is in fact weightier than ever.

This story, "IT has never been easier -- or harder," was originally published at InfoWorld.com. Read more of Paul Venezia's The Deep End blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.