News

 
   

The Trouble With Troubleshooting

With So Much Time Spent Finding Problems, Troubleshooting Is A Burden Itself
By Curt Harler
 
June 29, 2007 - In the classic symphony “The Sorcerer's Apprentice,” a lad starts to work with a simple magical broom. But it won't quit working. He tries cutting it in half, but it doubles instead. He soon finds himself in the same fix many network managers find themselves today: Life has gotten a lot more complex. A tool that was supposed to simplify everything has grown out of all proportion to manage.

Like the sorcerer's apprentice, many IT managers need a magician to help restore control. The first tip from network gurus is not to let things get out of control. For some, it is almost too late — but the situation can be salvaged.

“The problem is real, but it is based more on things being different today,” says James Messer, director of technical marketing at Network General (www.Processor.com/NetworkGeneral). “We've reached a new level of complexity and are really turning up the volume.” He points out video, VoIP, multitiered database applications, IP-V6, and mobile access. But it is not hopeless.

“Fundamentally, we are still gathering the same information,” he continues. IT still uses SNMP and still counts packets.

What is different, Messer says, is how IT uses the information it gathers. “You will be totally overloaded if you try to look at everything that you can gather,” he says. “You have to examine your goals for gathering information.”

Because business processes are different, those goals will be different in a call center than in a branch office or for a bank than for county government.

Get It Under Control

According to Tim Clark, vice president of sales for North America at eG Innovations (www.Processor.com/eGInnovations), the only way an SME can hope to control growing networks is with automation. “How else do you get your arms around it?” he asks. He points out that the average computer-based business service today has 28 components, with a range from two to 64 components.

“The complexity is resoundingly real,” agrees David Messina, vice president of marketing for Xangati (www.xangati.com). “It is a direct result of the boom in networked endpoints. It is no longer just desktops, printers, and servers but VoIP phones, PDAs, storage devices, video cameras, and medical devices. There is a burgeoning number of sanctioned and unsanctioned (e.g. Skype, P2P, IM) applications these endpoints produce and consume.”

“The IT organization is quickly losing the ability to keep track of the tangled web of communication that is actually going on over its network infrastructure,” Messina adds. “Monitoring in this new complex web of enterprise communication is no longer a sustainable practice. These tools generate too much information about too many things to be actionable for an IT staffer. That is why 74% of the time, according to Forrester Research, it is the end user — not the IT organization — that notices a problem first. In other words, monitoring tools leave IT in a position where they are unfortunately reactive instead of proactive.”

“When a service goes down, it is impossible to know where to start troubleshooting,” Clark says. The typical process — starting at the middle, working in one direction, and backtracking if all appears well — is time consuming. “Gartner did a study that said 80% of the time spent to fix a problem was spent on diagnosis. Only 20% to fix it,” he continues. “A lot of finger-pointing typically goes on. The result is a typical problem on an average N-tier network takes 22 hours to resolve.”

Messer offers two pieces of advice to SMEs. “First, keep it simple,” he says. “Rightsize your analysis tools today,” he says. “If you spend a little time now, your analysis tools will grow with the organization.”

That brings him to point two: Plan for the future. “Look forward. Expect growth,” Messer advises. Network General, and many other vendors, buy into this approach by providing testers and analysis tools that are easy to grow. Simply purchase another license for the software, and the existing tool is upgraded — nothing more to configure, nothing new to get shipped. This allows an SME to start small and grow.

Messer says IT needs to move ahead with the changing business worldview. It is not enough to understand bits and bytes, but IT needs to be able to troubleshoot a network to meet the goals and needs of the organization.

However, according to Enterprise Management Associates, 58% of the time troubleshooting tools will provide IT insufficient information and will result in a problem going unresolved. “That is a troubling statistic and means that productivity and revenue-affecting issues can potentially persist indefinitely,” Messina says. He says the reason these tools come up short is because each one was optimized for a specific part of the infrastructure — the network, an application, the server — and the information they collect cannot easily be pieced together. “What is missing is a way to understand how all these different elements interact with each other in the context of which specific applications; only in this manner can you have the right comprehensive view to do proper troubleshooting,” he says.

Know What You Have

Messer's advice to either the sorcerer's apprentice or to network managers would be the same: Get things back under control one step at a time. “Examine how each link or box works. See what it ties to. You don't get anything done until you start doing it.”

He advises talking to people who have been around awhile, documenting their institutional knowledge. “Start the process. It sounds daunting,” he says, “but if you start small, you'll gain perspective. Sit with people and document what they know.”

In fact, he says, the job typically is more one of understanding and coordinating existing knowledge than of inventing anything new.

“Increasing complexity has even more dramatic consequences for the smaller and midsized enterprise organizations, which tend to be shorter staffed and have smaller budgets for troubleshooting tools,” Messina notes. That all-hands-on-deck effort to track the core problem becomes incredibly taxing on the productivity of the IT organization, given that 90% of the effort in resolving a problem is in the identification work,” he says.

Moreover, the loss of end-user productivity tied to complex application performance-affecting problems has more of an effect on an SME. Lost productivity results in a loss of nimbleness, which is generally the competitive advantage that smaller organizations have over their larger competitors.

“Automate as much of the process as you can,” Clark says. “It is almost impossible for a tech just out of school to know everything about a network. Even a senior person cannot know everything about 50 different applications. But, with good software, you can diagnose and troubleshoot problems.”

A standard view of the network — whether for Oracle (www.Processor.com/ORCL) database or help desk — will make the job easier, he says.

“A maxim that an SME IT manager should live by is that change is inevitable, and as such, they should make sure to invest in tools that evolve well with that change,” Messina notes. “This has often been a hard credo to live by in the network management sector because the integration times for these tools are often so long that the network has evolved by the time they have been fully implemented.”
 
© Copyright 2007. eG Innovations All rights reserved.
All other company, brand and product names may be trademarks or registered trademarks of their respective holders.