Welcome to my blog, chasing the Nines. This is a column about continuous improvement, leadership, and the delivery of quality services. We’re going to explore everything from the basics of understanding your environment to offering best practices for improving your organization’s approach to infrastructure and service management. Our journey will most certainly take us down the often divergent paths of the art and science of IT management. Comments and questions from the community are always welcome!
I have spent many years in the technology industry as a customer, a vendor, a representative for a large telecommunications carrier, and as a consultant. Consistently, I have observed IT organizations of varying sizes struggle with the delta between what is expected and what they can realistically deliver. The expectations are often dynamic if leadership is not aligned and the majority of the time, the delta between the user community and the technology community can feel like you are looking across the Grand Canyon!
That being said, the “art” is really in understanding and managing what’s in between those two points of reference… The “science” is using the right tools. Last week, I was attending parent night at my child’s school. The math teacher said something that resonated with me and was directly related to how I go about my work as a management and technology consultant. He said: “In my world, there are often multiple ways to get to the right answer; some easier than others. My goal is to teach my students that they have multiple tools at their disposal and when to choose the right one”.
Thus, every organization that I have ever worked with always seems to start with the “science” – They spend an inordinate amount of time researching and comprehending all the “standards” that have been published across our industry before they really step back and take the time to do some deep introspection and figure out what they are really being asked to deliver, why, and by whom. The techniques and methodologies are voluminous – ITIL, IT Service Management, Six Sigma, and so on. The reality is that these are the guidelines, tools, and techniques to get you to the goal-line once you really understand what’s expected. So, what I am asking of you here is to consider how you can be both an artist and a scientist as you chase the nines for your organization…
Until next time, I’m going to leave you on your own to explore a few simplistic but quite possibly complex questions: Just what does your organization expect? Just how many “Nines” does your organization really need? And, what are you realistically delivering today? Go for a run, a walk, a bike ride, or, whatever you do where you can seek solace and ponder that question. I’d be willing to bet that there are multiple answers to each of those questions depending upon whom you ask. I promise that we’ll circle back on these questions and more in the coming weeks.
As an economist and a technology executive, he has spent his career working with Fortune 500 companies across the professional service, manufacturing, financial, advanced technology, and telecommunications sectors. He has extensive experience managing profit and loss operations, developing and adopting new technology, leading product launches, managing IT infrastructure, global network operations, and driving sales and marketing organizations towards excellence.
Buck is recognized as an exceptional leader with the demonstrated ability to manage internal and external relationships, solve business problems, and implement strategy that nets revenue and profitability. He leads by example and develops organizational capacity supporting high performance work cultures and customer driven environments. Buck can be reached via telephone at 513-583-1516 or e-mail.
The Leader in Network Knowledge
thanks
thanks for the invite...I'll try to keep up as much as possible
In my experience the answer
In my experience the answer to the question is two and three, but your mileage will vary. Three nines gets you to about nine hours per year, four nines down to about an hour and five nines down to five minutes. Costs are exponential, so the benefits must also be exponential.
For most existing services, just look at how the service is currently hosted and supported, and that pretty much tells you which "nine" they picked. And it's usually right.
Look for disconnects to find what to investigate. For example, infrastructure support is on-call 24x7 but application support is 9x5. The servers are clustered, but most problems are reported through the help desk. These types of disconnects can show groups with different understandings and hence, different expectations. Or it might really make sense given a specific service. But it's start.
The reason might not always be obvious. You might conclude that shipping is four-nines because we customers need on-time delivery. Or the reason could be that if they can't ship they have to shutdown the production line due to limited storage space. Knowing the real reason is critical to getting an organization to design and operate a service effectively and react properly to unforeseen situations.
So my approach is to designate multiple service levels (I used gold, silver, bronze - real original) and then get agreement on how each part of the organization (IT and business) will participate at each level. Gets everyone on the same page and "jumping" at the same height. And, of course, determine the cost at each level. A service that needs five-nines to succeed but can't generate the revenue to support it has to fail.
For example, from Operations a bronze level might be basic system up/down, silver might be thorough systems management and gold might include both systems and application management. Clear distinctions and each level plays a part in reaching the availability Service Level Agreement.
Ultimately this tends to drive down the number of "nines" required for a service, but more importantly it drives alignment and priorities.
The answer, very correct.
Thanks Paul, my experience also. The nines are very dangerous if used alone as the only criteria, an one second interruption in manufacturing systems, steel, paper, etc mills from one week to a month, can cause huge damages when the process is going. When the process is in rest, who cares, you have all time in the world. Almost as bad is to lose the office back-end when there are ten thousand active customers in your offices, an ATM network at the busiest time, etc - 4am, not much damage. Some systems, like 24x7 warehousing management for a factory can actually work if it's up 10 minutes, down 10 minutes, and up again - if it can handle the 20 minute load in 10 minutes especially if the user devices have an offload capability. Frustrating, yes, but gets the job done.
Or anything between, a security / safety system where alerts, etc have to get through no matter what 24x7 but "doughnut calls" and tomorrows weather reports are not so important and can be delivered later (if ever) - different nines for different traffic in same infrastructure and no guarantees that the traffic patterns stay same over time.
Guaranteeing nines today is more difficult than it used to be, needs much more planning, design and contract work because almost no vendor can deliver the whole infrastructure and also manage it. In some projects I used to sit a week or two with customer lawyers and system people to go through all this in contract negotiations, talk about headache. It has to include "everything" - some customer top management has no humor?
The nines also have to be watched / monitored, the environments change all the time. It can lead to interesting confrontations, back to negotiations rather than to the court! Once you promise something, you better be proactive, otherwise it sooner or later bites.
Monthly meetings and staying in friendly contact with customers can sometimes be very useful, more than once saved (my skin) us from surprises. Just amazingly hard to convince some vendor management today, they just don't have the experience and often the sales/PM don't have the technical savvy to see the problems before they happen.