It’s been a long tradition around our house that I cook the Christmas dinner. I enjoy cooking but don’t get the time to do it as often as I would like; my wife does the great majority of day-to-day cooking, so she enjoys not having to cope with the Christmas feast.
I approach the preparations like I would an engineering project. Recipes are selected, ingredients and quantities go into an Excel spreadsheet, and a categorized shopping list is developed. When the ingredients are brought home I sort them into groups on a table with a printout for the recipe under each group. Cooking schedules are worked out to optimize stovetop and oven availability, temperatures, and pot sizes.
Like most amateur cooks I measure my ingredients carefully. I often make modifications to a recipe from one time to the next (increase the baking time of the pecan pie by 15 minutes, add a little orange juice to the Waldorf salad, cook the cheese grits the night before so they firm up a bit), but when the action starts I adhere strictly to the specified measurements and temperatures.
Have you ever watched a master chef work? There’s surprisingly little measuring going on. He or she knows the ingredients intimately, how their flavors interact, how temperature affects them. For the master chef, cooking is an art. For me, it’s an exercise in accurately quantifying components to get an expected outcome.
My Christmas dinner is predictable and consistent. It’s not art, but it tastes good.
Unlike cooking, when we design or operate a network we don’t want a “master chef” that is striving for something creative and exciting. Excitement is seldom a good thing in networking. We want predictability and consistency. We want quantification, not art.
Yet all too often we encounter people who claim that certain aspects of network design are “more art than science.” What those people are really saying is, “I have no idea how to quantify what I’m doing.”
The inability to put numbers around a networking project makes life difficult for us in a number of ways. How can we do a useful cost/benefit analysis of the project if we cannot quantify the benefits? How can we do a risk analysis if we can’t quantify the risks? How do we “sell” the project to upper management as an improvement in reliability, security, or performance if we cannot explain reliability, security, or performance with numbers? Even cost is not always quantified as accurately as it should be.
Take, for example, a simple change request form. Most of these have a section entitled “Risk Assessment” or something to that effect, and in most cases all that section contains is a requirement for you to rate the risk on a scale of 1 to 5 or perhaps as high, medium, or low. Maybe there’s also an area for you to write a brief verbal description of what you think the risks might be. Such subjective assessments are almost never useful, and are usually there so that someone responsible for change management can say they at least asked about risk.
The vagueness and hand-waving that accompanies so many network project proposals can cause the CFO to view IT with a great deal of mistrust. Here’s a guy whose life revolves around numbers. If you go to him asking for a few million dollars for a new network project but cannot make a case – backed up by believable data in the language the CFO understands – for why those millions of dollars are a promising investment rather than just a cost, you are less likely to get your project funded.
For that matter, do you yourself know that the project will be worth the cost, even before you go to the CFO and swamp him with technical jargon and fuzzy claims that the project will improve network reliability/security/performance?
In the next few posts I’m going to write about how you can quantify things you might have thought were not measurable, how to better assess risks and benefits, and how to put numbers around your project proposals that will get the attention and support of the boardroom. I’m not going to torture you with Bayesian probability or stochastic simulations; I’m going to write about the sorts of questions you should be asking yourself in order to make a concept quantifiable, define a means of measuring it, and reducing the uncertainty around the concept.
But for now, I’ve got to go brine my turkey.
Merry Christmas!
Jeff Doyle is president of Jeff Doyle and Associates, an IP network consultancy. Jeff is the author of Routing TCP/IP, Volumes I (read an excerpt) and II and of OSPF and IS-IS: Choosing an IGP for Large-Scale Networks. He is a frequent speaker on IPv6, MPLS, and large-scale routing.
The Leader in Network Knowledge
Merry Christmas
This topic is definitely interesting and very relevant, considering the current global economic situation. Hope to see your follow up posts on this soon. Best of luck with your cooking.
Wishing you and your family a Merry Christmas.
Willy
yes good point...
...however, being vague or ambiguous sometimes is all we have time for. Many "professional" managers (i.e., not IT experience to speak of) or managers from the server or app side of the house have major misconceptions about networking.
For instance... why consult with network at all about putting new servers in or turning new apps/services up? There's a port right? Just plug it in!
There is the myth that one more server or one more app should be OK, its a big network... so no need to specifically address "network" now.
So when it does come time to do a network project, then suddenly we have to shine the rust off of our project management and design skills... yet management wants answers NOW. It takes time to design/test/validate/acquire/install/migrate "network."
Re: good point
This is a good lead-in for some of what I'll be writing about. You say "being vague or ambiguous sometimes is all we have time for." That's a very good example of why I want to write on this topic: Determining the value of information. Perhaps the value is low enough, in some cases, that vague and ambiguous is good enough. But what, on the other hand, might the cost be of that ambiguity? How can you reduce the uncertainty to the point that you can make a case to management, using convincing numbers, to fund a project or even to fund getting more numbers?
--Jeff
I'm looking forward to this
I'm looking forward to this series of articles... The value of information, indeed!
Also thanks for writing the OSPF/ISIS book... love it!
Measurement needs tools
Jeff this is a great topic ...
We have standardized cups, spoons and scales to measure the ingredients, temperature and time for cooking. These tools are easily available at all grocery stores. However, the situation with networking is not so. The tools available are usually too expensive (tens of thousands of dollars) for even medium size networks. Even those that are available generally use simulation which generally has a difficult planning process. The network designer
1. designs the network
2. makes all kinds of traffic model assumptions
3. runs simulation
4. tweaks the network then runs simulation and repeat step 3.
In this process step 2 requires good understanding of stochastic processes - if wrong assumptions are made planning process becomes a case of "garbage in - garbage out". Step 3 requires access to expensive tools and training. Step 4 should be titled "magic happens here". It all gets complicated quickly - forgetting the questions CFO is asking.
Your insights on techniques and best practices for network design are eagerly awaited.
Merry Christmas
Re: Measurement needs tools
Hi Samia,
This is a terrific comment. I especially agree with you that modeling is expensive. I've written before about OPNET, for example; I use their SP Guru Network Planner for modeling in my own consulting practice, because it's a powerful and effective tool. But it's far from cheap. And, as you say, even with a tool like OPNET if you put bad information in you're going to get bad information out.
The question I ask, however, is: Expensive relative to what? Yes these tools are expensive, but what's the cost of not knowing what they can tell you? If an application for measuring some aspect of your network costs $100,000 but saves you $2M in project expenses and/or operational expenses over several years, it's an investment. If it saves you only $20,000, it's a waste.
How do you insure that you're spending your money on the right tool(s)? Knowing the questions to be answered goes a long way toward making the right purchasing decisions.
That's a big part of what I plan to write about: Not the modeling itself, but the questions to be asked before even gathering the information about your network. It's another side of the same process for making a business case for a network project: How do you define vague terms like "risk" and "security"? How do you boil an intangible down to something that can be measured?
--Jeff
Quantification
Hi Jeff,
I agree that the IT world needs more structure and predictability; in particular carrier networks have a lot of vulnerability to this problem since they tend to run their equipment far closer to the edge of performance limits. Growing large networks requires forethought and planning... however, corporate budget cycles frequently are rather unforgiving of this if you never planned far ahead for the requirement. This leads to the wild hand-waving and assumptions you reference above.
The finite-element analysis software modeling you reference in the BoBus 888 story is only as useful as the data it receives. Sadly, in real IT environments historical network traffic data can't be gathered and modeled quickly enough to make meaningful budget numbers. Contrast this with the BoBus model though... material elasticity curves, resonant frequencies, and fluid-dynamic properties for the components are well-known and quantifiable. This is the fundamental difference between the industries, that arguably does make networking a bit more of an art. Ours is a very young industry and I think our tools and processes have not caught up with the requirements.
The biggest practical issue is coming up with quantifiable models for how the traffic behaves and will grow over time so you can build a meaningful model. At a minimum, this must be done on a per-site basis and broken out into a src/dst matrix; sometimes even within local server farms or functional silos within a particular POP. Most companies (read IT managers and Sr Engineers) don't make this a priority until it is far too late to gather meaningful data... I think 12 months of historical data is a useful number... Furthermore, many don't have the experience in long-term protocol analysis tools and data organization to build a case that can be abstracted into yearly growth rates.
I'm in the process of building a network architecture requirements checklist, and quite honestly never even thought to add historical traffic records to my list of requirements; thank you for raising this important point of network modeling.
Cheers,
\m
Post new comment