First of all, I want to say that I have tremendous respect for Gartner, the $1+ billion IT analyst firm that has been described as the most important technology advisor to the Fortune 500. Gartner covers all of IT with continuous information services, analyst briefings, and consulting services. It needs to be mentioned at the start here that technically Farpoint Group and Gartner are competitors, but the overlap between what they and we do is tiny. I don't think we've ever competed directly for the same opportunity. Farpoint Group's business services cover just a small portion of what Gartner does, and, of course, we only work only in wireless and mobile. Really, we do business entirely differently from one another.
But while I generally do not criticize the competition, I must take issue with what has become one of Gartner's trademarks, the Magic Quadrant. Not a week goes by that I don't receive a call about how one's position in this Cartesian comparison might be improved. I don't have an answer to that question, and such motivated this entry.
Magic Quadrant is (at first glance, anyway) a very simple methodology for competitive comparison, positioning suppliers on two axes, Ability to Execute (Y-axis) and Completeness of Vision (X-axis). While one might be tempted to simply select the supplier upmost and farthest to the right (i.e., the best positioned in what is called the "Leaders Quadrant", described as those who "execute well against their current vision and are well positioned for tomorrow"), even Gartner cautions against this:
Keep in mind that focusing on the leaders' quadrant isn't always the best course of action. There are good reasons to consider market challengers. And a niche player may support your needs better than a market leader. It all depends on how the provider aligns with your business goals.
I couldn't agree more, but consider the following as well. With respect to the other quadrants:
- "Visionaries [lower right] understand where the market is going or have a vision for changing market rules, but do not yet execute well."
- "Niche Players [lower left] focus successfully on a small segment, or are unfocused and do not out-innovate or outperform others."
- "Challengers [upper left] execute well today or may dominate a large segment, but do not demonstrate an understanding of market direction."
Well, none of these sound like a ringing endorsement to me. Would you buy from a company that doesn't execute well (whatever that means, by the way), is unfocused, not innovative, or has no understanding of market direction (again, whatever that is - but it sounds bad regardless)? In fact, it doesn't seem like any player in other than the Leader's Quadrant would even be worth consideration. Based on my experience (having used or worked with products from almost every vendor in the enterprise WLAN space), though, such is far from the truth. What about throughput, capacity, scalability, support for time-bounded traffic, management features, and on and on? Does Gartner do a comprehensive technical comparison under "Ability to Execute"? Is "completeness of Vision" even a valid criterion for comparison in a field where both technologies and products, to say nothing of underlying standards, continue to evolve rapidly? I think, then, that the Magic Quadrant approach oversimplifies to the point of inconsequence.
So, while the Cartesian coordinate system used is beyond question, I'm missing the Magic part. In fact, I'm not sure there's much value here at all. This conclusion is important, because I have seen WLAN purchasing decisions based largely on the Magic Quadrant methodology alone ("if Gartner likes it, it must be good"), and I think anyone who uses this single point of (questionable) data as the basis for such an incredibly important decision may be doing themselves a serious disservice. Even Gartner admits to the limitations if you read carefully. And, of course, one has to agree with the methodology in the first place - and I'm not the first to post question here; Magic Quadrant has been frequently criticized in the press - see here for just one example. And if you want to have a look at a sample report and make up your own mind, you can see one here courtesy of Aruba Networks. It's an interesting read regardless - but suitable as a key element in making an important infrastructure purchasing decision? I think not.
Now, I know it's important in busy, time-challenged industrialized societies to reduce the criteria for any given decision to the minimal number of elements required. We do this all the time in essentially all aspects of our lives - a semester's or even a year's worth of classroom time, for example, gets reduced to a letter grade. Four years of college is summarized in a GPA. That hot (or not) person across the bar quickly ends up somewhere on a scale of 1 to 10, especially as the 2:00 AM deadline approaches. This penchant for simplicity and rapidity in decision-making may in fact be an end-result of evolution so far - when fight-or-flight comes into play, one doesn't call a meeting, set up a multi-page decision tree, and have a review session before the final decision is made. Time is of the essence, and survival can depend on it. Besides, if someone else has already done the thinking, why not just go with the flow? That, by the way, is herd mentality, and you can see it in action on Wall Street any business day.
But, and speaking as a member of the profession, one should never simply take the word of an analyst with a cleaver but simplistic analysis methodology when making a decision of business-critical importance. We (analysts) offer opinions (with, again, the reasoning behind those opinions far more important than the opinions themselves) based on knowledge, experience, and focus, but we can't reduce purchasing decision of business-critical importance to picking the answer off an X-Y plane. And wireless LAN purchasing decisions need to be made, IMHO, on factors well beyond an analyst's opinion of ability to execute and completeness of vision, again whatever those might be.
Indeed, I'm often concerned that too many analysts, even at the big firms, have far too little hands-on time with the products that they recommend, instead relying on conversations with the vendors themselves. They also often have too little experience in the demands of real-world applications, as well as in network operations. In short, real-world experience is the most important element, I've found, in making a good analyst - and in making the right purchasing decision.
And real-world experience dictates, in my view, anyway, that a subjective analysis of two ill-defined variables for which only incomplete data can exist in many if not most cases makes for much less than what's required to pick the right wireless LAN - and, for that matter, any other element of the network or IT arsenal.