Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Is Magic Quadrant data reliable?

With a growing number of online resources, is the Magic Quadrant still the No. 1 source of vendor information?

Is the Gartner Magic Quadrant still a reliable source for vendor information?

The Gartner Magic Quadrant has been around for years. Gartner analysts place vendors within the quadrant using a formula that's partially based on customer interviews -- customers that are largely supplied by the vendors themselves. End-user customers subscribe to Gartner analyses in an effort to gain market information for their procurement practices. In IT, no one wants to be the one to buy the wrong technology. But these days, there are many ways to learn about new technology without the Magic Quadrant analysis.

If you scour the Internet, you can find many predictions that Gartner and the other think tanks have gotten completely wrong, and some that were spot on the mark. As a matter of fact, sometimes it is quite humorous to read their dueling reports. I presented on the topic of hits and misses at a conference in Australia a few years ago. It was one of the funnier presentations I have had the pleasure to deliver.

Founded in 1979, Gartner now comprises several divisions, including consulting and analysis sectors. As a CEO or CIO, it is a task to learn about new technologies and products that may impact your business. The Magic Quadrant has become a way to determine who the mature players are in the market, reveal upcoming or visionary players, identify niche players, and pinpoint challengers to the leaders. But how does a company get onto the Magic Quadrant?

The details are proprietary, of course, but in order to catch an analyst's ear, there is, at a minimum, time involved. One must have customers that can be interviewed that will say nice things, as a company obviously wouldn't put forth a customer to provide a bad reference. The analysts then act on what they believe to be pertinent information. While this was a great thing before the Internet and social media, the amount of information about a particular vendor or product available today makes Gartner's analysis-less subscription less distinctive -- not to mention the subscription's annual costs. Furthermore, the Magic Quadrant is reviewed every year or two, which means that in some cases, there is a substantial gap between new reports. Should you happen to peer at the Quadrant in the second year of its existence, you may be missing out on some great visionary or niche players that are worthy of consideration.

We all know technology changes rapidly. Companies are bought and sold regularly. Is there really anything new that happens in the leaders quadrant? Not that often. The information to gain is what happens in the other quadrants, but Gartner certainly isn't the only definitive source. There are many think tanks. There are also peer-based sites and conferences. Ombud.com, LinkedIn and other sites provide forums for common discussion (good and bad).

Let's face it; the main players have been around for a long time. I personally, however, have been involved in evaluations, only to find that in some cases there are far superior products out there than those reflected in the leaders quadrant -- based on various criteria such as packets passed, wire speed, price power and others.

If you do rely on the Magic Quadrant, I think you are missing out on some potentially amazing technology that might not have come to the attention of the right analysts to be included. Do your homework and check other sites, peers, etc. After all, if you remember, back in school, every reference paper had to have multiple references. References are so easy to come by these days with the Internet and social media. I say, let the quadrant be a tool if you choose, not the only tool.

This was last published in June 2014

Dig Deeper on Network Infrastructure

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Do you rely on the Gartner Magic Quadrant for vendor information, or do you also research other sources like Ombud or LinkedIn?
I think you've missed the point with your article. People have misunderstood the intention of the Magic Quadrant. If you do some fact-finding on the Garner site, you'll find the following documentation, "Magic Quadrants and MarketScopes: How Gartner Evaluates Vendors Within a Market." On the first page it states the following: "Magic Quadrants and MarketScopes offer visual snapshots of a market’s direction, maturity and participants." I don’t interpret anything in that statement to indicate that the Magic Quadrant is anything more than a market analysis. Anyone who uses it without testing and validation in their own environment, clearly doesn't understand the purpose of market analysis. There's even a section in this document that instructs one how to use it: "Your needs and circumstances should determine how you use the Magic Quadrant, not the other way around. To evaluate vendors in the Leaders quadrant only and ignore those in other quadrants is risky and thus discouraged....Use a Magic Quadrant to narrow your list of choices, but don’t base your decision only on the model." People misuse the Magic Quadrant in an attempt to shortcut the challenging process of evaluating and building solutions. It's intellectual laziness and Gartner becomes a convenient scapegoat for those professionals whose scientific rigor is no better than Cult Cargo Science.