Analyzing the analysts

Here's what you need to consider when choosing an IT research provider.

By ANTHEA STRATIGOS
Network World, 04/17/00

When the health of your network is on the line, sometimes it's prudent to get a second opinion of the diagnosis before you operate. One of the specialists to turn to for a consultation is the IT research provider.

An IT research firm provides information, analysis and advice that helps companies track technology trends, benchmark operations, and identify and evaluate vendors. It's a bit ironic that before you can make any decisions, you'll need to pick an advisory service.

Evaluation advice
The analysts
  More info on IT research providers.
Different priorities
  IT pros vs. IT vendors
Selection criteria
  a comparison chart

If you work for a big company that has lots of high-risk technology decisions at stake, you'll most likely select two or three IT research firms to rely on. No one firm has the corner on advice, so buyers of research generally use a portfolio of services to compare opinions and data.

There are plenty of choices, ranging from large, influential companies to boutiques with highly specialized knowledge and depth of content and expertise. Each has its own strengths and weaknesses. To help you sort through the options, we've assembled a basic guide to the IT research players that focus on various aspects of networking. While this chart isn't an all-inclusive list, it features the best-known firms that are worth evaluating.

An important factor to consider is that IT research firms have two major audiences: IT users and IT vendors. The leading companies got their start serving one group or the other, and these roots drive the way the firms operate. To find out if a firm's content and advice can meet your requirements or if it's better-suited to the needs of vendors, go to Network World Fusion, DocFinder 7728, to examine a list of attributes.

Paying for information

IT professionals are generally satisfied with their IT research services, according to Outsell, a company in Burlingame, Calif., that tracks the information industry. The research providers aren't loved, nor are they despised.

Many high-tech executives have come to rely on IT research companies as a form of security - a trusted and neutral external resource for substantiating or bolstering a particular decision. While some clients complain about pricing, analyst availability or unprofessional salespeople, few are willing to do without IT research altogether.

The average company spends about 2% of its IT budget on research, with some variations depending on the overall size of the company. This is a substantial investment for many companies. If you're worried that an IT research firm may be out of your reach, there are several attractively priced offerings for small firms to consider.

Meta Group, for example, offers substantial discounts for small businesses. Other companies offer bundling deals that represent big savings. Most of the time, you can also find cost-effective alternatives if you don't need access to analysts.

But the bad news is that pricing is all over the map. Standard pricing is a concept that receives little more than lip service, and most IT research subscribers pay much less than list price. Discounting has created a more difficult buying process because clients are often left with the feeling that if they had done something else, they might have gotten a better deal. What's more, complex packaging options make it difficult to compare services.

Shopping list

Here are some tips to keep in mind when you're shopping for IT research services:

Know your needs and bottom line. Be realistic about the number of services you require, the topics covered, how many people need the data, and what type of inquiry privileges you need.

Coordinate and consolidate subscriptions within and across IT research firms to leverage buying power.

Move subscription renewals as close as possible to the same time and ideally have that time coincide with the end of the month, quarter or fiscal year. Companies sometimes slash prices when their salespeople are pressed to meet quotas.

Go out to bid and generate a request for proposal (RFP) where required. The RFP not only reinforces that you've clarified your needs, but gives the supplier concrete objectives to address. Compare pricing with publicly available information. For example, Gartner Group publishes General Services Administration pricing, which represents its best price in the marketplace.

Play with bundling. Sometimes a firm that doesn't want to discount its price will give you concessions such as additional content or access to analysts.

Evaluate alternative sources. The Internet is creating competition for this part of the information industry. New entrants pop up every day. Trade publications such as Network World have created robust, searchable Web sites rich with content.

Last but not least, remember you don't get what you don't ask for.

Different priorities
Here are some ways in which IT executives and vendors might use an IT research service.

IT PROS
(Vice President of IT, Network Director, CIO) IT VENDORS
(Vice President of Marketing, Product Development Director)

Identify and locate vendors. Analyze markets; develop market opportunity assessments.

Assess vendor features and benefits; evaluate and select technology. Track competitors.

Procure and negotiate hardware and software contracts. Identify and size markets; track market share and brand awareness.

Implement technology. Establish pricing.

Manage and benchmark operations and application development. Test and launch new products; perform market research; support sales rollouts.

Track technology trends; develop technology strategy. Establish and implement advertising, public relations and marketing communications programs.

SELECTION CRITERIA

Attribute Priorities for IT professionals Priorities for IT vendors

Time sensitivity Everything moves at lightning speed. Having accurate vendor and pricing and trouble-shooting information is critical. Once the purchase decision is made, time sensitivity wanes. Tracking competitive movements is important given the fast-changing industry. Information ages quickly.

Packaging Reports and briefings combined with analyst inquiry is critical. This allows for bottom-line opinion and dialogue during critical decision-making periods. Access to quantitative data and timely news with analysis is critical. Analyst inquiry a bonus, but less important to this audience.

Medium All media work. With tons of information coming at the IT professional, quick e-mail and phone access to analysts a plus. �Push� in the form of proactive analyst calls and print and e-mail with synthesized analysis is also good. This group doesn�t have time to search. Searchable, electronic access to tabular data a plus. Historical use of print format made access to the data difficult and cumbersome.

Quality Trusted opinion; can the user have confidence in the expert�s opinion; what facts back up the opinion? Statistically correct methods less important to ensure credible data. Benchmarking and technology assessment methodologies must be in place and defensible. Trusted numbers and market share, forecasting and customer tracking methodologies. Being able to explain defensible, quantitative methodologies a must.

Delivery
frequency Up-to-date pricing and benchmarking information important. This group mostly wants the data when needed. Demand driven. Most of the market information is retrospective. Thus quarterly market share is better than annual market share, because the snapshot of the market isn�t as old. Forecasts based on historical market shares are also deemed more accurate.

Coverage
period Historical and forecast information less relevant to this marketplace. Ability to perform retrospective research important to building trend analysis. Historical data a plus. Must forecast current year and years 2-5 to be considered a player.

Subject
coverage Ability to cover many technologies a plus. Ability to understand IT deployment relative to certain vertical industries a bonus. Must be able to cover product categories at a granular level. Ideal vendor can cover technology, market and competitor trends, equally. Few firms have demonstrated ability to do this well.

Geographic
coverage Geographic coverage less important except for understanding the new vendors on the block and who they may be aligned with. (Other user implementations and alliances add credibility; tracking this is important) Being able to serve vendors in all regions and giving them data about all the regions they want to sell to is critical.

Subjectivity/
objectivity Opinion-based orientations appropriate for the buyers. Ability to tie opinion to benchmarking or pricing data a bonus. Important to consider whether IT research vendor will state negative opinions about IT vendors who are part of their clientele. Also at issue: will they cover companies who are not clients? They should, but sometimes don�t. IT vendors care less about opinion and more about fact. Measurement must be fact-based using statistically correct and defensible methodologies. These methodologies are by nature more rigorous than those required for benchmarking.

Sources and
research methodology Rely on numerous secondary sources. Personal interviews with industry contacts, clients, and IT vendors. Benchmarking relies on input from participating members Rely on numerous secondary sources with heavy emphasis on 10K, 10Q and annual reports. Also reliant on Dun & Bradstreet data. Personal interviews with industry contacts, and clients. Much of the data for market share provided by vendors about vendor�s shipments -- a shortcoming of the entire process. Demand-based market share largely absent in the industry. Customer loyalty and brand tracking based on statistically correct quantitative surveys.

Pricing and
price vs.
budget Continuous subscription pricing with ongoing access to analysts a good model for this group. Heritage in print giving designated access to a specific user has made pricing models for electronic delivery cumbersome. Same as IT user. Many clients wanted access to numbers without access to analysts. Required more flexible pricing and unbundling strategies in order to penetrate a more limited market.

Sales methods and sales professional�s knowledge High-priced service requires field sales, usually supported by telemarketing which generates leads. Lower-priced information products sold using other channels. Same as IT user. Requires a field sales force savvy in how clients use their data and how data is differentiated from competitors.

Corporate revenue and number of employees and analysts Influence matters. A user must know that their IT research vendor covers all the technologies that are relevant and won�t miss any of the relationships between technology alternatives. Clout with the vendors matters and serving a large user base is important. Larger IT research vendors who serve IT vendors need clout in order to influence vendors to supply their data. Size is an indicator that the market share data is both comprehensive and reportable to a large audience.

Corporate culture Knowledge workers are at the heart of these firms� success. A culture that�s antagonistic to its staff can mean hiring or turnover problems; a critical issue for marketing and providing high-level, credible, consistent expertise. Hiring philosophies indicate the type of analysts hired. Same as IT user. Consistency matters because the numbers must �hang together� year to year; a problem if analysts turn over. Industry expertise almost always required.

Related links

Stratigos is co-founder and president of Outsell, a Burlingame, Calif., research firm that focuses exclusively on the information industry. She can be contacted at astratigos@outsellinc.com

Evaluation advice
Follow these tips for assessing if an IT research provider can meet your needs.

The analysts
More info on IT research providers.