Skip to main content

What to Look For in an Analyst Report

Jonah Kowall

The Internet is an amazing medium for anyone looking to articulate an opinion. Everyone should practice writing and expressing themselves, as it's a great tool to build throughout life. The ability to publish information is everyone's right. However, credibility is a whole different challenge. Why do we trust what is published in the New York Times, but don't trust what is published in the tabloids? It comes down to rigor in journalistic practices. In research, it's also tied to a strict methodology or process.

During my time at Gartner, I learned a great deal about the differences between analyst firms, mostly by meeting and discussing things with friends at other analyst firms. I quickly learned which firms will write a whitepaper, and which firms will not create marketing materials for software companies for pay.

In the case of Gartner, the analyst has the freedom to publish anything, if they follow the extremely rigorous research process and can defend the opinion they are creating as an analyst. The process for analysts to publish branded documents, such as a Magic Quadrant, is over 30 pages, but publicly a small subset of this is disclosed. Clients get another deeper look in this document (Gartner subscribers only).

During the publication process for any document, there is rigorous peer review, management review, and editing to handle any issues in process or the fact base. Aside from this process and methodology, the analyst speaks with hundreds of end users of a particular technology through the year on phone calls and at conferences. This allows the analyst to comprehend the reality of a market versus what vendors may care to share with an analyst.

When witnessing small analyst firms attempting to assess markets without end user perspective and without speaking to all the vendors in the research — while blatantly requesting and collecting money directly from vendors before research is even drafted — the red flags come up. I discourage any organization from participating in these blatant acts of extortion. When vendors sponsor and fund this research, it just enables the lie to persist, year after year. This is clearly a major violation of journalistic integrity. The vendors who pay continually jam this poorly crafted research down end users' throats, and avoid the questions about where it comes from. This occurs regularly.

The violations don't just stop there, but clearly there are researchers who infringe on specific research formats, whether it's the Gartner Magic Quadrant, the Forrester Wave, or the IDC MarketScape. The lawyers of the respective firms get involved in these disputes, but small single person "analyst firms" seem to do this regularly and get slapped with cease and desist letters.

Leading researchers in the public domain should be able to publicly discuss, dispute, and learn from others in public forums. You'll see top analyst firms participate in conferences, panels, LinkedIn groups, Twitter, and other public forums. If a researcher is going to build a fact-based opinion of something, they should be able to participate and defend that position. Many of these smaller analyst firms or independent researchers avoid doing so, continually hide, or many times do not even have a name associated with research. I give big kudos and credibility to the researchers and analysts who stand behind what they publish.

The Latest

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 5 covers the infrastructure and hardware supporting AI ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 4 covers advancements in AI technology ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 3 covers AI's impact on employees and their roles ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 2 covers the challenges presented by AI, as well as solutions to those problems ...

In the final part of APMdigest's 2025 Predictions Series, industry experts offer predictions on how AI will evolve and impact technology and business in 2025 ...

E-commerce is set to skyrocket with a 9% rise over the next few years ... To thrive in this competitive environment, retailers must identify digital resilience as their top priority. In a world where savvy shoppers expect 24/7 access to online deals and experiences, any unexpected downtime to digital services can lead to significant financial losses, damage to brand reputation, abandoned carts with designer shoes, and additional issues ...

Efficiency is a highly-desirable objective in business ... We're seeing this scenario play out in enterprises around the world as they continue to struggle with infrastructures and remote work models with an eye toward operational efficiencies. In contrast to that goal, a recent Broadcom survey of global IT and network professionals found widespread adoption of these strategies is making the network more complex and hampering observability, leading to uptime, performance and security issues. Let's look more closely at these challenges ...

Image
Broadcom

The 2025 Catchpoint SRE Report dives into the forces transforming the SRE landscape, exploring both the challenges and opportunities ahead. Let's break down the key findings and what they mean for SRE professionals and the businesses relying on them ...

Image
Catchpoint

The pressure on IT teams has never been greater. As data environments grow increasingly complex, resource shortages are emerging as a major obstacle for IT leaders striving to meet the demands of modern infrastructure management ... According to DataStrike's newly released 2025 Data Infrastructure Survey Report, more than half (54%) of IT leaders cite resource limitations as a top challenge, highlighting a growing trend toward outsourcing as a solution ...

Image
Datastrike

Gartner revealed its top strategic predictions for 2025 and beyond. Gartner's top predictions explore how generative AI (GenAI) is affecting areas where most would assume only humans can have lasting impact ...

What to Look For in an Analyst Report

Jonah Kowall

The Internet is an amazing medium for anyone looking to articulate an opinion. Everyone should practice writing and expressing themselves, as it's a great tool to build throughout life. The ability to publish information is everyone's right. However, credibility is a whole different challenge. Why do we trust what is published in the New York Times, but don't trust what is published in the tabloids? It comes down to rigor in journalistic practices. In research, it's also tied to a strict methodology or process.

During my time at Gartner, I learned a great deal about the differences between analyst firms, mostly by meeting and discussing things with friends at other analyst firms. I quickly learned which firms will write a whitepaper, and which firms will not create marketing materials for software companies for pay.

In the case of Gartner, the analyst has the freedom to publish anything, if they follow the extremely rigorous research process and can defend the opinion they are creating as an analyst. The process for analysts to publish branded documents, such as a Magic Quadrant, is over 30 pages, but publicly a small subset of this is disclosed. Clients get another deeper look in this document (Gartner subscribers only).

During the publication process for any document, there is rigorous peer review, management review, and editing to handle any issues in process or the fact base. Aside from this process and methodology, the analyst speaks with hundreds of end users of a particular technology through the year on phone calls and at conferences. This allows the analyst to comprehend the reality of a market versus what vendors may care to share with an analyst.

When witnessing small analyst firms attempting to assess markets without end user perspective and without speaking to all the vendors in the research — while blatantly requesting and collecting money directly from vendors before research is even drafted — the red flags come up. I discourage any organization from participating in these blatant acts of extortion. When vendors sponsor and fund this research, it just enables the lie to persist, year after year. This is clearly a major violation of journalistic integrity. The vendors who pay continually jam this poorly crafted research down end users' throats, and avoid the questions about where it comes from. This occurs regularly.

The violations don't just stop there, but clearly there are researchers who infringe on specific research formats, whether it's the Gartner Magic Quadrant, the Forrester Wave, or the IDC MarketScape. The lawyers of the respective firms get involved in these disputes, but small single person "analyst firms" seem to do this regularly and get slapped with cease and desist letters.

Leading researchers in the public domain should be able to publicly discuss, dispute, and learn from others in public forums. You'll see top analyst firms participate in conferences, panels, LinkedIn groups, Twitter, and other public forums. If a researcher is going to build a fact-based opinion of something, they should be able to participate and defend that position. Many of these smaller analyst firms or independent researchers avoid doing so, continually hide, or many times do not even have a name associated with research. I give big kudos and credibility to the researchers and analysts who stand behind what they publish.

The Latest

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 5 covers the infrastructure and hardware supporting AI ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 4 covers advancements in AI technology ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 3 covers AI's impact on employees and their roles ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 2 covers the challenges presented by AI, as well as solutions to those problems ...

In the final part of APMdigest's 2025 Predictions Series, industry experts offer predictions on how AI will evolve and impact technology and business in 2025 ...

E-commerce is set to skyrocket with a 9% rise over the next few years ... To thrive in this competitive environment, retailers must identify digital resilience as their top priority. In a world where savvy shoppers expect 24/7 access to online deals and experiences, any unexpected downtime to digital services can lead to significant financial losses, damage to brand reputation, abandoned carts with designer shoes, and additional issues ...

Efficiency is a highly-desirable objective in business ... We're seeing this scenario play out in enterprises around the world as they continue to struggle with infrastructures and remote work models with an eye toward operational efficiencies. In contrast to that goal, a recent Broadcom survey of global IT and network professionals found widespread adoption of these strategies is making the network more complex and hampering observability, leading to uptime, performance and security issues. Let's look more closely at these challenges ...

Image
Broadcom

The 2025 Catchpoint SRE Report dives into the forces transforming the SRE landscape, exploring both the challenges and opportunities ahead. Let's break down the key findings and what they mean for SRE professionals and the businesses relying on them ...

Image
Catchpoint

The pressure on IT teams has never been greater. As data environments grow increasingly complex, resource shortages are emerging as a major obstacle for IT leaders striving to meet the demands of modern infrastructure management ... According to DataStrike's newly released 2025 Data Infrastructure Survey Report, more than half (54%) of IT leaders cite resource limitations as a top challenge, highlighting a growing trend toward outsourcing as a solution ...

Image
Datastrike

Gartner revealed its top strategic predictions for 2025 and beyond. Gartner's top predictions explore how generative AI (GenAI) is affecting areas where most would assume only humans can have lasting impact ...