Skip to main content

The 3 Questions Every Product Leader Should Ask When Evaluating a New AI Tool

Ranjan Goel
VP of Product
LogicMonitor

All eyes are on the value AI can provide to enterprises. Whether it's simplifying the lives of developers, more accurately forecasting business decisions, or empowering teams to do more with less, AI has already become deeply integrated into businesses. However, it's still early to evaluate its impact using traditional methods. Here's how engineering and IT leaders can make educated decisions despite the ambiguity.

1. Does my current team have the technical ability to implement this?

Even the most advanced technology won't deliver its full potential if it isn't implemented and maintained properly. Leaders must ask:

Can my existing team do this? Can we train them to do an AI implementation in a timely manner?

Or will we need to hire additional staff?

None of the answers to the above questions spell disaster for implementing AI, they do help create a clearer picture of what's possible for your specific team. Given how quickly AI is evolving, upskilling or reskilling is likely required for most organizations. Whether through training or hiring, implementation needs to be feasible.

2. Am I willing to implement this at its current stage?

AI is full of promises — some near-term, some further off. When evaluating AI vendors, it's important to recognize that the technology's current capabilities may continue to evolve rapidly. If the current proof of concept meets most of your needs, great!

Decision makers should evaluate whether the AI tool provider they're entertaining is open to working closely to iterate the tool. Most AI tools are not yet mature enough for all potential use cases to be available already.

3. So you want to move forward. How do you justify the investment?

Think of the ROI of AI as falling into two categories: business benefits and financial benefits.

Most AI tools today offer value in terms of business benefits, such as improved customer experience, enhanced employee productivity, and faster rollouts of new features or products. Businesses using AI can differentiate better from competitors as more innovative in their products and service offerings.

The other category is financial benefits, which, in addition to the above, will undoubtedly catch the attention of the C-suite and board of directors. These include factors like improved top-line growth or improving margins. Quantifying solid financial benefits from AI tools is starting to make its way, especially for domain-specific AI applications like IT operations, medical or retail. This is an area where a partnership with the AI tool vendor and decision-maker can greatly improve the quality of ROI calculation to account for key use cases.

It's rarely one person's responsibility to ask and answer all these questions. These considerations should involve the broader team and be viewed holistically. Some tools that are still in their infancy may be worth the risk if they check many of the other boxes. A more significant monetary investment could be the right choice if the technology addresses a critical need for your team that otherwise couldn't be met. Ask these questions, and reevaluate often.

Ranjan Goel is VP of Product at LogicMonitor

Hot Topics

The Latest

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

When most people think about cybersecurity, they picture firewalls, encryption, and access controls — technical tools designed to protect systems and data. But beneath the technology lies a deeper set of principles about trust, decision-making, and resilience ... The best leaders don't eliminate risk. They manage it intelligently. And in many ways, cybersecurity offers a surprisingly useful playbook for doing exactly that ...

Many organizations assumed their infrastructure strategy was settled. It had been implemented, optimized and built into long-term plans. Recent changes in technology and vendor consolidation are forcing a second look. Cloud outages and licensing changes have exposed how much dependency exists on a small number of platforms. As a result, organizations are reevaluating whether those decisions still hold up under current conditions ...

Edge AI is strategically embedded in core IT and infrastructure spending across industries, according to the 2026 Edge AI Survey from ZEDEDA. The research shows that 83% of C-suite and IT executive respondents say edge AI is important to their core business strategy ...

As AI adoption accelerates, operational complexity — not model intelligence — is becoming the primary barrier to reliable AI at scale, according to the State of AI Engineering 2026 from Datadog ... The report highlights a compounding complexity challenge as AI systems scale ... Around 5% of AI model requests fail in production, with nearly 60% of those failures caused by capacity limits ...

For years, production operations teams have treated alert fatigue as a quality-of-life problem: something that makes on-call rotations miserable but isn't considered a direct contributor to outages. That framing doesn't capture how these systems fail, and we now have data to show why. More importantly, it's now clear alert fatigue is a symptom of a deeper issue: production systems have outgrown the current operational approaches ...

I was on a customer call last fall when an enterprise architect said something I haven't been able to shake. Her team had just spent four months trying to swap one AI vendor for another. The original plan said three weeks. "We didn't switch vendors," she told me. "We rebuilt half our integrations and discovered what we'd actually been depending on." Most enterprise leaders don't expect that to be the experience ...

Ask any senior SRE or platform engineer what keeps them up at night, and the answer probably isn't the monitoring tool — it's the data feeding it. The proliferation of APM, observability, and AIOps platforms has created a telemetry sprawl problem that most teams manage reactively rather than architect proactively. Metrics are going to one platform. Traces routed somewhere else. Logs duplicated across multiple backends because nobody wants to be caught without them when something breaks. Every redundant stream costs money ...

80% of respondents agree that the IT role is shifting from operators to orchestrators, according to the 2026 IT Trends Report: The Human Side of Autonomous IT from SolarWinds ...

The 3 Questions Every Product Leader Should Ask When Evaluating a New AI Tool

Ranjan Goel
VP of Product
LogicMonitor

All eyes are on the value AI can provide to enterprises. Whether it's simplifying the lives of developers, more accurately forecasting business decisions, or empowering teams to do more with less, AI has already become deeply integrated into businesses. However, it's still early to evaluate its impact using traditional methods. Here's how engineering and IT leaders can make educated decisions despite the ambiguity.

1. Does my current team have the technical ability to implement this?

Even the most advanced technology won't deliver its full potential if it isn't implemented and maintained properly. Leaders must ask:

Can my existing team do this? Can we train them to do an AI implementation in a timely manner?

Or will we need to hire additional staff?

None of the answers to the above questions spell disaster for implementing AI, they do help create a clearer picture of what's possible for your specific team. Given how quickly AI is evolving, upskilling or reskilling is likely required for most organizations. Whether through training or hiring, implementation needs to be feasible.

2. Am I willing to implement this at its current stage?

AI is full of promises — some near-term, some further off. When evaluating AI vendors, it's important to recognize that the technology's current capabilities may continue to evolve rapidly. If the current proof of concept meets most of your needs, great!

Decision makers should evaluate whether the AI tool provider they're entertaining is open to working closely to iterate the tool. Most AI tools are not yet mature enough for all potential use cases to be available already.

3. So you want to move forward. How do you justify the investment?

Think of the ROI of AI as falling into two categories: business benefits and financial benefits.

Most AI tools today offer value in terms of business benefits, such as improved customer experience, enhanced employee productivity, and faster rollouts of new features or products. Businesses using AI can differentiate better from competitors as more innovative in their products and service offerings.

The other category is financial benefits, which, in addition to the above, will undoubtedly catch the attention of the C-suite and board of directors. These include factors like improved top-line growth or improving margins. Quantifying solid financial benefits from AI tools is starting to make its way, especially for domain-specific AI applications like IT operations, medical or retail. This is an area where a partnership with the AI tool vendor and decision-maker can greatly improve the quality of ROI calculation to account for key use cases.

It's rarely one person's responsibility to ask and answer all these questions. These considerations should involve the broader team and be viewed holistically. Some tools that are still in their infancy may be worth the risk if they check many of the other boxes. A more significant monetary investment could be the right choice if the technology addresses a critical need for your team that otherwise couldn't be met. Ask these questions, and reevaluate often.

Ranjan Goel is VP of Product at LogicMonitor

Hot Topics

The Latest

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

When most people think about cybersecurity, they picture firewalls, encryption, and access controls — technical tools designed to protect systems and data. But beneath the technology lies a deeper set of principles about trust, decision-making, and resilience ... The best leaders don't eliminate risk. They manage it intelligently. And in many ways, cybersecurity offers a surprisingly useful playbook for doing exactly that ...

Many organizations assumed their infrastructure strategy was settled. It had been implemented, optimized and built into long-term plans. Recent changes in technology and vendor consolidation are forcing a second look. Cloud outages and licensing changes have exposed how much dependency exists on a small number of platforms. As a result, organizations are reevaluating whether those decisions still hold up under current conditions ...

Edge AI is strategically embedded in core IT and infrastructure spending across industries, according to the 2026 Edge AI Survey from ZEDEDA. The research shows that 83% of C-suite and IT executive respondents say edge AI is important to their core business strategy ...

As AI adoption accelerates, operational complexity — not model intelligence — is becoming the primary barrier to reliable AI at scale, according to the State of AI Engineering 2026 from Datadog ... The report highlights a compounding complexity challenge as AI systems scale ... Around 5% of AI model requests fail in production, with nearly 60% of those failures caused by capacity limits ...

For years, production operations teams have treated alert fatigue as a quality-of-life problem: something that makes on-call rotations miserable but isn't considered a direct contributor to outages. That framing doesn't capture how these systems fail, and we now have data to show why. More importantly, it's now clear alert fatigue is a symptom of a deeper issue: production systems have outgrown the current operational approaches ...

I was on a customer call last fall when an enterprise architect said something I haven't been able to shake. Her team had just spent four months trying to swap one AI vendor for another. The original plan said three weeks. "We didn't switch vendors," she told me. "We rebuilt half our integrations and discovered what we'd actually been depending on." Most enterprise leaders don't expect that to be the experience ...

Ask any senior SRE or platform engineer what keeps them up at night, and the answer probably isn't the monitoring tool — it's the data feeding it. The proliferation of APM, observability, and AIOps platforms has created a telemetry sprawl problem that most teams manage reactively rather than architect proactively. Metrics are going to one platform. Traces routed somewhere else. Logs duplicated across multiple backends because nobody wants to be caught without them when something breaks. Every redundant stream costs money ...

80% of respondents agree that the IT role is shifting from operators to orchestrators, according to the 2026 IT Trends Report: The Human Side of Autonomous IT from SolarWinds ...