Skip to main content

The 3 Questions Every Product Leader Should Ask When Evaluating a New AI Tool

Ranjan Goel
VP of Product
LogicMonitor

All eyes are on the value AI can provide to enterprises. Whether it's simplifying the lives of developers, more accurately forecasting business decisions, or empowering teams to do more with less, AI has already become deeply integrated into businesses. However, it's still early to evaluate its impact using traditional methods. Here's how engineering and IT leaders can make educated decisions despite the ambiguity.

1. Does my current team have the technical ability to implement this?

Even the most advanced technology won't deliver its full potential if it isn't implemented and maintained properly. Leaders must ask:

Can my existing team do this? Can we train them to do an AI implementation in a timely manner?

Or will we need to hire additional staff?

None of the answers to the above questions spell disaster for implementing AI, they do help create a clearer picture of what's possible for your specific team. Given how quickly AI is evolving, upskilling or reskilling is likely required for most organizations. Whether through training or hiring, implementation needs to be feasible.

2. Am I willing to implement this at its current stage?

AI is full of promises — some near-term, some further off. When evaluating AI vendors, it's important to recognize that the technology's current capabilities may continue to evolve rapidly. If the current proof of concept meets most of your needs, great!

Decision makers should evaluate whether the AI tool provider they're entertaining is open to working closely to iterate the tool. Most AI tools are not yet mature enough for all potential use cases to be available already.

3. So you want to move forward. How do you justify the investment?

Think of the ROI of AI as falling into two categories: business benefits and financial benefits.

Most AI tools today offer value in terms of business benefits, such as improved customer experience, enhanced employee productivity, and faster rollouts of new features or products. Businesses using AI can differentiate better from competitors as more innovative in their products and service offerings.

The other category is financial benefits, which, in addition to the above, will undoubtedly catch the attention of the C-suite and board of directors. These include factors like improved top-line growth or improving margins. Quantifying solid financial benefits from AI tools is starting to make its way, especially for domain-specific AI applications like IT operations, medical or retail. This is an area where a partnership with the AI tool vendor and decision-maker can greatly improve the quality of ROI calculation to account for key use cases.

It's rarely one person's responsibility to ask and answer all these questions. These considerations should involve the broader team and be viewed holistically. Some tools that are still in their infancy may be worth the risk if they check many of the other boxes. A more significant monetary investment could be the right choice if the technology addresses a critical need for your team that otherwise couldn't be met. Ask these questions, and reevaluate often.

Ranjan Goel is VP of Product at LogicMonitor

Hot Topics

The Latest

Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...

 

Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...

When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...

If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...

Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...

AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...

Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...

Enterprise networks rarely behave the same way for very long. A routing adjustment in one region may unexpectedly alter application performance in another. A cloud migration may introduce hidden dependencies that go unnoticed until an outage occurs. All the while, the network is managed by several different teams, each of whom use different tool sets — and as a result, have different views of the network ... There’s usually an engineer who remembers why traffic fails over a certain way between sites, or which transparent firewall was added where. The problem is that human memory cannot scale alongside enterprise-scale networks ...

Ask an infrastructure team how confident they are in their ability to govern AI, and most will tell you they've got it handled. A recent survey of 406 IT decision-makers and platform engineering leaders found 86% expressing exactly that confidence. Ask the same group whether they have a formal written AI governance policy, and the number drops to 30%, according to Spacelift's Infrastructure Automation Report ...

In MEAN TIME TO INSIGHT Episode 27, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and Parker Hathcock, EMA Research Director covering IT Service/Operations (ServiceOps), discuss observability unification in modern IT operations ... 

The 3 Questions Every Product Leader Should Ask When Evaluating a New AI Tool

Ranjan Goel
VP of Product
LogicMonitor

All eyes are on the value AI can provide to enterprises. Whether it's simplifying the lives of developers, more accurately forecasting business decisions, or empowering teams to do more with less, AI has already become deeply integrated into businesses. However, it's still early to evaluate its impact using traditional methods. Here's how engineering and IT leaders can make educated decisions despite the ambiguity.

1. Does my current team have the technical ability to implement this?

Even the most advanced technology won't deliver its full potential if it isn't implemented and maintained properly. Leaders must ask:

Can my existing team do this? Can we train them to do an AI implementation in a timely manner?

Or will we need to hire additional staff?

None of the answers to the above questions spell disaster for implementing AI, they do help create a clearer picture of what's possible for your specific team. Given how quickly AI is evolving, upskilling or reskilling is likely required for most organizations. Whether through training or hiring, implementation needs to be feasible.

2. Am I willing to implement this at its current stage?

AI is full of promises — some near-term, some further off. When evaluating AI vendors, it's important to recognize that the technology's current capabilities may continue to evolve rapidly. If the current proof of concept meets most of your needs, great!

Decision makers should evaluate whether the AI tool provider they're entertaining is open to working closely to iterate the tool. Most AI tools are not yet mature enough for all potential use cases to be available already.

3. So you want to move forward. How do you justify the investment?

Think of the ROI of AI as falling into two categories: business benefits and financial benefits.

Most AI tools today offer value in terms of business benefits, such as improved customer experience, enhanced employee productivity, and faster rollouts of new features or products. Businesses using AI can differentiate better from competitors as more innovative in their products and service offerings.

The other category is financial benefits, which, in addition to the above, will undoubtedly catch the attention of the C-suite and board of directors. These include factors like improved top-line growth or improving margins. Quantifying solid financial benefits from AI tools is starting to make its way, especially for domain-specific AI applications like IT operations, medical or retail. This is an area where a partnership with the AI tool vendor and decision-maker can greatly improve the quality of ROI calculation to account for key use cases.

It's rarely one person's responsibility to ask and answer all these questions. These considerations should involve the broader team and be viewed holistically. Some tools that are still in their infancy may be worth the risk if they check many of the other boxes. A more significant monetary investment could be the right choice if the technology addresses a critical need for your team that otherwise couldn't be met. Ask these questions, and reevaluate often.

Ranjan Goel is VP of Product at LogicMonitor

Hot Topics

The Latest

Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...

 

Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...

When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...

If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...

Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...

AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...

Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...

Enterprise networks rarely behave the same way for very long. A routing adjustment in one region may unexpectedly alter application performance in another. A cloud migration may introduce hidden dependencies that go unnoticed until an outage occurs. All the while, the network is managed by several different teams, each of whom use different tool sets — and as a result, have different views of the network ... There’s usually an engineer who remembers why traffic fails over a certain way between sites, or which transparent firewall was added where. The problem is that human memory cannot scale alongside enterprise-scale networks ...

Ask an infrastructure team how confident they are in their ability to govern AI, and most will tell you they've got it handled. A recent survey of 406 IT decision-makers and platform engineering leaders found 86% expressing exactly that confidence. Ask the same group whether they have a formal written AI governance policy, and the number drops to 30%, according to Spacelift's Infrastructure Automation Report ...

In MEAN TIME TO INSIGHT Episode 27, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and Parker Hathcock, EMA Research Director covering IT Service/Operations (ServiceOps), discuss observability unification in modern IT operations ...