New Relic Integrates with NVIDIA NIM
June 24, 2024
Share this

New Relic is integrating its platform with NVIDIA NIM inference microservices to reduce the complexity and costs of developing, deploying, and monitoring generative AI (GenAI) apps.

Customers can use New Relic AI monitoring to gain broad visibility across the AI stack for applications built with NVIDIA NIM, all with a simplified setup and enhanced data security. This complements the security features and ease of use of NVIDIA NIM’s self-hosted models, which accelerates generative AI application delivery. Together, New Relic integrated with NVIDIA NIM can help customers adopt AI faster and achieve quicker ROI.

“In today’s hyper-competitive market, organizations cannot afford to wait years for AI ROI,” said New Relic CEO Ashan Willy. “Observability solves this by providing visibility across the AI stack. We are pioneering AI observability by extending our platform to include AI apps built with NVIDIA NIM. Combining NVIDIA’s AI technology with our expertise in observability and APM gives enterprises a competitive edge in the AI race."

“As enterprises race to adopt generative AI, NVIDIA NIM can help businesses quickly deploy applications in production,” said NVIDIA Director of AI Software Amanda Saunders. “New Relic’s integration with NVIDIA NIM enables IT and development teams to optimize their AI applications by rapidly observing and responding to operational insights.”

New Relic AI monitoring provides a broad view of the AI stack, along with key metrics on throughput, latency, and costs while ensuring data privacy. It also traces the request flows across services and models to understand the inner workings of AI apps. New Relic extends its in-depth monitoring to NVIDIA NIM, supporting a wide range of AI models including–Databricks DBRX, Google's Gemma, Meta's Llama 3, Microsoft's Phi-3, Mistral Large and Mixtral 8x22B, and Snowflake's Arctic. This helps organizations deploy AI applications built with NVIDIA NIM confidently, accelerate time-to-market, and improve ROI.

Key features and use cases for AI monitoring include:

- Full AI stack visibility: Spot issues faster with a view across apps, NVIDIA GPU-based infrastructure, AI layer, response quality, token count, and APM golden signals.

- Deep trace insights for every response: Fix performance and quality issues like bias, toxicity, and hallucinations by tracing the lifecycle of AI responses.

- Model inventory: Easily isolate model-related performance, error, and cost issues by tracking key metrics across NVIDIA NIM inference microservices in one place.

- Model comparison: Compare the performance of NVIDIA NIM inference microservices running in production in a single view to optimize model choice based on infrastructure and user needs.

- Deep GPU insights: Analyze critical accelerated computing metrics such as GPU utilization, temperature, and performance states; understand context and resolve problems faster.

- Enhanced data security: In addition to NVIDIA’s self-hosted model’s security advantage, New Relic allows you to exclude monitoring of sensitive data (PII) in your AI requests and responses.

This integration follows New Relic's recent addition to NVIDIA’s AIOps partner ecosystem. Leveraging NVIDIA AI’s accelerated computing, New Relic combines observability and AI to streamline IT operations and accelerate innovation through its machine learning, and generative AI assistant, New Relic AI. New Relic offers the most expansive observability solution with 60+ AI integrations including NVIDIA GPUs and NVIDIA Triton Inference Server software.

New Relic AI monitoring is available as part of its all-in-one observability platform and offered via its usage-based pricing model.

Share this

The Latest

July 23, 2024

The rapid rise of generative AI (GenAI) has caught everyone's attention, leaving many to wonder if the technology's impact will live up to the immense hype. A recent survey by Alteryx provides valuable insights into the current state of GenAI adoption, revealing a shift from inflated expectations to tangible value realization across enterprises ... Here are five key takeaways that underscore GenAI's progression from hype to real-world impact ...

July 22, 2024
A defective software update caused what some experts are calling the largest IT outage in history on Friday, July 19. The impact reverberated through multiple industries around the world ...
July 18, 2024

As software development grows more intricate, the challenge for observability engineers tasked with ensuring optimal system performance becomes more daunting. Current methodologies are struggling to keep pace, with the annual Observability Pulse surveys indicating a rise in Mean Time to Remediation (MTTR). According to this survey, only a small fraction of organizations, around 10%, achieve full observability today. Generative AI, however, promises to significantly move the needle ...

July 17, 2024

While nearly all data leaders surveyed are building generative AI applications, most don't believe their data estate is actually prepared to support them, according to the State of Reliable AI report from Monte Carlo Data ...

July 16, 2024

Enterprises are putting a lot of effort into improving the digital employee experience (DEX), which has become essential to both improving organizational performance and attracting and retaining talented workers. But to date, most efforts to deliver outstanding DEX have focused on people working with laptops, PCs, or thin clients. Employees on the frontlines, using mobile devices to handle logistics ... have been largely overlooked ...

July 15, 2024

The average customer-facing incident takes nearly three hours to resolve (175 minutes) while the estimated cost of downtime is $4,537 per minute, meaning each incident can cost nearly $794,000, according to new research from PagerDuty ...

July 12, 2024

In MEAN TIME TO INSIGHT Episode 8, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses AutoCon with the conference founders Scott Robohn and Chris Grundemann ...

July 11, 2024

Numerous vendors and service providers have recently embraced the NaaS concept, yet there is still no industry consensus on its definition or the types of networks it involves. Furthermore, providers have varied in how they define the NaaS service delivery model. I conducted research for a new report, Network as a Service: Understanding the Cloud Consumption Model in Networking, to refine the concept of NaaS and reduce buyer confusion over what it is and how it can offer value ...

July 10, 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog. In fact, 83% of container costs were associated with idle resources ...

July 10, 2024

Companies prefer a mix of on-prem and cloud environments, according to the 2024 Global State of IT Automation Report from Stonebranch. In only one year, hybrid IT usage has doubled from 34% to 68% ...