Free eNews Subscription

Datadog Launches LLM Observability to Enhance GenAI Monitoring and Security

By Stefania Viscusi

Datadog, a provider of monitoring and security solutions for cloud applications, has introduced LLM Observability, a new offering designed to streamline the monitoring, optimization, and security of Generative AI applications. The launch marks a significant advancement in supporting AI application developers and machine learning (ML) engineers and enables organizations to deploy generative AI features more efficiently and reliably in production environments.

As AI technologies rapidly evolve, organizations across industries are looking to integrate generative AI capabilities into their operations. However, deploying and managing these AI models poses significant challenges due to their intricate nature, non-deterministic behavior, and inherent security risks.

With Datadog’s LLM Observability offering, these challenges are mitigated by providing comprehensive visibility into LLM chains, so users can pinpoint errors, anomalies, and security vulnerabilities.

"The Datadog LLM Observability solution helps our team understand, debug and evaluate the usage and performance of our GenAI applications," said Kyle Triplett, VP of Product at AppFolio. “With it, we are able to address real-world issues, including monitoring response quality to prevent negative interactions and performance degradations, while ensuring we are providing our end users with positive experiences.”

Features and benefits of the observability tools include:

  • Enhanced Visibility and Monitoring: LLM Observability offers detailed insights into each step of the LLM chain, allowing developers to identify and resolve issues such as errors and unexpected responses, commonly known as hallucinations. Operational metrics like latency and token usage can be monitored in real-time to optimize performance and control costs efficiently.
  • Quality and Safety Evaluations: Users can also evaluate the quality of AI applications based on criteria like topic relevance and toxicity. This helps in maintaining the integrity of AI-generated content while mitigating security and privacy risks.
  • Integration and Scalability: Unlike traditional tools, Datadog's solution integrates seamlessly with its Application Performance Monitoring (APM) capabilities. It supports major platforms including OpenAI, Anthropic, Azure OpenAI, and Amazon Bedrock, providing a unified dashboard for monitoring and optimizing AI applications across diverse environments.

"There's a rush to adopt new LLM-based technologies, but organizations of all sizes and industries are finding it difficult to do so in a way that is both cost effective and doesn't negatively impact the end user experience," said Yrieix Garnier, VP of Product at Datadog. "Datadog LLM Observability provides the deep visibility needed to help teams manage and understand performance, detect drifts or biases, and resolve issues before they have a significant impact on the business or end-user experience."

Edited by Erik Linask
Get stories like this delivered straight to your inbox. [Free eNews Subscription]
Related Articles

New Report Finds Optimistic CMO Attitudes to GenAI

By: Tracey E. Schelmetic    7/17/2024

According to a new study by Boston Consulting Group (BCG), 80% of CMOs surveyed said that GenAI is already improving automation, speed and productivit…

Read More

New Research Study Reveals Trends in GenAI Technology Investment

By: Tracey E. Schelmetic    7/16/2024

A new study released by ISG Software Research (formerly Ventana Research) has found that GenAI is leading the charge when it comes to making artificia…

Read More

InMoment Introduces AI Studio to Complement its Experience Improvement Platform

By: Tracey E. Schelmetic    7/16/2024

Customer experience (CX) platform company InMoment recently announced the launch of InMoment AI Studio, an internal framework for developing and manag…

Read More

C-Suite Faces Pressure in Generative AI Rush

By: Greg Tavarez    7/15/2024

A new study from RWS found that nearly nine in 10 top-level leaders feel the heat to quickly deploy GenAI solutions on a large scale.

Read More

HPE Teams with NVIDIA to Build Fast AI Supercomputer for AIST in Japan

By: Greg Tavarez    7/12/2024

HPE recently announced it will construct an NVIDIA-powered next-generation supercomputer for AIST.

Read More