Back to catalog
OpenLLMetry
Apache-2.0OpenLLMetry replaces proprietary APM agents with open-source OpenTelemetry instrumentation for LLMs, routing traces and token metrics to any collector.
Open-source alternative to — compare it against:
Overview
Datadog charges premium rates for proprietary agents and confines generative AI telemetry inside a vendor-controlled cloud. OpenLLMetry instruments model invocations, vector databases, and multi-step agent frameworks using vendor-neutral OpenTelemetry extensions. By exporting standardized traces over standard collector protocols to existing observability backends, it eliminates the operational overhead of closed monitoring silos.
Why OpenLLMetry instead of Datadog
Datadog treats generative AI monitoring as a proprietary feature, requiring teams to install vendor-specific tracer libraries, accept vendor-defined metrics, and route telemetry exclusively through Datadog ingestion pipelines. OpenLLMetry provides equivalent visibility into token counts, latency distributions, prompt inputs, and completion payloads by wrapping underlying software development kits with standard OpenTelemetry spans. Because it adheres directly to upstream OpenTelemetry semantic conventions, operational teams retain ownership of trace schemas rather than adapting infrastructure to vendor-specific formats.
The trade-off centers on operational responsibility. Datadog provides turnkey dashboards, preconfigured alerts, and automated anomaly detection, whereas OpenLLMetry is strictly an instrumentation layer that neither stores nor visualizes data. Teams adopting OpenLLMetry route spans through an OpenTelemetry Collector into storage backends like Grafana Tempo or SigNoz, or configure an OTLP endpoint on their existing monitoring service. For platform engineers unwilling to duplicate tracing infrastructure with dedicated tools like LangSmith, OpenLLMetry keeps AI telemetry aligned with existing operational workflows.
What it does well
- Automated instrumentation across major inference providers, wrapping calls to OpenAI, Anthropic, AWS Bedrock, Google Gemini, and Ollama without manual span boilerplate.
- Deep agent framework visibility, capturing intermediate steps, tool calls, and retrieval contexts inside LangChain, LlamaIndex, CrewAI, and LangGraph pipelines.
- Standard OpenTelemetry GenAI semantic conventions, ensuring token usage, system prompts, and model identifiers map uniformly across disparate telemetry backends.
- Vector database tracking, measuring retrieval latency and query metadata for Pinecone, Qdrant, Chroma, Weaviate, and Milvus alongside model inference.