OpenLIT

OpenLIT builds on OpenTelemetry semantic conventions to capture traces, token expenses, and GPU metrics across AI applications. Its single-container platform pairs an embedded OTLP receiver with ClickHouse storage to keep telemetry inside private infrastructure. Developers and DevOps engineers run the entire ingestion and dashboard stack without external vendor subscriptions.

Why OpenLIT instead of LangSmith

LangSmith charges per seat, metered trace ingestion rates, and retention fees that climb rapidly when production systems run multi-turn agent loops or large-volume prompt chains. Beyond the software subscription, shipping raw prompts, context documents, and model outputs to third-party endpoints introduces network egress overhead and data residency compliance burdens. OpenLIT removes per-seat pricing and trace ingestion surcharges by writing telemetry directly into a self-hosted ClickHouse cluster running inside your own network perimeter.

The fundamental tradeoff balances vendor neutrality against ecosystem-specific tooling. LangSmith delivers deep debugging workflows, dataset curation, and prompt playground features built expressly for LangChain and LangGraph projects. OpenLIT adheres to vendor-neutral OpenTelemetry standards, allowing spans, error traces, and token metrics to route into existing infrastructure such as Grafana Tempo, SigNoz, or Datadog without custom adapters. In return, your team takes on the operational duty of managing ClickHouse storage, configuring disk retention policies, and scaling ingestion pipelines.

What it does well

• Instruments more than fifty model providers, vector stores, and agent frameworks through vendor-neutral Python, TypeScript, and Go SDKs.
• Collects hardware-level GPU utilization, memory usage, and temperature metrics across NVIDIA, AMD, and Intel chipsets alongside token throughput.
• Adheres to standard OpenTelemetry GenAI semantic conventions so telemetry flows directly into existing corporate collectors and dashboards.
• Tracks inference expenditures down to individual requests with customizable pricing files for fine-tuned and private models.
• Executes eleven automated evaluation checks, such as hallucination, toxicity, and bias detection, using context-aware evaluation judges.
• Integrates with local AI coding assistants, including Cursor and Claude Code, through a dedicated CLI that monitors editor-driven agent interactions.