opik

<b>English | 简体中文 | Español | Français | Deutsch | 日本語</b>

Opik: Open-Source LLM Observability, Evaluation & AI Agent Tracing

<b>Opik is the open-source LLM observability and evaluation platform for AI agent tracing, LLM evaluation, prompt management, and production monitoring.</b> Built by Comet. Apache-2.0 licensed, free to self-host the full platform, with 20,000+ GitHub stars.

<b>Website</b> • <b>Slack Community</b> • <b>Twitter</b> • <b>Changelog</b> • <b>Documentation</b>

_{Last updated: 2026-07-17}

🚀 What is Opik? • ⚡ Quick Start • 📊 How Does Opik Compare? • ❓ FAQ • 🛠️ Opik Server Installation • 💻 Opik Client SDK • 📝 Logging Traces 🧑‍⚖️ LLM as a Judge • 🔍 Evaluating your Application • ⭐ Star Us • 🤝 Contributing

🚀 What is Opik?

Opik covers the full LLM application lifecycle, from the first trace in development to production monitoring, for teams building LLM apps and AI agents. Key offerings include:

• AI Agent Tracing & Observability: Deep tracing of LLM calls, conversation logging, and agent activity, with full trace trees for multi-step agents and tool calls.
• LLM Evaluation: Datasets, experiments, and LLM-as-a-judge metrics for hallucination detection, moderation, and RAG assessment.
• Prompt & Agent Optimization: The Opik Agent Optimizer SDK to improve prompts and agents.
• Production-Ready Monitoring: Scalable dashboards and online evaluation rules.
• Opik Guardrails: Features to help you implement safe and responsible AI practices.
• CI/CD Evaluation: A PyTest integration to test LLM pipelines on every commit.

Key capabilities include:

• Development & Tracing:
• Track all LLM calls and traces with detailed context during development and in production (Quickstart).
• Extensive 3rd-party integrations for easy observability: Seamlessly integrate with a growing list of frameworks, supporting many of the largest and most popular ones natively (including recent additions like Google ADK, Autogen, and Flowise AI). (Integrations)
• Annotate traces and spans with feedback scores via the Python SDK or the UI.
• Experiment with prompts and models in the Prompt Playground.

• Evaluation & Testing:
• Automate your LLM application evaluation with Datasets and Experiments.
• Leverage powerful LLM-as-a-judge metrics for complex tasks like hallucination detection, moderation, and RAG assessment (Answer Relevance, Context Precision).
• Integrate evaluations into your CI/CD pipeline with our PyTest integration.

• Production Monitoring & Optimization:
• Log high volumes of production traces: Opik is designed for scale (40M+ traces/day).
• Monitor feedback scores, trace counts, and token usage over time in the Opik Dashboard.
• Utilize Online Evaluation Rules with LLM-as-a-Judge metrics to identify production issues.
• Leverage Opik Agent Optimizer and Opik Guardrails to continuously improve and secure your LLM applications in production.

Who it's for: ML engineers building LLM-powered agents, AI teams moving from prototype to production, and engineering teams that need open-source, self-hostable observability they can run in their own environment.