Vibe coding tools observability with VictoriaMetrics Stack and OpenTelemetry

AI-powered coding assistants have transformed how developers write software. Tools like Claude Code, OpenAI Codex, Gemini CLI, Qwen Code, and OpenCode have introduced what many call “vibe coding” — a new paradigm where users describe their intent and AI agents handle the implementation details.

But as these tools become integral to development workflows, a critical question emerges: how do we understand what’s happening under the hood?

Observability matters if you are an individual developer tracking token usage and costs, a team lead optimizing resource use, or an enterprise administrator enforcing compliance and governance policies.

The good news is that all major vibe coding tools now support OpenTelemetry, the vendor-neutral industry standard for telemetry data. Combined with open source VictoriaMetrics Observability Stack, you can build a powerful, cost-effective monitoring solution that provides deep insights into your AI-assisted development workflows.

In my previous article, “AI Agents Observability with OpenTelemetry and the VictoriaMetrics Stack”, I covered the fundamentals of monitoring AI agents using auto-instrumentation libraries. Today, I want to explore how to configure observability specifically for command-line vibe coding tools and examine the rich telemetry data they provide out of the box.

Here’s why implementing observability for AI agents is worth the effort:

• Cost Management: AI coding assistants consume tokens with every interaction, and costs can accumulate quickly. Understanding your token usage patterns helps you optimize prompts, choose appropriate models, and avoid budget surprises.

• Performance Insights: Are certain types of tasks taking longer than expected? Are there patterns in API latency that affect your workflow? Understanding these dynamics helps you work more efficiently.

• Usage Analytics: Which features are developers using most? How does tool adoption vary across projects? This data informs training investments and workflow improvements for teams.

• Debugging and Troubleshooting: When something goes wrong, having detailed logs and traces of what happened helps you quickly identify root causes.

• Compliance and Governance: Enterprise environments demand audit trails of AI tool usage, including what prompts were sent, what changes were made, and who made them.

The VictoriaMetrics Stack provides a complete, unified backend for all your observability signals:

• VictoriaMetrics: High-performance time-series metrics storage and querying with native OpenTelemetry support.

• VictoriaLogs: Efficient log storage and analysis with a powerful query language.

• VictoriaTraces: Distributed traces storage for visualizing complete request flows.

All three components accept data via the OpenTelemetry protocol (OTLP), so any tool that supports OTLP can send telemetry directly without custom integrations.

For quick setup instructions, including Docker Compose and Helm configurations, refer to my previous article on AI agents observability.

The easiest way to get started is to create deployments in VictoriaMetrics Cloud. Managed VictoriaMetrics is already available today and managed VictoriaLogs support is planned soon. If you’d like to run a PoC, please contact us. Otherwise, you can jump right into VictoriaMetrics Cloud with a free trial (no credit card required).

Now let’s look at how to configure each vibe coding tool to send telemetry to the VictoriaMetrics Stack.

This section is based on the official Claude Code documentation (and unlike other tools, Claude Code has excellent documentation on this topic).

Claude Code from Anthropic provides comprehensive OpenTelemetry support for both metrics and events (logs). The telemetry system is opt-in and designed with privacy in mind — sensitive information is never included in telemetry data, and user prompts are redacted by default.

To enable telemetry in Claude Code, set the following environment variables:

# Enable telemetry collection
export CLAUDE_CODE_ENABLE_TELEMETRY=1
# Enable cumulative counters instead of delta counters because VictoriaMetrics does not support delta
export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=cumulative

# Configure exporters for metrics and logs
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp

# Set the OTEL collector endpoint and protocol
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 # your OpenTelemetry collector endpoint

# Run Claude Code
claude

Remember to replace the collector endpoint with your own. For example, if you deploy everything locally, you can use:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

The default export interval is 60 seconds for metrics and 5 seconds for logs. During initial setup and debugging, you might want shorter intervals:

export OTEL_METRIC_EXPORT_INTERVAL=10000  # 10 seconds
export OTEL_LOGS_EXPORT_INTERVAL=5000     # 5 seconds

It also supports more advanced configuration options according to OpenTelemetry specification.

For organizations wanting centralized control, Claude Code supports managed settings files. Create a configuration file that will be distributed via MDM or other device management tooling:

{
  "env": {
    "CLAUDE_CODE_ENABLE_TELEMETRY": "1",
    "OTEL_METRICS_EXPORTER": "otlp",
    "OTEL_LOGS_EXPORTER": "otlp",
    "OTEL_EXPORTER_OTLP_PROTOCOL": "http/protobuf",
    "OTEL_EXPORTER_OTLP_ENDPOINT": "https://otel-collector.company.com:4318",
    "OTEL_EXPORTER_OTLP_HEADERS": "Authorization=Bearer company-token"
  }
}

Environment variables in managed settings have high precedence and cannot be overridden by individual users, ensuring a consistent telemetry collection across your organization.

Organizations with multiple teams can add custom attributes to segment telemetry data:

export OTEL_RESOURCE_ATTRIBUTES="env=production,department=engineering,team.id=platform"

These attributes appear on all metrics and events, enabling filtering by team, department, or env in your dashboards.

Claude Code provides environment variables to control which attributes are included in metrics:

Lower cardinality means better performance and lower storage costs, but less granular data for analysis.

Claude Code exports a rich set of metrics:

Beyond metrics, Claude Code exports detailed events via the OpenTelemetry logs protocol:

To enable user prompt logging (if your compliance requirements allow):

export OTEL_LOG_USER_PROMPTS=1

This section is based on the official OpenAI Codex documentation and the source code of Codex CLI.

OpenAI Codex provides OpenTelemetry export of logs and traces to track runs, including API requests, streaming events, prompts, tool approvals, and results. Like Claude Code, telemetry is disabled by default and requires explicit opt-in.

Codex uses a TOML configuration file located at ~/.codex/config.toml.

Add an [otel] section to enable telemetry:

[otel]
environment = "production"  # defaults to "dev"
log_user_prompt = false     # redact user prompts unless explicitly enabled

# Enable events (logs)
[otel.exporter.otlp-http]
endpoint = "http://otel-collector:4318/v1/logs" # your OpenTelemetry collector endpoint
protocol = "binary"

# Enable traces
[otel.trace_exporter.otlp-http]
endpoint = "http://otel-collector:4318/v1/traces" # your OpenTelemetry collector endpoint
protocol = "binary"

Remember to replace the collector endpoints with your own.

Codex currently exports logs and traces, but not metrics. For metrics, you would need to derive them from log data.

You can also use standard OpenTelemetry environment variables to configure the exporter. For example, organizations with multiple teams can add custom attributes to segment telemetry data:

export OTEL_RESOURCE_ATTRIBUTES="env=production,department=engineering,team.id=platform"

Codex adds a consistent set of attributes to every exported OpenTelemetry log event:

With telemetry enabled, Codex emits the following event types (exported as OpenTelemetry logs). Each event includes the common metadata above, plus per-event attributes:

To enable user prompt logging for codex.user_prompt events (if your compliance requirements allow), you should set log_user_prompt parameter to true in your configuration.

This section is based on the official Google Gemini CLI documentation.

Gemini CLI provides one of the most comprehensive OpenTelemetry implementations among vibe coding tools, with support for both metrics and logs following OpenTelemetry GenAI semantic conventions.

All telemetry behavior is controlled through .gemini/settings.json:

{
  "telemetry": {
    "enabled": true,
    "target": "local",
    "otlpEndpoint": "http://otel-collector:4318",
    "otlpProtocol": "http",
    "logPrompts": false 
  }
}

Remember to replace the collector endpoints with your own.

Environment variables can override these settings:

To enable user prompt logging (if your compliance requirements allow), you can set logPrompts parameter (or GEMINI_TELEMETRY_LOG_PROMPTS environment variable) to true.

Gemini CLI provides exceptionally detailed telemetry across multiple categories:

Gemini CLI also exports events following OpenTelemetry GenAI semantic conventions for events:

Gemini CLI also exports metrics following OpenTelemetry GenAI semantic conventions for metrics:

This section is based on the official Qwen Code documentation.

Qwen Code follows a similar architecture to Gemini CLI and provides comparable OpenTelemetry support.

Configure telemetry in ~/.qwen/settings.json:

{
  "telemetry": {
    "enabled": true,
    "target": "local",
    "otlpEndpoint": "http://localhost:4318",
    "otlpProtocol": "http",
    "logPrompts": false
  }
}

Remember to replace the collector endpoints with your own.

Qwen Code also supports CLI flags for quick configuration changes:

qwen-code \
  --telemetry \
  --telemetry-target local \
  --telemetry-otlp-endpoint http://otel-collector:4318 \
  --telemetry-otlp-protocol grpc

Environment variables can override settings in the configuration file:

To enable user prompt logging (if your compliance requirements allow) you can set logPrompts parameter (or QWEN_TELEMETRY_LOG_PROMPTS environment variable) to true.

Qwen Code exports events with the qwen-code.* prefix:

A unique feature of Qwen Code is the detailed diff statistics in file operations, including ai_added_lines / model_added_lines, ai_removed_lines / model_removed_lines, user_added_lines, and user_removed_lines. This helps you understand how much AI-generated code survives human review.

OpenCode releases announced support for OpenTelemetry but unfortunately, I haven’t managed to get OpenCode to send telemetry. There is no documentation on this, and the methods I found didn’t work for me.

Nevertheless, OpenCode is a fantastic tool, and I hope this feature will be fully implemented in the future.

With telemetry flowing into VictoriaMetrics Stack, you can build insightful dashboards and queries that answer important questions about your AI-assisted development workflows using the power of MetricsQL for metrics and LogsQL for logs and traces.

Across all vibe coding tools, you can typically obtain the following types of information:

• Token Usage and Cost Analysis:
  • Total token consumption by model, type (input/output/cache), and over time
  • Cost tracking (currently only Claude exposes cost metrics directly)
  • Cache efficiency metrics to optimize prompt engineering
• Productivity and Efficiency:
  • Session and user counts and active time tracking
  • User prompt analysis (length, frequency)
  • Configuration and policy usage (approval modes, sandbox settings)
  • Context compression events when conversations approach limits
• Tool Usage and Performance:
  • Which tools are used most frequently
  • Tool execution success/failure rates
  • Tool latency
  • Permission decision patterns (accept/reject/auto-accept)
  • MCP server usage
• API Health and Troubleshooting:
  • API call latency and error rates
  • Detailed error logs with context
  • Request/response analysis for debugging
• Team Usage Segmentation:
  • Usage by department / team
  • Individual user analytics
  • Multi-environment support

I haven’t found good publicly available dashboards, so I decided to create them.

These dashboards are available in the VictoriaMetrics-Community/vibe-coding-dashboards GitHub repository and on Grafana dashboards.

• Claude Code
  • Dashboard file
  • Grafana dashboards page
• OpenAI Codex
  • Dashboard file
  • Grafana dashboards page
• Google Gemini
  • Dashboard file
  • Grafana dashboards page
• Qwen Code
  • Dashboard file
  • Grafana dashboards page

These dashboards contain a lot of information, but of course, they don’t cover all needs. So if you are missing any information or see any errors, we welcome pull requests. You can also use the dashboard as a basis for your own dashboards, tailored to your specific requirements and scenarios.

The convergence of AI-powered coding tools and modern observability standards creates exciting opportunities to understand and optimize our development workflows. All major vibe coding tools now speak OpenTelemetry, and the VictoriaMetrics Stack provides a cost-effective, high-performance backend for storing and analyzing the resulting telemetry.

For tracking personal token usage, managing team costs, or implementing enterprise governance, the combination of OpenTelemetry and VictoriaMetrics gives you the visibility you need:

• Near-real-time usage analytics
• Performance and reliability dashboards
• Cost and token burn monitoring
• Deep debugging via structured tool events and traces
• Cost-effective and scalable observability backend
• and much more!

Start with basic token and cost tracking, then expand into tool usage analytics, performance monitoring, and productivity metrics as your needs evolve. The observability infrastructure you build today will serve you well as AI coding tools become ever more central to software development.

Happy vibe coding!