How to Use telemetry.dev: The Beginner's Guide (First Trace, Tokens and Cost Per Span, and the MCP Arc)
CodexFuse Team · 2026-11-03T07:54:20.852Z · 14 min read min read
Learn how to use telemetry.dev: sign up free, send your first trace, read spans and costs, use the read-only MCP server, and guard data with prompt-capture rules.
If you have shipped an AI feature and then watched a user ask why it answered badly, cost a fortune, or took forty seconds, you have run into the problem that teaches you how to use telemetry.dev. Your model provider's dashboard shows you that calls happened; it does not show you the agent run as one connected story — which tool call timed out, which prompt version was live, or which model burned the budget. This guide walks you through the beginner's path: sign up, send your first trace, read the span waterfall, let your coding agent investigate its own traces through the read-only MCP server, and set the one privacy rule that matters before you instrument anything real. This telemetry.dev tutorial follows the fastest path the vendor itself teaches — the one-prompt coding-agent onboarding — rather than the manual SDK route. If you already monitor servers and containers, the mental bridge is short: a Prometheus beginner guide covers the infrastructure layer, while telemetry.dev watches what your LLM calls actually do.
What Is telemetry.dev, Exactly?
telemetry.dev is an LLM observability platform for AI applications and agents, built on OpenTelemetry. A trace groups the model calls, tool steps, and retrievals in one request or agent run into a single waterfall, and every span carries its tokens, cost, latency, and errors — so you can find the slow step, inspect the failed call, and see where your budget goes.
The platform sits in the observability layer of your stack: any framework or language that can emit standard OTLP over HTTP can send data to it, with no vendor lock-in. It normalizes incoming spans into one model of spans, tokens, and cost, and surfaces the four metrics its team says developers actually check: volume, cost, p95 latency, and error rate.
How Much Does telemetry.dev Cost?
Pricing is per workspace, per month, and the vendor's pricing page is the only source you should trust — third-party directories still show older numbers. Here is the current structure as of early November 2026.
| Plan | Price | Ingestion units | Retention | Projects | Seats |
|---|---|---|---|---|---|
| Free | $0 — no credit card | 30,000 / month | 14 days | 1 | 2 |
| Pro | $25 / month | 500,000 / month | 30 days | 5 | 5 |
| Team | $50 / month | 2M / month | 90 days | Unlimited | Unlimited |
One ingestion unit equals one normalized span, one OTLP log record, or one metric data point — a trace with five spans costs five units. The Free plan quietly allows about 20% headroom before ingestion is rejected, and it resets at 00:00 UTC on the first of the month. Pro does not add automatic overage charges; ingestion simply continues above the allowance. Team adds priority support, and there are no contracts — you can cancel, upgrade, or downgrade at any time.
The Product Hunt launch post (October 28, 2026) also mentions a code, PHBU7A47FH, that was said to grant one month of Pro. I could not verify that the code still works — the vendor site shows no promo-code field or expiry date, so treat the code as a launch-time offer until you test it at signup.
How Do You Sign Up and Send Your First Trace?
You do not need a credit card to start: the Free plan exists with no card, and the vendor advertises roughly five minutes from zero to a first trace. The flow has four steps.
- Make a project API key. After signing up, you create a project and generate an API key — this is the key that authenticates trace ingestion, and it must never ship to the browser or leak into a public repo.
- Give the one prompt to your coding agent. Instead of reading setup docs, you hand your agent one prompt. It finds your stack, adds the right integration, sends a test trace, and then lists the environment variables you need to set. If you have never driven an agent this way, the Claude Code beginner guide explains the mechanics of agent-driven workflows.
- Configure the integration or an OTLP exporter. Your agent either installs one of the drop-in SDKs or points your existing OpenTelemetry setup at the ingest endpoint.
- Run one instrumented request. Trigger a single agent run or model call, then open its trace in your project dashboard.
Integration guides on the vendor's site cover coding agents (Claude Code, Cursor, opencode, OpenClaw, Hermes, Oh My Pi, Pi), model providers (OpenAI, Anthropic, Google Gen AI, Amazon Bedrock, LiteLLM, OpenRouter), and frameworks (Vercel AI SDK, TanStack AI, LangChain and LangGraph, CrewAI, Pydantic AI, Eve), plus the official TypeScript MCP SDK v2. Any other language works through the standard OTLP/HTTP exporter.
What Does a Trace Show You?
Every agent run becomes one trace, and every step — each model call, tool call, or retrieval — becomes a span in a waterfall with its duration and its share of the run's total time. Opening a trace answers four questions at a glance: which step was slow, which call failed, which models and tools were used, and how much the whole run cost.
Cost is calculated at ingest from the call's real token usage against a server-side model-pricing catalog — not estimated on your client. Cache-read and cache-write tokens use separate rates where available, reasoning tokens are not charged separately, and an explicit cost set on a span takes precedence. If the catalog has no matching model pricing, the computed cost is simply unavailable rather than guessed.
| What each span carries | What it tells you |
|---|---|
| Duration + share of run (%) | Where the latency lives — the slow step stands out |
| Model and provider | Which model answered this step (and whether a cheaper one would do) |
| Input / output / cached tokens | Real usage per step, not an estimate |
| Cost in USD, split per span | Input, cache-read, cache-write, reasoning, and output cost separated |
| Error text | The failed call's actual error message, in context |
| Environment + session | Production vs staging, and the user session it belonged to |
Trace lists summarize runs as spans plus cost — for example, nine spans costing $0.0124 — and you can search traces by name or filter by status, environment, framework, or session. There is also a dedicated MCP analytics dashboard that shows spans, failures, and p95 duration by method and tool, from both the client and the server — useful the moment your agents start calling MCP tools, since it exposes p95 client overhead and calls that never reached a server.
Which SDK or Integration Should You Use?
There are three ways data gets in, and the honest beginner answer is: use the drop-in SDK for your language unless you already run OpenTelemetry.
The TypeScript SDK is the @telemetry-dev/sdk package on npm, with provider packages like @telemetry-dev/openai and @telemetry-dev/ai-sdk. The Python SDK is the telemetry-dev package on PyPI. Both wrap a function in an observe() decorator or call, turning it into a span with inputs, outputs, tokens, and errors captured automatically. If you already instrument with OpenTelemetry, point your OTLP/HTTP exporter at the ingest endpoint — the SDKs themselves send standard OTLP/HTTP under the hood. The Claude SDK and the Claude API can also send their native OpenTelemetry spans directly.
pip install telemetry-dev export TELEMETRY_DEV_API_KEY=td_live_your_key_here
The Python SDK reads the API key from the TELEMETRY_DEV_API_KEY environment variable, fails open as a silent no-op when the key is absent, and never raises its internal errors into your code. If you are new to agent-assisted setup, the OpenCode beginner guide covers how coding agents edit and verify code on your behalf.
How Does the MCP Server Let Your Agent Read Its Own Traces?
Once traces are flowing, you can connect your AI agent to the read-only MCP server at https://telemetry.dev/mcp — for example with claude mcp add --transport http telemetry https://telemetry.dev/mcp — and ask it to investigate its own failures, latency, and cost. The server is read-only by design: an agent cannot change projects, settings, keys, or telemetry. A telemetry CLI ships as an alternative for terminal workflows — the telemetry.dev CLI is the same interface agents reach through the MCP server, just driven by you instead.
The server exposes seven tools. By default, agents cannot read captured prompts, outputs, or tool arguments — that requires checking a separate permission at approval time, which is the sane default for production data.
| MCP tool | What it returns |
|---|---|
list_projects | Your projects and their metadata |
search_sessions | Sessions with trace counts, errors, tokens, and cost |
search_traces | Traces filtered by name, status, environment, framework, service, session, end user, model, provider, operation, agent, or tool |
get_trace | One trace's spans with models, tools, timing, tokens, cost, error text, and logs — plus a dashboard link |
get_span_payload | Full payload of a span (needs extra permission) |
list_logs | Logs attached to traces |
get_usage_summary | Requests, errors, latency, tokens, cost, and active users over 7/30/90 days vs the previous period, broken down by model, provider, end user, or operation |
How Do Prompts, Evals, and Alerts Fit Together?
telemetry.dev is not only a trace viewer — the same week of the October 2026 launch added prompt management, evals, and alerts, and the three form one loop. You manage prompts without a deploy: keep prompt versions in telemetry.dev with labels, have your code fetch the version labeled production, and every trace shows which prompt version ran. A rollout promotes a version to production for 1 to 99% of calls, and you can move the label back if the new prompt misbehaves. Versions are compared on calls, errors, p50 latency, and cost per call.
Evals close the quality loop: build datasets of inputs with expected outputs, run LLM judges, code checks, rules, or human scores over them, and compare prompt versions against a baseline — in CI or live, where a sample of production traces is scored continuously. Alerts close the cost loop: checks run every five minutes against your project history, and rules cover monthly budgets, cost-per-trace jumps, agent loops (one turn above $2 or 30 spans), model and tool errors, ingestion errors, missing prices, quotas running out, and eval-score drops. Destinations are email, Slack, Discord, and webhooks today; Microsoft Teams, PagerDuty, Opsgenie, and Google Chat are marked as planned. An alert resolves after three healthy checks, with reminders every six hours.
What Is the Prompt-Capture Rule?
Prompts and responses are captured and stored by default, and this is the one privacy decision you must make before instrumenting a real application: turn off prompt capture for an environment before storage if it can see sensitive data, because the vendor's own redaction cannot catch everything. Their wording is unambiguous: "Redaction does not catch all sensitive data."
Built-in redaction patterns are always on (secrets, personal data, API keys, Bearer and JWT tokens, emails, SSNs, phone numbers, Luhn-passing card numbers), and projects can add up to 20 custom RE2 patterns — but server-side controls run only after your data reaches telemetry.dev, and pattern matching misses things like names, addresses, and non-standard secret formats. If you have never wired secrets through your repository safely, the Git beginner guide is the prerequisite habit. The product is also cloud-hosted only: no self-host option, no public repository, and no data-location statement appears in any vendor material I found.
telemetry.dev vs LangSmith: Which Should You Pick?
telemetry.dev is about one week into its relaunched feature set, and the vendor publishes no comparison against LangSmith or any other observability product — the only comparisons anywhere are third-party evaluations, which largely conclude that the product is too new for anyone but its maker to have a track record. That is the honest beginner framing: LangSmith and Langfuse are established tools with years of history, while telemetry.dev is the new entrant whose differentiators — one-prompt onboarding, server-side cost math across its pricing catalog, and a read-only MCP server for self-investigating agents — you can only evaluate by running it.
The practical test is cheap: the Free plan needs no card, thirty thousand ingestion units is a comfortable trial allowance, and if the trace waterfall does not answer "where did the money go" in one sitting, the established tools are still there.
7 Beginner Mistakes to Avoid
| Mistake | What happens | Fix |
|---|---|---|
| Hardcoding the API key | The key ships to browsers, bundles, or public repos | Set TELEMETRY_DEV_API_KEY as an environment variable only |
| Instrumenting production before reading the privacy docs | Prompts and responses land in a cloud you never intended | Decide on the prompt-capture toggle per environment first |
| Judging cost from one trace | A single run looks cheap; thousands of runs do not | Watch cost-per-trace trends and the p95, not one waterfall |
| Ignoring agent-loop alerts | One looping agent burns more than a week of normal traffic | Keep the agent-loop alert on with its defaults |
| Comparing old pricing blog posts | Third-party directories list stale tiers | Trust only the vendor pricing page at signup time |
| Reading redaction as a guarantee | "Redaction does not catch all sensitive data" is the vendor's own line | Strip secrets in your instrumentation before export |
| Skipping the MCP server | You debug traces by hand forever | Connect the read-only MCP server and let your agent triage |
Sources
- telemetry.dev homepage — one-prompt onboarding, MCP server, prompt-capture wording — https://telemetry.dev/
- telemetry.dev pricing — plans, ingestion units, retention, seats — https://telemetry.dev/pricing
- telemetry.dev integrations directory — coding agents, providers, frameworks — https://telemetry.dev/docs/integrations
- telemetry.dev docs — SDKs, OTLP ingest, cost calculation — https://telemetry.dev/docs
- telemetry.dev MCP server docs — tools, read-only scope, auth — https://telemetry.dev/docs/platform/mcp-server
- telemetry.dev MCP features — span analytics, failure categories — https://telemetry.dev/features/mcp
- telemetry.dev prompts features — versions, labels, rollouts — https://telemetry.dev/features/prompts
- telemetry.dev evals docs — datasets, evaluators, experiments — https://telemetry.dev/docs/evals
- telemetry.dev alerts features — rules, destinations, cadence — https://telemetry.dev/features/alerts
- telemetry.dev privacy docs — redaction patterns, retention — https://telemetry.dev/docs/platform/privacy
- Product Hunt — telemetry.dev launch post, promo code, maker — https://www.producthunt.com/products/telemetry-dev
- everydev.ai — telemetry.dev directory profile, Pro price cross-check — https://www.everydev.ai/tools/telemetry-dev
- PyPI — telemetry-dev Python SDK install and quickstart — https://pypi.org/project/telemetry-dev/0.2.2/
- Activepieces — independent evaluation, "too new for a track record" context — https://www.activepieces.com/blog/how-to-evaluate-telemetry-dev-llm-observability
Frequently Asked Questions
How long does it take to send your first trace?
About five minutes, per the vendor's own framing: sign up with no card, create a project API key, hand your coding agent the one prompt, and run one instrumented request. The coding agent does the integration work — you set the environment variables it lists.
Does telemetry.dev require its own SDK?
No. The drop-in TypeScript and Python SDKs are the fast path, but any language works by pointing a standard OpenTelemetry OTLP/HTTP exporter at the ingest endpoint. The Claude SDK and the Claude API can also send their native OpenTelemetry spans directly.
Can you self-host telemetry.dev?
No. Every plan is per workspace, per month, and the SDKs send OTLP to the vendor's ingest. No self-host option, public repository, or data-location statement appears in any published material I found.
What does an ingestion unit mean on the Free plan?
One unit is one normalized span, one OTLP log record, or one metric data point — so a five-span trace costs five units. The Free plan includes 30,000 units a month, rejects ingestion above roughly 36,000, and resets at 00:00 UTC on the first of the month. For an llm observability free tier, thirty thousand spans is a genuinely usable trial allowance — enough to instrument a side project and watch a full week of real traffic before spending anything.
Can the MCP server change anything in your account?
No. It is read-only by design: an agent can list projects, search sessions and traces, and summarize usage, but cannot change projects, settings, keys, or telemetry. Reading captured prompts and tool arguments needs a separate opt-in at approval time.
