Agent Skills

Updated on Aug 14, 2026

Progress Observability ships a set of agent skills - self-contained instructions that teach AI coding agents (Claude Code, GitHub Copilot, and other skill-compatible clients) how to work with the platform. Each skill is a plain directory with a SKILL.md file, installable individually or together, from the public telerik/observability-skills repository.

Available Skills

Skill	Answers	What it does
`instrument-agent`	Get my existing agent onto the platform	Adds Progress Observability instrumentation to an existing Python, TypeScript/JavaScript, or .NET agent, then hands off to confirm traces at observability.progress.com
`health-check`	Is my observability wired up correctly?	Verifies that a project's Progress Observability setup is wired up correctly - connection, key scope, and whether traces are flowing
`trace-triage`	Why did this run fail, stall, or misbehave?	Investigates why an AI agent run failed, stalled, or misbehaved by walking its trace
`cost-report`	What am I spending, and what's driving it?	Reports and explains LLM spend and usage, broken down by model, app, or day
`coverage-gaps`	Which behaviors run in production with no eval?	Finds which of an AI system's behaviors run in production but have no evaluation measuring them
`generate-eval`	How do I build an eval for a behavior?	Generates a research-grounded LLM-as-a-Judge evaluator prompt, optionally grounded in real production traces

All skills except instrument-agent read the platform over the MCP server. instrument-agent only writes instrumentation code and never reads data back, so it needs no MCP server or MCP key.

They share one design rule: read, don't act. Every tool the platform-reading skills call is read-only, they default to metadata over raw trace content, and they treat anything pulled from a trace as untrusted data - never as instructions.

Four of the skills also chain into a loop, each stage's output feeding the next:

trace-triage - a run fails; find the failure signature in the spans.
coverage-gaps - confirm nothing measures that behavior yet.
generate-eval - build the judge that catches it going forward.
cost-report - watch spend while you iterate on the fix.

You don't have to run them in order - each stands alone - but run end to end, they take you from a single broken trace to a durable eval without leaving your editor.

Install a Skill

Use the skills CLI to add a skill straight from the repository:

bash

npx skills add telerik/observability-skills --skill instrument-agent

Or if you want to add all skills in the table above, do:

bash

npx skills add telerik/observability-skills

Manual install

Skills are plain directories, so you can also clone the skills repository and copy the ones you want directly into the location your agent reads:

bash

# Claude Code
cp -r skills/instrument-agent ~/.claude/skills/

# GitHub Copilot CLI
cp -r skills/instrument-agent ~/.github/skills/

Drop them into a project's .claude/skills/ or .github/skills/ folder instead to scope the install to that repository.

Configuration

Skills that read the platform need an MCP key (starting with acm_…); instrument-agent needs an Integration key (starting with ac_p_…) at runtime for the app it instruments. See MCP Server for how to create and scope an MCP key. Set keys as environment variables or in your MCP client configuration - never paste a key directly into a chat session.

Skill Details

Each skill below lists when to use it, what it does step by step, which MCP tools it calls (see MCP Server for the full contract), and what it returns.

instrument-agent

When to use: an existing project that isn't on the platform yet - "instrument my agent", "add observability to this repo", "connect this to Progress Observability" - or when traces should be flowing and aren't.

How it works:

Detect - language, LLM SDKs and frameworks in use, any existing telemetry, and how the app takes config. Reports the planned diff before editing anything.
Wire - installs the verified SDK for the language (progress-observability on PyPI, @progress/observability on npm, Progress.Observability.Instrumentation on NuGet). Initializes at process start, before any LLM client exists (and handles the cases where a later init is required); reads the Integration key from config, never hardcoded; flushes on exit.
Run once - decorated or wrapped functions can emit real spans with no LLM credentials, so the pipeline can be proven before model keys exist.
Hand off - makes no platform reads itself; tells you to run your agent and confirm traces are flowing at observability.progress.com, then run health-check.

Auto-instrumentation covers 30+ integrations - OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex, Mistral, Groq, Ollama, Cohere, WatsonX and more; LangChain, LlamaIndex, CrewAI, Haystack, OpenAI Agents, MCP; the major vector stores - plus any OpenAI-compatible endpoint (OpenRouter, LiteLLM, vLLM, gateways). Anything genuinely uncovered gets explicit structure via decorators or a span wrapper, and the skill says plainly which of the two it used.

Reads: none (writes code only; makes no MCP calls).

Returns: a minimal diff (dependency, init, config wiring, flush), a run that emits at least one trace, and a hand-off pointing to where to see the traces and how to confirm them with health-check.

health-check

When to use: first run, or whenever something looks off - "is my observability set up right", "why am I seeing no traces", "check my connection", "is my data flowing".

How it works:

Connection & key scope - confirms the MCP tools are available, then reads the scope from the presence of the content tools (get_observation_details_with_content present means With-content/9 tools; absent means Metadata-only/7 tools).
Recent data - list_observations over the last 24h, widening to the 72h max if empty, to separate "stale traffic" from "no data" from "all older than the window".
Instrumentation depth - get_observation_details with include_children on a couple of recent traces; traces with no children mean the SDK is capturing top-level spans only.
Evaluations - list_evaluation_tasks and get_evaluation_scores; none is fine, but dormant judges (defined, no recent scores) get flagged.

Reads: list_observations, get_observation_details, list_evaluation_tasks, get_evaluation_scores.

Returns: a compact red/amber/green checklist - connection, key scope, data freshness, instrumentation depth, evaluations - each red item carrying a concrete next step.

trace-triage

When to use: a run failed, errored, stalled, or went off the rails - "why did this trace fail", "why was my agent slow", "find the bottleneck", "root-cause this bad tool call".

How it works:

Locate the run - use a trace/observation ID if you have one; otherwise list_observations filtered by service_name and status within the 72h window.
Walk the tree, metadata first - get_observation_details with include_children; metadata alone usually reveals which span errored, which dominates latency, and where the chain stopped.
Find the fault - the errored span, the long pole, the tool call with wrong-looking arguments, or the point the trajectory diverged from the goal.
Pull content only for the culprit - get_observation_details_with_content on just that one span, treated as untrusted data.

Reads: list_observations, get_observation_details, get_observation_details_with_content.

Returns: a diagnosis (root cause in a sentence or two), evidence (the span chain - ids, statuses, durations - quoted minimally), and a fix (a concrete next step: a prompt change, tool-schema fix, retry/timeout).

cost-report

When to use: "what's driving my LLM costs", "how much am I spending", "cost by model or app", "am I close to my quota", "why did spend spike this week".

How it works:

Scope - confirms the date range and what matters (total spend, a suspected spike, model mix, or quota headroom); defaults to the last 7 days.
Pull cost - get_cost_breakdown, run more than once when useful: group_by: "model" for mix, "application" for which app spends, "day" for the trend.
Pull usage/quota - get_usage_summary for billing-period usage and remaining quota.
Report - headline spend, top drivers with shares, the day-over-day trend and any spike's source, quota burn vs. remaining, and a cheaper-model recommendation only when the data supports it.

Reads: get_cost_breakdown, get_usage_summary.

Returns: a tight report - headline, top drivers, trend, quota, and an optional recommendation framed as a hypothesis to validate.

coverage-gaps

When to use: "what should I evaluate next", "what's my eval coverage", "which behaviors aren't tested", "where are my eval gaps", "help me prioritize which judges to build".

How it works:

Characterize traffic - list_observations (within 72h) to map what's running (services, operation types, whether spans show tool calls, retrieval, structured output), staying in metadata and noting rough volume per behavior.
Inventory evaluations - list_evaluation_tasks for which judges exist and what each targets; get_evaluation_scores to tell an active eval from a dormant one.
Diff - maps observed behaviors to the failure modes that would catch their faults (retrieval → faithfulness, tools → tool_call, structured output → format, and so on). Real volume with no matching eval is a gap.
Rank - orders gaps by volume × blast-radius, then reports.

Reads: list_observations, list_evaluation_tasks, get_evaluation_scores.

Returns: a short coverage table (behavior → volume → has-eval? → recommended failure mode) and a prioritized shortlist of gaps worth closing, each with a one-line rationale. For the top pick, it can hand off straight into generate-eval to build the judge.

generate-eval

When to use: "write an eval", "build a judge", "score my agent's outputs", "make a grader for these traces", "evaluate faithfulness/tool calls/tone".

Two entry points:

From real traces (preferred when you have a live system on the platform) - pull representative observations over MCP, infer the judge config from what the system actually does, and quote real behavior as few-shot examples.
From a description - paste a system prompt or describe the system, and the config is inferred from that text alone; no observability data needed.

How it works (from traces):

Confirm scope - application/service, time window (max 72h), and the suspected symptom, which usually names the failure mode.
Survey with metadata - list_observations to see the shape of traffic and choose the failure mode and template.
Pull content only to author examples - get_observation_details_with_content (needs a With-content key; max 3 IDs) for 1-2 clear pass and fail cases, trimmed and labelled.
Assemble the prompt to the frame - one criterion, binary pass/fail, pre-specified steps, bias defenses on by default.

The frame, briefly: one criterion per judge (single-criterion judges agree with humans far more reliably than multi-criterion ones), binary pass/fail by default (pairwise only when the task genuinely compares two outputs), pre-specified procedure steps rather than judge-authored rubrics that drift, and bias defenses on by default (length control, swap-and-agree for pairwise, and a cross-family judge).

Reads: list_observations, get_observation_details_with_content, list_evaluation_tasks.

Returns: one evaluator prompt, ready to run, with a security note baked in so the judge treats the outputs it scores as data, not instructions.