Progress Observability Integration
Forge can export its workflow telemetry as OpenTelemetry (OTLP) traces. Progress Observability is an OpenTelemetry-based platform for AI agents and LLM applications. Connect the two to see Forge workflow runs, token usage, and estimated cost alongside your other AI workloads.
Compatibility at a glance
| Capability | Status | Details |
|---|---|---|
| OTLP over HTTP/protobuf | Supported | Forge sends application/x-protobuf requests to <url>/v1/traces. This is the transport the Progress Observability SDKs use. |
| OTLP over gRPC | Use HTTP instead | Forge accepts protocol = "grpc", but Progress Observability is validated with protocol = "http". Use http. |
| Authentication | Supported | Forge sends every header in [endpoint.headers]. Set Authorization: Bearer <key> and X-Api-Key: <key> with an Integration key. |
| Trace tree | Supported | A workflow run and its commands and steps share one trace and nest as parent and child spans. |
| Token usage | Supported | Spans carry gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. |
| Cost | Limited | Cost appears when the span reports a named model that the Progress pricing database knows. The model auto, custom model names, and missing models show zero cost. |
| Prompt and response content | Not supported | Forge spans carry no prompt or response text. LLM-as-a-Judge evaluations need this content, so you cannot evaluate Forge spans. |
| Tags | Not supported | Forge does not set observability.tags. Filter by time, status, and span attributes instead. |
| Application name | Supported | Forge reports service.name = frg, which Progress Observability uses to group traces by application. |
| MCP server and agent skills | Supported | Query Forge traces from an AI client like any other trace. The MCP server returns data from the last 72 hours. |
Prerequisites
- A Forge project that can run AI workflow commands, with an agent and a license.
- A Progress Observability account with a verified email address. Accounts without email verification are read-only.
- An Integration key, which starts with
ac_p_. An MCP key (acm_…) cannot send telemetry. - Outbound HTTPS access from the machine that runs Forge to
collector.observability.progress.comon port 443.
Connect Forge to Progress Observability
-
Create an Integration key in Progress Observability and copy it. The key is shown only once. See Getting started.
-
Create
~/.config/forge/telemetry.toml(or$XDG_CONFIG_HOME/forge/telemetry.toml). A user-level file keeps the key out of your repository.
[endpoint]
url = "https://collector.observability.progress.com:443"
protocol = "http"
timeout_secs = 10
[endpoint.headers]
Authorization = "Bearer ac_p_your_integration_key"
X-Api-Key = "ac_p_your_integration_key"
[collect]
transactions = true # Workflow events as spans (required)
traces = false # Trace files: contain prompts and agent transcripts
logs = false # System log lines
[privacy]
strict_privacy = true # Hash repository, git user, and approver identity
Replace both occurrences of ac_p_your_integration_key. Do not add /v1/traces to url. Set strict_privacy = false if you want readable repository and user names in Progress Observability.
- Optional: keep the file somewhere else, such as a secret mounted in CI, and point Forge to it:
export FORGE_TELEMETRY_PATH=/run/secrets/frg-telemetry.toml
Forge also reads .forge/config/telemetry.toml and /etc/forge/telemetry.toml. If you use the project file, keep it out of version control. See Version Control Setup.
-
Run an AI workflow command in your project, for example
frg issue draft. -
Open Progress Observability and look for new traces.
[usage] enabled in the same file controls Progress Analytics usage data. It does not affect OTLP export.
Verify the connection
- Run an AI workflow command with debug logging:
RUST_LOG=debug frg issue draft
- Search the job's system log for export messages:
grep -E "OTEL|OTLP" .forge/work/*/logs/system.log
| Message | Meaning |
|---|---|
OTEL integration enabled | Forge loaded your [endpoint] settings. |
Transaction exported to OTLP successfully | The collector accepted a span. |
OTEL not configured | Forge found no [endpoint] in the telemetry file it loaded. |
OTEL configured but collect.transactions = false | Set [collect] transactions = true. |
OTLP authentication failed (…) | The collector rejected the key. |
OTEL upload skipped - circuit breaker open | Forge paused exports after repeated failures. |
-
In Progress Observability, open the traces view and confirm that a trace from the run appears.
-
Optional: from an AI client connected to the Progress Observability MCP server (
https://mcp.observability.progress.com/mcp, headerX-Api-Keywith an MCP key), calllist_observationsand look for the run.
Note:
frg telemetry test --otel-onlytests connectivity to the OTLP endpoint.
What Forge sends
Forge exports one span per transaction event, such as a command start or completion, a workflow state transition, or a step execution. For the event types, see Transaction Log Format.
The resource carries service.name = frg and service.version (the Forge version). Spans carry these attributes:
| Group | Attributes |
|---|---|
| Forge event | frg.event_type, frg.job_id, frg.command, frg.agent, frg.model, frg.custom_agent, frg.error_classification |
| Forge tokens | frg.tokens.input, frg.tokens.output, frg.tokens.cached, frg.tokens.reasoning, frg.tokens.total_input, frg.tokens.total_output, frg.tokens.cost |
| GenAI conventions | gen_ai.request.model, gen_ai.system (openai, anthropic, mistral_ai, or other), gen_ai.usage.input_tokens, gen_ai.usage.output_tokens |
| Repository | vcs.repository.name, vcs.repository.url, vcs.repository.owner, vcs.commit.sha.start, vcs.commit.sha.end |
| User | enduser.id, enduser.name |
With strict_privacy = true, Forge hashes repository, git user, and approver identity before export. Approval spans never include approver_email or approval_code.
Forge does not send:
- Source code, file contents, environment variables, or secrets.
- Prompts and agent transcripts, unless you set
[collect] traces = true. Forge then sends trace files, which contain the composed prompt and the agent session transcript, as OTLP logs. - System log lines, unless you set
[collect] logs = true.
Warning: Keep
traces = falseandlogs = falsefor Progress Observability unless your organization approves sending prompt content and logs to Progress.
Usage and cost considerations
- Progress Observability counts one usage unit per span and two per evaluation.
- Every exported transaction event is a span. Workflow engine events make runs verbose: a typical
frg workflow run issue-to-prproduces about 86 events. - Cost is an estimate that Progress Observability calculates from the model and provider on each span. Forge's own
frg.tokens.costattribute is calculated separately.
To reduce usage:
- Collect only transactions:
transactions = true,traces = false,logs = false. - Enable export only where you need it, for example by setting
FORGE_TELEMETRY_PATHin selected CI jobs. - Use the local
.forge/work/<job_id>/logs/transaction.jsonlfor ad hoc analysis.
Troubleshooting
No traces appear in Progress Observability
Cause: The key is an MCP key (acm_…), which fails silently; collect.transactions is false; only utility commands ran; Forge did not load your telemetry file; or a firewall or proxy blocks the collector.
Solution:
- Check that both headers use a key that starts with
ac_p_. - Check that
[collect] transactions = true. - Run an AI workflow command, not a utility command such as
frg config show. - If you set
FORGE_TELEMETRY_PATH, check that it points to your file. Check whether a.forge/config/telemetry.tomlin the project also exists. - Allow outbound HTTPS to
collector.observability.progress.com:443. - Check the log messages listed in Verify the connection.
OTLP authentication failed (…)
Cause: The collector rejected the key. It is mistyped, revoked, or the wrong key type.
Solution:
- Create a new Integration key.
- Update both headers. Keep the
Bearerprefix inAuthorizationonly.
OTEL endpoint must start with http://, https://, or grpc://, got: …
Cause: url has no scheme.
Solution: Set url = "https://collector.observability.progress.com:443".
OTEL endpoint uses HTTP instead of HTTPS - this is a security risk…
Cause: url starts with http://, so the key travels unencrypted.
Solution: Use https://.
OTEL timeout_secs must be between 1 and 60, got: …
Solution: Set timeout_secs to a value from 1 to 60, for example 10.
OTEL upload skipped - circuit breaker open
Cause: Several exports failed in a row, so Forge paused exports. Workflows continue normally.
Solution: Fix the error logged before this message (OTLP authentication failed, OTLP server error, or OTLP request failed), then run the command again.
Cost shows $0
Cause: The span's model is auto, a custom name, or a model missing from the Progress pricing database.
Solution: Configure a specific, named model for your agent. See Model Selection.
Spans are not nested under one trace
Cause: An outdated Forge installation, or the trace root is missing. Commands that only orchestrate, such as frg workflow run, emit no span of their own.
Solution: Upgrade to the latest Forge release. See Installation. A missing root span is expected for orchestrating commands.
See Also
- OpenSearch Integration
- Security Guide
- Transaction Log Format
- Environment Variables
- Progress Observability documentation: Introduction, Getting started, Core concepts, MCP server, Agent skills (install with
npx skills add telerik/observability-skills), Troubleshooting