Handling Failures with Loops and Retries
Use these recipes to repeat work until a condition holds, retry failed operations, and keep loops from running forever. Each recipe is a fragment you add to a workflow file. Run frg workflow validate <name> after every change. For field details, see the Workflow TOML Schema Reference.
Choose a mechanism
| Goal | Use |
|---|---|
| Repeat work until a check passes | A loop state with loop_enabled, loop_counter, and escape_conditions |
| Retry a flaky shell command or check | retry_count and retry_delay_seconds on the operation |
| Retry an AI command that produced incomplete output | A follow-up state with modifiers = ["retry"] |
| Stop runaway workflows | [workflow.loop_detection] and per-state max_visits |
Loop until a check passes
This complete workflow polls for a build artifact. It waits, checks, and loops back until the file exists, or aborts after 20 attempts.
workflow_schema_version = "1.0.0"
[workflow]
name = "wait-for-artifact"
description = "Poll until a build artifact exists"
version = "1.0.0"
required_context = []
[workflow.initial_state]
name = "wait"
[[workflow.states]]
name = "wait"
description = "Wait before checking again"
max_visits = 25
loop_enabled = true
loop_counter = "poll_attempts"
operation = { id = "sleep", type = "shell", command = "sleep 30", command_windows = "powershell -Command Start-Sleep -Seconds 30" }
on_success = "check_artifact"
on_failure = "poll_failed"
[[workflow.states.escape_conditions]]
counter_value = 20
action = "abort"
error_message = "Build artifact not found after 20 checks"
[[workflow.states]]
name = "check_artifact"
description = "Check whether the build artifact exists"
max_visits = 25
operation = { id = "artifact-exists", type = "file_exists", path = "dist/app.zip", on_false = "fail" }
on_success = "completed"
on_failure = "wait"
[workflow.terminal_states]
success = ["completed"]
failed = ["poll_failed"]
[[workflow.states]]
name = "completed"
description = "Build artifact found"
[[workflow.states]]
name = "poll_failed"
description = "Polling step failed"
How it works:
waitis the loop state. Each visit advancespoll_attempts.check_artifactexits the loop on success and returns towaiton failure.- When
poll_attemptsreaches 20, theabortescape ends the workflow witherror_message. max_visits = 25on both states keeps loop detection from stopping the loop before the escape condition does.
Ask a human before looping further
Replace the abort escape, or add one before it, with an approval escape that sends the run to an approval state. The built-in issue-to-pr workflow uses this pattern after 15 code-generation iterations.
[[workflow.states.escape_conditions]]
counter_value = 10
action = "approval"
approval_gate = "await_more_polling"
[[workflow.states.escape_conditions]]
counter_value = 20
action = "abort"
error_message = "Build artifact not found after 20 checks"
[[workflow.states]]
name = "await_more_polling"
description = "Polling has run 10 times"
on_success = "wait"
on_failure = "poll_rejected"
[workflow.states.approval]
gate_id = "more_polling"
message = "The artifact is still missing after 10 checks. Approve to keep polling or reject to stop."
timeout_seconds = 3600
approval_gate names the approval state to transition to. Add poll_rejected to [workflow.terminal_states].
Note:
frg workflow validatewarns that the approval state is unreachable, because escape transitions are not counted as reachable paths. The built-inissue-to-prworkflow shows the same warning forawait_iteration_approval. Warnings do not block a run.
Leave a loop for another state
Use transition to route to a fallback state instead of stopping:
[[workflow.states.escape_conditions]]
counter_value = 5
action = "transition"
target_state = "notify_team"
transition requires target_state, approval requires approval_gate, and abort requires error_message. Validation fails with, for example, Escape condition with action 'transition' requires 'target_state'.
Act on every Nth iteration
Use a counter_matches check to branch on a loop counter. This state, from the built-in issue-to-pr workflow, keeps the agent session on most iterations and routes every 3rd iteration to a state that runs code create with --clear:
[[workflow.states]]
name = "context_counter"
description = "Continue with same session? (clear context every 3rd iteration)"
operation = { id = "context_check", type = "counter_matches", counter_name = "code_iterations", counter_expression = "{{COUNTER}} % 3 != 0", on_false = "fail" }
max_visits = 30
on_success = "create_code"
on_failure = "create_code_clear"
[[workflow.states]]
name = "create_code_clear"
description = "Generating code with fresh context"
command = { target = "code", operation = "create", modifiers = ["clear"] }
max_visits = 10
on_success = "check_tasks_clear"
on_failure = "retry_code_create"
Write expressions as {{COUNTER}} <operator> N, or as {{COUNTER}} % N == M or {{COUNTER}} % N != M.
Loop an AI command until all tasks are done
frg code create usually completes a few tasks per run. Pair it with a tasks_complete check to loop until the plan's task list has no unchecked items:
[[workflow.states]]
name = "create_code"
description = "Generating code from plan"
max_visits = 30
loop_enabled = true
loop_counter = "code_iterations"
command = { target = "code", operation = "create" }
on_success = "check_tasks"
on_failure = "retry_code_create"
[[workflow.states.escape_conditions]]
counter_value = 15
action = "approval"
approval_gate = "await_iteration_approval"
[[workflow.states]]
name = "check_tasks"
description = "Checking if tasks are completed"
operation = { id = "tasks_done", type = "tasks_complete", on_false = "fail" }
max_visits = 30
on_success = "code_review"
on_failure = "create_code"
Retry an AI command with the retry modifier
When an AI command fails, typically because expected output files are missing, route to a state that runs the same command with --retry. The retry continues the agent session with a prompt listing the missing outputs.
[[workflow.states]]
name = "code_review"
description = "Reviewing generated code"
command = { target = "code", operation = "review" }
on_success = "await_review_approval"
on_failure = "retry_code_review"
[[workflow.states]]
name = "retry_code_review"
description = "Retrying to generate review.md"
command = { target = "code", operation = "review", modifiers = ["retry"] }
on_success = "await_review_approval"
on_failure = "review_failed"
Retry a flaky step or check
Set retry_count and retry_delay_seconds on a shell step or check. timeout_seconds limits each attempt.
[[workflow.states]]
name = "wait_for_service"
description = "Wait until the local service responds"
operation = { id = "health", type = "command_success", command = "curl -fsS http://localhost:8080/health", timeout_seconds = 30, retry_count = 5, retry_delay_seconds = 10, on_false = "fail" }
on_success = "run_smoke_tests"
on_failure = "service_down"
A state can also carry a [workflow.states.retry] table (max_retries, retry_delay, timeout, retry_conditions). None of the built-in workflows use it. See the schema reference.
Tune loop detection
Loop detection runs for every workflow. Raise its limits for workflows with legitimate long loops, and choose whether hitting a limit pauses or fails the run.
[workflow.loop_detection]
max_state_visits = 5 # default limit per state
max_transitions = 150 # limit for the whole run
on_loop_detected = "approval_gate" # or "fail"
[[workflow.states]]
name = "create_code"
max_visits = 30 # per-state override
# ...
- Use
"approval_gate"for iterative AI work. When a limit is hit, the run pauses at a dynamic approval with the visit and transition counts. - Use
"fail"for deterministic workflows where any repeat indicates a bug. max_visits = 0is rejected:State '<name>' has max_visits=0, which would prevent any execution.
Respond to a loop-detection approval
When loop detection pauses a run, the message lists the state, Visit count, Transition count, and two options:
- Approve: Reset counters and continue from current state.
- Reject: Terminate workflow gracefully.
frg workflow approve # show the pending approval
frg workflow approve --code <CODE> --email dev@example.com # continue
frg workflow approve --code <CODE> --email dev@example.com --resume-at code_review # continue elsewhere
frg workflow reject --code <CODE> --email dev@example.com --reason "Agent is stuck"
Troubleshooting
Loop state must have at least one escape condition
Cause: A state sets loop_enabled = true without any [[workflow.states.escape_conditions]].
Solution:
- Add an escape condition with
action = "abort"(witherror_message) oraction = "transition"(withtarget_state). - Run
frg workflow validate <name>.
Loop state <name> has no escape that guarantees termination
Cause: This is a warning. The loop's only escapes are approval or continue, so a series of approvals could keep it running.
Solution: Add an abort escape at a high counter_value. You can also accept the warning, as the built-in workflows do.
Loop detection stops a loop earlier than expected
Cause: The state's visit limit is lower than your escape counter_value. States without max_visits use max_state_visits.
Solution:
- Set
max_visitson each state in the loop to a value above the highestcounter_value. - Raise
max_transitionsif the whole run is long. - To trace limit checks, run with
RUST_LOG=debugand look forLoop detection configuration extracted,Visit count incremented, andNo per-state visit limits configured, using global defaults.