Learning from failures,
one workflow at a time.

When an agent fails, the final answer rarely tells the whole story. A failure might come from a missing check, an unhelpful handoff between agents, or a workflow that sends the same kind of task down the wrong path.

Failure-Driven Workflow Refinement studies how those failures can become a useful signal for improving the workflow itself. The framework, CE-Graph, organizes counterexamples into recurring modes and uses them to guide verified changes.

A failure is more than a score.

A single evaluation score can tell us that a workflow needs work. It gives much less information about where to intervene. Looking at individual executions makes a different question possible: which parts of this workflow repeatedly contribute to unsuccessful outcomes?

The unit of improvement becomes the workflow graph: the agents, their roles, and the connections that determine how information and intermediate results move through the system.

Use failures to decide where to change the workflow, then check whether the change helps.

From counterexamples to workflow edits.

CE-Graph follows a structured feedback loop rather than treating every unsuccessful execution as an unrelated incident:

1. Describe the failure. Extract signatures from failed executions so that the evidence is more specific than a correct-or-incorrect label.
2. Find recurring modes. Group related signatures to identify patterns across counterexamples.
3. Propose and verify. Use those patterns to guide workflow edits and evaluate the proposed changes.

Make the feedback actionable.

The connection between diagnosis and intervention matters. A recurring pattern is useful when it helps identify a concrete change that can be tested, such as revising an agent role or changing the flow of information between steps.

Verification closes that loop. A proposed edit is a hypothesis about how to improve the workflow; its value depends on what happens when the edited workflow is evaluated.

execute the workflow
  → collect failed executions
  → extract and group failure signatures
  → propose targeted graph edits
  → verify the proposed changes
  → repeat

A direction for reliable agents.

This work fits into a broader interest in agents that learn from their own limitations. Expert selection, reasoning under uncertainty, and workflow refinement each ask how an agent system can use evidence to make better decisions.

For CE-Graph, that evidence comes from failures. The goal is to connect what went wrong in an execution to a specific, testable improvement in the system that produced it.

Paper and implementation