An agent on revenue work marks every finished run as done, including the ones somebody on your team then had to fix, finish or stop. For the VP Marketing or founder deciding whether an agent that drafts replies, routes leads or updates the CRM gets more of the job or less, this page settles which number to keep and what it should change.
The number is the share of runs a person had to rescue. The decision comes from the reason written beside each rescue, because a rescue rate cannot tell an agent that gets things wrong from a guard that stops it for the right reason, and those two call for opposite responses.
Empact Partners, a consultancy, has spent the years since 2020 running go-to-market work inside B2B software companies, and we now build the agents their teams run, which we call marketing engineering. Measuring an agent on revenue work belongs to our RevOps workstream, the plumbing that carries a lead from first touch to closed.
A Run Needed A Person When Someone Had To Step Into It
Start with the unit, because the count is only as honest as what it counts. A run is one turn of the job the agent was given: one reply drafted, one lead routed, one article taken from an approved topic to the live page. Every run ends one of two ways. Either it reaches the result it was meant to reach with nobody stepping in, or a person has to step in.
A rescue is a person stepping into the run to do part of what the agent was meant to do alone: releasing something it stopped on, correcting what it produced, finishing it, or throwing it away. An approval is different. The design asked for it before the work began, so it counts as part of the job and never against the agent.
We run two agents on our own work, and they show where that line falls. One drafts the replies to our own prospects. The other is the content system our consultants use to research, write and publish this site.
| Agent | Planned, so not a rescue | A rescue |
|---|---|---|
| Reply agent | A teammate’s plain go on a draft that was built to wait for one | A changed line, a hold, or a teammate taking the conversation over |
| Content system | The topic a consultant approves with its instructions, and the consultant’s read of the live page | Releasing a run that stopped short of the page, notes that change the article, dropping the run |
What practitioners on Reddit say about counting the saves
“Success without intervention is the metric that matters. A long green checkmark that needed three human saves is still a supervised workflow, not autonomy.” — r/AI_Agents, Sep 2026
“I’d separate the reasons for human intervention: correcting an error, approving an action, and choosing a direction to explore shouldn’t all count as the same lack of autonomy.” — r/AI_Agents, Sep 2026
The share is then rescued runs over all finished runs, in a period short enough to act on. For an agent that runs a few times a day, read it by day and decide by the week.
Log Every Run, Including The Ones That Failed
A share is only as good as its denominator, and the denominator is where teams flatter an agent without meaning to. A failed run drops off the dashboard because nothing shipped. A run somebody restarted by hand shows up as a fresh success. And the rescue that happens a day later, when somebody reads the output, never reaches the agent’s log at all.
Keep one row per run from the first run, in a spreadsheet if that is what you have:
| Column | What it answers |
|---|---|
| Run | Which unit of work, and when it started |
| Allowed to | What the agent was permitted to do that day, so a change of scope shows in the log |
| Outcome | Finished, failed, or still going |
| Stepped in | Who, or nobody |
| Cause | Why, from a short fixed list you add to only on purpose |
| Minutes | How long the person spent, review included |
Four rules decide what goes in the denominator:
Our content system’s log already keeps most of those columns. It holds each run, when it started, how it ended, every time a person stepped into it and why the run stopped. It does not record minutes, and neither does the reply agent’s. Add that column first, for a reason the second measure makes plain.
Subtract The Rescue Time From The Hours The Agent Claims
The share tells you how often your team gets pulled back in. It cannot tell you whether those interruptions ate the saving, so the second measure is net hours: what the agent returned after every minute a person spent on its work. The arithmetic runs in this order:
- Take the baseline before the build: how long one unit of the job takes a person today, timed on a handful of real ones rather than estimated.
- Multiply it by the runs that reached their result. That is what the agent claims.
- Subtract the minutes people spent reviewing, the reviews that found nothing included, then the rescue minutes, then the upkeep: every hour spent changing the agent’s rules or fixing what it stopped on.
What is left is net hours. Read it beside the share and never instead of it, because a low share with long rescues can lose time while a higher share of one-minute releases still pays.
We cannot hand you our own subtraction in minutes. Neither agent logs them, and somebody reads every draft the reply agent writes.
What our record does show is which cost turned out to be large. Releasing a held run took one command. What cost was the decision behind it, handed back to the same person again and again, and then the session it took to fix the cause. A decision someone has to make twice belongs in the subtraction, along with the upkeep.
The Cause Of A Rescue Decides What Happens Next
We learned to read the cause before the rate between 26 October and the morning of 1 November. Our content system logged 36 finished runs on this site, failed ones included, and a person had to step into 17 of them. As one number, that is an agent needing rescue on nearly every other article, and the obvious response is to give it less.
Ten of those seventeen fell on 29 and 30 October, and all ten stopped at the same point for one reason. Each article was ready to publish, and the site showed a change waiting to go live that was not the article’s. Publishing would have shipped it too, so each run stopped and asked, as its rule said. I released every one of them by hand.
Nothing was wrong with the articles, and nothing was wrong with the stop. I had answered that exact question on 26 October, and the answer never reached the rule, so every later run asked it again.
On 30 October at 11:31 the rule changed at the root. The publish now reads what the waiting change is from the live site, records it, and goes on. In the two days since, 11 runs have finished and 1 needed a person, a duplicate I had started twice and dropped.
.png)
The strongest objection to counting rescues at all came from the same r/AI_Agents thread:
“Counting interventions tells you the gate is missing; the number only moves when something deterministic sits at each step boundary.” — r/AI_Agents, Sep 2026
The commenter is right that the number moves when a gate changes. Ours moved because a gate existed and stopped runs it should have let through, and the count is how we found it.
We wrote the lesson into the rules that same morning, and it is the one to copy. A hold that repeats for the same cause is a fault in your own system, never ten decisions for one person. Find the gate, fix it at its root, then release the held runs with a note saying what changed, so the next run does not stop on it again.
Narrow The Agent On The First Failure Nothing Could Catch
The opposite case came a week earlier, on the reply agent. On 23 October the plan was still to let it answer prospects alone until a meeting was booked. Then we saw that a prospect had received two emails from us, the day after one sender per company went into its playbook. Nothing checked that rule, so nothing stopped the second email.
That afternoon I took sending away from the agent. Every rule a script could verify became a check that refuses the email, and on its first run that check refused 14 drafts: eight promised an attachment that was not there, and five quoted a price nobody had asked for. One incident was enough, because it was a failure no gate could have caught.
Put the two weeks side by side and the threshold is a cause, never a percentage:
.png)
Widen One Rung At A Time, Where Code Can Check The Next Rung
Scope comes back the way it went, in steps, each written down before it is granted. The reply agent has five written levels of what it may do alone. The first sends nothing without a teammate, and the last leaves people only the decisions. Each level between adds named kinds of reply it may send itself. Three things hold at every level:
The agent is still on the first. Raise a level only when code can check what the next one lets the agent do alone, because a rule nobody checks is the failure that narrowed it in the first place.
The method splits into the part you can start now and the part that is somebody’s job:
Empact Partners runs the same method on the agents it builds for partners. The work starts with an audit of the work the team repeats and the systems that work writes to, and each agent is then scoped to a single job, with a named consultant answerable for its output.
The measure is settled before anything is built, whether that is hours handed back, pipeline, or a figure the team already puts in front of its board. The rescue log starts on the first run with a cause on every row, and the decisions about scope come out of it.
If your team is stepping into an agent’s work more often than anyone planned, book a call with me and let’s find out whether we are the right people to take it on.
