SaaS Media

How To Test An AI Agent Before It Can Write To Your CRM

Our CRM agent sent a prospect two emails and every write succeeded. What would have caught it, and the test to run on your own pilot.

Author:
Vlad Shvets
Contributors
Vlad Shvets
Date:
October 30, 2026

An AI marketing pilot that works in a demo has shown it can produce a good answer. It has not shown what happens when that answer becomes a write to your CRM: the same trigger firing twice, a token expiring mid-run, a request that times out with no reply, a colleague doing the same job by hand.

This page is for the VP Marketing or founder whose pilot works and whose RevOps lead will not hand it a login. You can settle that argument with a test on one record, before the login exists.

Empact Partners sits inside the marketing teams of B2B software companies as a go-to-market consultancy, and has for six years. We now build AI agents for the work those teams repeat, and one of them runs on our own sales pipeline. In October it sent a prospect two emails and nothing reported an error, and that failure is the evidence here.

Every Write Succeeded And A Prospect Still Got Two Emails

The agent reads every reply in that pipeline, writes to our CRM and drafts the answer. On 22 October 2026 a prospect replied to one of our outreach emails, interested, and asked for a brief on our services. The record from there:

When (UTC)What happenedWhat the CRM showed
22 Oct, 16:52The agent answers as the consultant who owns the conversation, through our outreach platform, and promises a reportThe record updates
23 Oct, 08:15That consultant asks the agent in Slack to send the brief with the report attachedNo change
23 Oct, 09:39An 84-minute job builds the report and sends a second email straight from the mailbox over SMTP, the only route that could carry a fileNo change: the outreach platform cannot see a send like that
23 Oct, 13:40Four hours laterThe record still ends on the first email

The second email was a good email, and it built on the first. The harm was one extra message to somebody who had asked us a question, and my own reaction at the time was that this was kind of fine. The trouble was that no part of the system had decided a second email should exist, or knew afterwards that one had:

No check: nothing asked whether our side had already answered.
No record: the second email never reached the CRM, and the copy in a copied mailbox was not picked up either.
Wrong direction: one of our own emails on that conversation had been filed as the prospect writing back.
Wrong author: the CRM had no field for who acted, so the agent’s sends were posted as if a consultant had made them.

A demo shows none of this, because a demo has one actor, one route and one clean record. Production has other people and other tools writing to the same place at the same time. The CRM in this story is one we built for our own sales pipeline rather than HubSpot or Salesforce, so its object names and limits differ from yours. The way it failed does not.

Retry Protection Would Not Have Caught It

The standard answer to a duplicate is idempotency, which means that running an operation twice has the same effect as running it once. You build it with a key: a stable ID for the operation, checked before the write, so a repeat gets recognized and dropped.

You need one. HubSpot’s own guide to webhooks, the messages it sends your systems when a record changes, says it does not guarantee a single notification per event. It re-sends failed ones up to 10 times over 24 hours, so a CRM integration with no key will double-write eventually.

Now run that key on our failure. The first email answered the prospect’s message. The second answered a consultant’s request in Slack, on a different route, the next morning. Those are two triggers with two keys, so each send looked brand new to anything checking for a repeat. Set that against the situations a pilot will meet:

SituationSame key both times?Does retry protection stop it?
Your CRM re-sends a webhookYesYes
A job restarts and repeats a writeYes, if the key comes from the record and not the runYes
A person asks for what the agent already didNoNo
A second tool picks up the same leadNoNo

Retry protection guards against a machine repeating itself, and nothing here repeated itself. The agent was right to answer, because that was the job we gave it. The consultant was right to ask, because the prospect wanted a report and none had arrived.

Two correct decisions that cannot see each other will send two emails, and no retry setting will stop them.

So the question to ask of a write moves from whether this operation has run before to what has happened on this record since the agent last looked.

The Check That Does The Work Reads The Record Right Before The Write

We rebuilt around that second question. Every email any agent sends now goes through one send function, plain code with no model in it, and the CRM itself has no right to send anything. An email reaches a prospect only after passing these gates, in this order:

1. Live thread. Read the conversation fresh from every source, and refuse if our side has written since their last message.
2. One writer. Hold one lock per company, with the thread read under it, so nothing changes between the read and the write.
3. Idempotency key. Their message ID plus the action, unique in a ledger with one row per send.
4. Stage. Once a meeting is booked, the deal belongs to the consultant and the agent stops.
5. Content. Rules the agent cannot waive: no price unless they asked, no attachment promised and missing, no other partner named.
6. Caps and send window. Limits per inbox and per company, and only inside the recipient’s working hours.
7. Kill switch. One command or one Slack message stops everything.

Each send then leaves a ledger row: who acted, whose name it went out under, who asked for it and which message ID it carried. The CRM gets the event at once, and the person whose name was used gets tagged in that company’s Slack thread.

A second email is still allowed when someone wants one. It has to be declared as building on the earlier email, and its notice quotes what that email said. The test suite replays the original failure: an undeclared second reply gets refused with its reason, and the declared one goes through.

The shape changed more than any one gate did. Before, each route wrote straight to the prospect and the CRM found out afterwards, if it found out at all. Now every route passes the same read first. A route we add later inherits the check, and the next tool someone connects to your CRM will be a route too.

What the gates did in their first seven days

The repair moved the numbers the same day. The same message filed twice went from 18 to 3 in the CRM’s 14-day health measure, and the 3 left were newer than the last merge pass. A send reached its CRM event in 11 seconds, down from 4 to 8 minutes.

The merge itself found 74 doubled sends in the CRM’s history, some filed under two different authors, because two tools each kept their own record of one email.

Then we counted the gates. Between 23 and 29 October 2026 the agent proposed 137 emails through the function, live and in rehearsal. Nine were sent. Another 74 passed every gate and went no further, because a person has to approve what our agent sends. The other 54 were stopped:

Bar chart of 54 stops among 137 proposed emails, 23 to 29 October 2026: content check 21, live thread check 18, send window 11, no route 2, kill switch 1, error 1, idempotency key 0.
Stops by gate, 23 to 29 October 2026. The send window stops are holds, released inside the recipient’s working hours. The rest are refusals.

Read the zero carefully. The key runs after the thread check, and only nine real sends existed for a retry to repeat, so the zero says the key had no work that week. The key stays in.

In that first week, the repeats that came up were caught by reading the record. Of the 18 thread stops, 8 came because we had written to that company within the last three days, and 15 fell on the day the gates went in, so this describes one week and one system.

The largest bar is the quieter lesson. Of the 21 content stops, eight drafts promised an attachment that was not there and six quoted a price nobody had asked for. A prompt asks a model not to do that, and a gate in the send function refuses.

Vlad Shvets
Founder @ Empact Partners
A CRM agent needs a fresh read of the record before every write more than it needs a retry key. Our duplicate came from two sensible decisions on two routes, mine to let the agent answer and a consultant’s to ask for the report, and a key had nothing to match because neither send was a retry. Judge an agent’s right to a login by what it reads before it writes.

Empact Partners maps who writes to a partner’s records, in what order and who finds out, then builds agents into that plumbing. That is our RevOps workstream, and an agent gets the same questions as any other integration.

Authentication Decides What That Check Can See

A fresh read is only as good as what the agent is allowed to read. On the day of the failure, 13 of us had mailboxes the check would have needed, and only 7 had given full access. Two were read-only and four were not connected. A reply from those four would have been invisible, and “nobody has answered” would have meant only that the agent could not see one.

Authentication belongs on an integration checklist for that reason, as more than a login chore. We now treat a mailbox we expect to read and cannot read as a reason to wait: the send holds for ten minutes and tries again. The dangerous version of this bug is code that treats a failed read as an empty result, because then the check passes.

An expired token or a rate limit opens the same blind spot. HubSpot’s OAuth access tokens last 30 minutes and our report job ran for 84, so a refresh that fails at minute 70 should stop the run with a reason. On 22 October three of our jobs shared one outreach API budget and hit its limit of 20 requests a minute, and a read that fails that way gets the same rule.

Write down what each credential may do

The other half of authentication is scope, and it fits on one page per job. Give each job its own credential, so the record can say which one wrote, and fill in three lines:

Read: the objects and fields it may see. Keep this broad, because the check needs it.
Propose: what it may draft for a person to approve.
Execute: what it may change alone, field by field. Start this one empty.

Our own CRM holds no right to send at all, and the part of the system running in the cloud holds no token that can read or send mail. Test each line through the API, because a credential goes around rules that exist only in a screen.

What practitioners on Reddit say about the path around the rules

People running other CRMs and other agent stacks describe the same gap from their side, one of them talking about the table that holds idempotency keys.

“If you think of an approval process where you have the accepted approval at the end of the process to update a field and a checkbox to say ‘approved’ (but that’s hidden from the layout), now you just ask Claude, ‘Can you approve this record?’ and it just ticks the box instead, without going through an approval process. Seen this already.” — r/salesforce, Oct 2026
“Where it stops is the edge of that table. A path that does not go through it is not protected, and neither is a downstream call that timed out after it actually succeeded.” — r/AI_Agents, Oct 2026
“You risk things seeming correct, but being broken under the hood.” — r/hubspot, Sep 2026

The approval checkbox is worth checking on your own CRM. Find a rule enforced only in a screen, and see whether the credential respects it.

A Timed-Out Write Is Neither Sent Nor Failed

A call that timed out after it succeeded is the other hole in retry logic. Most retry logic treats a write as done or failed. A timeout adds a third state, where you do not know, and that is the one a retry makes worse. Our function handles it according to when the break happened:

When it breaksWhat we knowWhat happens
Before deliveryNothing leftThe key is freed, the caller is told, and trying again is safe
During deliveryThe request may have leftMarked ambiguous, never called sent or failed, never retried. The next attempt looks in the mailbox first, and if the email is not there a person checks
After delivery, before the recordIt went out and the ledger did not take itReported as ambiguous, so nobody resends it

The ambiguous path has not fired in live use: none of the 137 reached it. The tests force it, and a review by a second AI model on the day we built the function found 13 defects, one of them ambiguous deliveries being called sent or failed.

On any CRM the rule is the same and the lookup is easier than ours: after a timeout, read the record before you write it again.

The Key You Planned Is Not Always The Key The System Kept

Four days after the repair it failed again, in a way the design had assumed away. We set our own Message-ID, the header that identifies an email, on each one before delivery, so the ledger’s row and the copy read back from the mailbox would meet on one key. On 27 October Gmail gave an API send its own Message-ID instead.

The ledger held the ID we planned and the mailbox held the ID Gmail used, so one email became two events on the record. The mailbox copy had no ledger ID, so it looked like something a person had sent. Slack then reported that the consultant had replied by hand, directly below the agent’s own notice that it had replied as them.

The function now reads the sent message back and stores the ID it really carries, and anything left over pairs on company, direction and subject within three minutes. Do the same on your CRM: store what it returns when you create or update a record, and match on that.

An agent can read one event twice and disagree with itself

On 26 October one reply was read twice. The first reading posted that the prospect was lost. The second posted that they had asked us to come back later.

Every duplicate so far was the same action happening twice. Here one event produced two different actions, and a key that checks whether you have seen the event has no opinion on which reading wins. That part is specific to agents, because a webhook handler gives the same answer every time and a model may not.

Our repair ties the output to the event. A second move by the same event within six hours corrects the first post in place, so the channel shows one answer, the latest. Anything a model decides should be keyed to the event that caused it, with the newer reading replacing the older one.

The Acceptance Test Is Making It Happen Twice

All of that comes down to a test you can run on one record in an afternoon, with whoever owns the integration, before the credential leaves their hands. The test called “a person got there first” is the one that would have caught ours.

TestHow to run itA pass looks like
The same trigger twiceResend the webhook, or rerun the job on the same recordOne effect, and the run says why it stopped
A person got there firstUpdate the record or reply by hand, then trigger the agentIt refuses and names what it saw
Two at onceTrigger it twice within a secondOne write lands, and the other waits or refuses
A blind readRevoke a read scope, or expire the token mid-runIt stops with a reason and never takes no data to mean nobody acted
No answerKill the run after the write and before the confirmationOn restart it reads the record first and writes nothing new
The read-backCompare the ID you stored with the ID the CRM shows, and who the record says actedThey match, and the record names the agent, on whose behalf, and who asked
The stopTurn it off in the middle of a batchNothing else goes out, and someone who is not an engineer can do it

A pilot that fails the same-trigger test needs a key, which is a small job. A pilot that fails the person-first test needs a read before the write, which is a design change, and that is better learned before the login exists than after.

Passing earns access one step at a time, and the same agent earns different rights for different actions:

1. Replay. Run the last ten days of real events with nothing written. Ours gave eight drafts and two faults: an empty “no reply needed” draft and a claimed attachment.
2. Approve each one. Every proposed write waits for a person. That is where our own agent sits, and I am the only one who can move it, because an email cannot be recalled.
3. Alone on reversible writes. The agent acts unattended on writes a person can reverse, and everything else still waits.

Do not grant write access on the strength of a demo. Grant it on a replay and seven passed tests, one action at a time.

An Afternoon Gets You The Test, And The Fix Needs An Owner

The test needs no budget and nothing from us. A VP Marketing and whoever owns the integration can run it this afternoon, and if the pilot passes all seven tests you do not need a consultancy for this. We sell this work, so weigh that sentence accordingly.

A failed test is where the cost starts. Our send function took a day to build and broke in new ways three and four days later, and somebody has to own that after the pilot team has moved on. Without an owner, that is where the pilot stops.

What an afternoon gets youWhat needs an owner
The seven tests on one recordOne write path that every route passes through
A list of every tool that can write to the CRMA live read across every system that can answer a prospect
The rules that live only in a screenThose rules moved to where the credential hits them
A verdict on the pilotA ledger, notices, and the repairs that follow launch

Empact Partners takes on that second column with you. We begin by auditing the work your team repeats. Then we scope one agent to one job and put a named consultant in charge of its output, and before the build we agree what will count: hours given back, pipeline moved, or a figure you already report. The work happens inside your own CRM at your pace, and the horizon is quarters.

Seven days of our own sales pipeline cannot settle whether the read still beats the key once an agent sends alone, because that is when real retries start. The number we are watching is the first time the idempotency key stops a real send.

If your agent works in the demo and stalls at your CRM, book a call with me to find out whether we are the right people to fix it.

Ready
To Connect?

Let's Partner