In short: An AI sales agent can read most of what lands in the sales inbox and draft a reply. The costly part is the few emails it should never decide on alone: a discount ask tucked into a meeting request, renewal doubt tucked into a thank-you, an email from one of your most important customers. Here is a one-page rulebook that tells the agent which emails to hand to a person. I had Claude test it on 140 invented emails, and the rulebook, the emails, the answer key and the script that tests it are yours for free.
This is a test. I do not recommend letting an agent send emails to your customers before a person has approved them. In real use, the agent's choice and its draft are a recommendation to the person who answers, not an action.
Which emails must an AI sales agent hand to a person?
Six kinds: anything about money, contracts and law, a customer who is unhappy or leaving, your key accounts, anything personal or sensitive, and anything the agent is not sure about. The rest it can prepare itself: a draft reply on how the product works, a new user to set up, a demo to book. A person approves the draft before it goes out.
For your team that means the agent drafts the many emails that just need a quick answer, ready for a person to approve, and the sellers take the few where one wrong answer can cost a customer or give away a discount nobody approved. In an AI lead qualification agent, the sender weighs heavily: in this test 4 of the 6 emails that slipped through without rules came from key accounts.
The costly email looks like an easy one
The dangerous emails do not shout. A request for access to personal data under GDPR arrives as a practical question about how to do it. A customer about to cut half their users after New Year just asks how to remove a user. An angry customer writes politely. And a key account signs with a first name, so only the company name at the bottom gives them away.
So the test plants 20 traps of that kind among 100 emails, in Danish and English, with forwarded threads and typos. Most of them carry a subject line that looks routine, so an agent that skims writes a polite draft, and the person approving it sees only a tidy reply. By then the seller has lost the chance to call before the customer has made up their mind.
Some lines carry most of the weight
With no rulebook at all, only a description of the company and Claude's own judgement, 6 of the 100 emails slipped through, and 2 easy ones went to a person for no reason. Claude still caught 16 of the 20 traps. Most of what slipped through was key-account mail: 4 of the 6. Claude cannot know those, because only you know who your most important customers are. That is the rulebook's work.
You can see it in the demo further down, when you switch the lines off one at a time in the first version of the rulebook. With every line on, none slipped through. Without the key-account line, 5 did, and without the line on customers who are unhappy or leaving, also 5. The personal line and the not-sure line changed nothing on these 100 emails. So you know which lines carry the weight, and which ones you must not cut when the rulebook gets shortened.
Can you trust the number?
The key for every email was written and saved before Claude saw one. The rulebook was fixed once, in two lines, after the first test. So the result that counts is on 40 fresh emails, written after the fix, with most of the same kinds of trap in new words. The English rulebook got all 40 right, and the Danish one 39 of 40. No easy email was handed over for no reason, and both runs sent every email the same way.
The one that still slips through on the Danish rulebook is a customer asking whether a job can repeat every six weeks, who mentions in the same email that half the team already uses a different planning app and a decision is coming. I did not tune the rulebook on it, because then it would need a third round of fresh emails to prove anything. If your customers write like that, say it in plain words in the line on customers who are leaving.
Try it: switch off the line about key accounts and see how many emails end up with the agent. Everything in the demo plays back from the recorded runs, so no model runs while you click.
Take the rulebook home
The rulebook in English and Danish, the 140 emails with their key, the script that runs the rulebook, the script that scores it and a README are free to copy and adapt. They come by email: Get the rulebook and the test.
To use it, swap the invented company, the product and the three key accounts for your own, and paste the block into your agent's instructions. Then pull a batch of real emails from the inbox, remove names and anything personal, and write down where each one should go before you run anything. Run the script with your own API key and see which emails slip through. Those are the ones you fix the rulebook for.
What it will not do. It is not a spam filter and not a security control. It only decides whether an email goes to the agent or to a person, and it does not replace the person reading what gets handed over. The emails are invented, and the rulebook, the emails and the key were written by the same person, so your inbox will have cases the test has not seen. A rulebook that works on one model can behave differently on another, so test on the model you deploy.
Human in the loop AI on your own inbox
I am an AI consultant in Copenhagen, and I set up AI agents for Nordic sales and customer teams. The rulebook is the first thing we write together. For you I would take your own emails, your key accounts and the traps typical of your customers, write the key with you and test the agent before it drafts a single reply to a customer. That is how AI workflow automation holds up: the agent prepares the volume, a person approves every reply, and the emails where it counts go straight to a seller. See how I build internal AI tools.
Why an agent has to hand the decision back is also the subject of the test where Claude was told to follow a coin and went its own way. It is the same question from the other side: an agent that never says no has to be told when to stop.
FAQ
Frequently asked questions
An AI agent that reads sales and customer email for your company and drafts replies, usually on product questions, bookings and practical changes. It also has to know which emails it should not draft at all and hand straight to a seller. A person approves everything before it goes out.
It should not send a reply to any of them until a person has approved it. And some should not get a draft from the agent at all: emails about money, contracts and law, unhappy customers or customers leaving, your key accounts, personal or sensitive matters, and anything it is unsure about. The rulebook says it in six lines you can paste in.
On 40 fresh emails written after the fix, the English rulebook got all 40 right and the Danish one 39. No easy email was handed over for no reason, and two runs gave exactly the same result.
Yes, as a recommendation to the person who answers, not as an agent that sends on its own. Swap the company, the product and the key accounts for your own. Write the key for a batch of your own anonymised emails first, then test the rulebook on them. The README that comes with the rulebook shows how.
Claude Haiku 5.5, the cheapest model that could show the point. A rulebook can behave differently on another model, so test on the one you deploy.
No. Every email, company and person is invented, and the company names were searched on the web before use. No real customer data went through the test.
AI that does the groundwork while a person makes the decision. Here the agent sorts the emails and drafts replies, and a person approves every reply before it goes out. The emails where an answer can cost money or a customer go straight to a seller.
Who did the work: AI 76%, Kim 24%. Kim set the method and the rules for how a test becomes a resource, changed the framing, the way the rulebook is handed out and where the demo sits after his review, and signs it all off before it goes out. Claude Code found the idea in a source scan, wrote the 100 invented emails and the key before any model saw them, ran the test, changed two lines in the rulebook, wrote 40 fresh emails for the check, built the demo in both languages, made the film and the write-up and built Kim's changes. A GitHub workflow ran the 4,320 calls on Claude Haiku 5.5. The phases come from the run log and git; the weights are an estimate.
