There are two genuinely different products sold under the same phrase. One generates a reply into an agent’s composer and waits. The other generates a reply and sends it. The model is the same, the prompt is often the same, and the difference in operational risk is total, because in the first case a human is the last thing between the model and the customer and in the second case nothing is.
The useful question is not which is better. It is which categories of ticket have earned the second one, and what evidence would constitute earning it.
The spectrum has three positions, not two
| Suggest | Approve | Autonomous | |
|---|---|---|---|
| Who sends | The agent, from their composer | A reviewer, from a queue of pending sends | Nobody; it goes out |
| Latency | Agent speed | Reviewer speed | Immediate |
| Best for | High-variety tickets where context matters | Predictable categories being evaluated for promotion | Narrow, verified, low-consequence categories |
| Failure mode | Agent sends without reading | Queue backs up and review becomes rubber-stamping | A wrong answer reaches a customer with your name on it |
| Reversible | Yes, before send | Yes, before send | No |
The middle position is the one teams skip and the one that matters most, because it is where you collect the evidence. An approve queue generates exactly the data needed to decide whether a category can be automated, a paired record of what the model wrote and what a human decided about it, while still keeping every send reversible. Skipping from suggest to autonomous means making the promotion decision with no measurement. The same three-position ladder applied to agents that act rather than write is covered in AI autonomy levels.
Edit rate is the promotion signal
The measurement that should govern promotion is edit rate: of the drafts produced for a given ticket category, what share did a human send materially unchanged?
Edit rate is the right metric because it is a revealed preference rather than a stated one. It does not ask an agent whether the AI is helpful; it records what they did when the text was in front of them and they were in a hurry. It also decomposes usefully. A category with a high edit rate is telling you something specific: either the knowledge behind it is wrong, or the tone is off, or the questions in that category depend on a record the drafter never read.
Three rules for using it honestly.
Measure per category, never in aggregate. An overall edit rate of 30% might be 5% on shipping questions and 80% on billing disputes. Automating against the average is how a billing dispute gets an automatic answer.
Require volume before drawing a conclusion. A category with nineteen drafts has no edit rate, it has anecdotes. Set a floor before you look.
Distinguish an edit from a rewrite. Changing a greeting is not the same as changing the answer. If you can only measure whether the text changed at all, your edit rate will be pessimistic and you will under-automate, which is the safer direction to be wrong in.
The instrumentation this needs is worth stating plainly, because it is not free: computing edit rate requires storing both the generated draft and the text actually sent, then comparing them. If your help desk only stores what was sent, you cannot measure this, and you should treat any automation decision made without it as a guess.
Confidence decides whether, category decides if
A confidence threshold and a category allow-list do different jobs, and using either alone produces a predictable mistake.
Confidence measures how well the generated text was supported by whatever was retrieved. It knows nothing about consequence. A model can be entirely well-grounded in a refunds policy article and still be the wrong thing to send to a customer disputing a charge, because the correct reply depends on their specific transaction and on a judgement about goodwill.
So the decision is two-stage. Category decides whether auto-send is permitted at all, from a fixed list you maintain deliberately. Confidence decides whether a permitted auto-send actually fires on this particular ticket, with the fallback being an approve queue rather than silence.
| Category | Auto-send | Why |
|---|---|---|
| Order status, tracking, delivery windows | Yes, if read from the record | Deterministic answer, verifiable, cheap to correct |
| Opening hours, locations, published policy | Yes | One right answer, no per-customer variation |
| How-to and product usage | With a confidence floor | Usually documented, occasionally version-specific |
| Refunds, credits, chargebacks | Never | Creates a financial commitment you cannot unsend |
| Policy exceptions and goodwill | Never | The answer is a judgement, not a fact |
| Account access, password and data deletion | Never | Security-sensitive and frequently a legal obligation |
| Complaints and anything already escalated | Never | The customer has explicitly asked for a person |
| Anything quoting a price or date not read from a record | Never | This is the definition of a plausible invention |
The last row is the general rule the others are instances of: a reply may be automated when every fact in it came from a record or a document, and must not be when any fact in it came from the model’s sense of what is usually true.
The failure case: the note the draft read
Here is a concrete way AI drafting goes wrong that has nothing to do with hallucination.
An agent is handling a live chat about a damaged item. They leave an internal note for a colleague: “third claim from this customer this quarter, photos look staged, checking with the warehouse before we credit anything.” A second agent picks the conversation up, clicks for a suggested reply to save time, gets a fluent paragraph, skims it and sends it.
The suggestion in Oneop’s live chat is given the last ten messages of the thread, and those ten include internal notes. The draft may have been written with the suspicion of fraud in its context window. It will not usually quote the note, but it does not need to: a model given “photos look staged” as context produces a colder, more procedural reply than the same model without it, and the customer receives the temperature of a note they were never meant to see.
The help desk’s ticket reply draft is built differently. It is given the subject, the customer and their company, the priority, the original message and the last eight public turns, never the internal notes. Two surfaces in the same product, two different context boundaries, and Oneop documents both rather than letting the stricter one cover the looser one. The operational rule that follows is simple and worth telling your team today: find out which of your drafting surfaces reads internal notes, and write notes accordingly.
How Oneop positions each of these
Oneop ships three drafting surfaces and one automated one, and the send decision sits in a different place in each.
The help desk reply draft grounds on the ticket and its last eight public turns, is instructed not to invent order numbers, refunds or policies it was not given, returns a draft into the composer, and cannot send. The live chat suggestion takes the last ten messages of the thread, returns a draft, and cannot send. Both are metered against the workspace’s AI credits, and a run against a spent balance says so rather than failing quietly.
The auto-responder is the approve position, and it is off until you switch it on. Once enabled, an inbound email produces a grounded draft that is queued as a real human approval rather than sent; a mailbox can separately opt into auto-send, which is the deliberate, explicit step from approve to autonomous. Nothing moves you across that line by default.
The genuinely autonomous surface is the chatbot’s knowledge-answer block, which replies to a visitor without a human in the loop, and its safety comes from the same principle as the category list above. It retrieves from your own content, requires the model to declare whether the retrieved context supported the answer, overrides that declaration to false when nothing was retrieved, keeps confidence model-derived or null rather than defaulting it, and discards the model’s text entirely on the ungrounded path in favour of the fallback you wrote. An autonomous reply is allowed because an unsupported one is thrown away, not because the model is trusted.
Across all of them, personal data is masked out of the text on its way to the model and to the log, credits are reserved before generation so no call happens that nobody is charged for, and a call the provider fails to complete is refunded rather than billed. How those controls are governed sits on one page, AI governance.
Start at suggest, instrument edit rate per category, promote one narrow category to approve, and only consider auto-send for categories where every fact in the reply came from a record. The decision about which work should not be automated at all is a separate and equally important one, made in when not to automate; the difference between a chatbot answering customers and an agent working for your own team is in AI agents vs AI chatbots.