Skip to main content
Check replies before sending (“Scores each reply and tries again when it falls below your target.”) is a second check on each reply. Before a reply reaches the customer, a reviewer scores it against your instructions, your knowledge and the tools the agent has; a reply below your target score is discarded and the agent tries again with the reviewer’s notes. Use it on agents where a wrong answer is costly, such as one that quotes policies or takes orders, and accept that each retry makes the reply slower. Earlier versions of Popcorn called this switch AI Manager.

​
Before you start

  • Click Agent in the sidebar and open the agent; the switch is on the Instructions tab, under How it replies.
  • Switch on Hand over to your team on the Handoff tab if you want the agent to hand over after failed retries.

​
Switch it on

1

Enable the check

On the Instructions tab, under How it replies, switch on Check replies before sending.
2

Set the target

Fill in Minimum reply score (0 to 1). The hint reads “Higher scores ask for better replies but retry more. We recommend 0.5 to 0.7.” The default is 0.6. A value outside 0 to 1 shows Enter a number from 0 to 1, like 0.6.
3

Set the retry limit

Fill in Maximum retries, the number of times a reply is regenerated when it scores below the target. The default is 3, and it must be a whole number of 1 or more. The hint warns that more retries can slow replies down.
4

Decide what happens after the last retry

Tick Hand over to your team after the last retry (“A person takes over when every retry scores below your target.”) to hand the conversation over instead of sending a reply that still scored below target. If Hand over to your team is off, the tick box is disabled and reads Turn on hand over to your team under Handoff to use this.
5

Save and publish

Click Save and review, then Publish changes.

​
How a reply is scored

The reviewer sees the same instructions and tool list as the agent, the conversation up to the customer’s last message, and the reply the agent just drafted. It gives a score between 0 and 1 from three criteria: Each criterion is rated 1.0 (perfect), 0.7 (minor issues), 0.5 (acceptable), 0.3 (poor) or 0.0 (failure), and the weighted total is compared with your target. A target of 0.6 therefore lets through replies with minor issues and blocks replies with a significant problem in any one area. Only replies in conversations assigned to the AI are scored. Replies your team writes are never scored.

​
What happens when a reply scores below target

  1. The reply is not sent. It is kept internally as a rejected attempt together with its score and the reviewer’s note.
  2. The agent is told “The manager reviewed your last response and said it needs improvements” with the note, and writes a new reply.
  3. The new reply is scored again. This repeats until a reply reaches the target or Maximum retries is used up.
  4. After the last retry:
    • With Hand over to your team after the last retry off, the last reply is sent as it is.
    • With it on, the agent is instructed to hand the conversation over instead of answering. The conversation appears in Unassigned and in the Needs you folder, the customer gets the usual handover confirmation, and your handover alerts are sent. The reason is recorded in the conversation’s activity history.
Replies that pass are sent normally; their score and note are stored with the message. This check does not use extra credits: a customer message the agent answers costs one credit whether the reply was accepted first time or retried.

​
Choosing the settings

  • Minimum reply score: start at the default 0.6. Raise it toward 0.7 only after checking the Inbox for replies that still got through with problems; every increase means more retries on borderline replies.
  • Maximum retries: each retry is a full new reply, so a customer can wait several times the normal response time when replies keep failing. Two or three is usually enough; a reply that fails three times rarely passes on the fourth.
  • Hand over to your team after the last retry: turn it on for agents whose wrong answers you would rather not send at all. Without it, the check can only improve replies, not stop them.

​
Check replies and Spotlight

This check works on each reply before it is sent. Spotlight scores whole conversations after the fact on six dimensions and lists conversations to review. Use Spotlight to see whether the agent’s replies are good enough in general, and Check replies before sending to enforce a minimum on every reply for a specific agent.