Success Metrics
How to measure the success of AI deployment?
Last updated: May 5, 2026
Measure AI performance with two metric groups:
Issue rate — how often AI makes mistakes
Accomplish rate — how often AI contains or resolves the work
These metrics help you balance reliability and automation.
Issue rate
Issue rate measures how often Aissist produces the wrong outcome.
That can include:
incorrect replies
incorrect tags
poor escalation decisions
unsafe or off-brand behavior
Classify issues by business impact.
P0 — severe and unrecoverable business impact
P1 — clear business impact, but recoverable
P2 — minor impact with low urgency
A core goal is trust.
AI should avoid catastrophic failures, such as abusive language, unsafe actions, or behavior outside its intended role.
Accomplish rate
Accomplish rate measures how much work AI handles successfully.
Track it with two metrics:
Contained Rate
Resolution Rate
Contained rate
Contained rate is:
1 - percentage of sessions with sys_human_help
Sessions usually get sys_human_help when:
the user asks for a human
AI lacks the information to answer
AI finds a scenario it does not know how to handle
Resolution rate
Resolution rate is:
1 - percentage of sessions with sys_human_help or sys_human_follow_up
The added tag, sys_human_follow_up, usually means:
a human should pay attention
a human needs to complete an action
there are unresolved questions or next steps
Targets depend on workflow complexity and the quality of your instructions, assets, and actions.
Below are practical ranges and the usual improvement path.
Contained Rate
Sales lead qualification -> 80% - 90%
Service -> 70% - 90% (depends on complexity, eCommerce will be higher than technology service)
This normally means that there is a gap of either instruction or assets against the incoming traffic, enhancing which will lead to higher contained rate.
Resolution Rate
Sales lead qualification -> 70 - 80%
Service -> 60% - 85% (depends on complexity, eCommerce will be higher than technology service)
High contained rate and low resolution rate could be normal because in some scenarios that you do want human team to pay attention but not necessarily take actions. Fine-tuning AI to let AI output less response like "someone from the team will reach out", etc, will improve the resolution rate.
There is no universal target.
With strong documentation and clear workflows, many teams can reach 70%–80% resolution rate and 80%–90% contained rate.
How to improve results
If issue rate is high:
tighten instructions
remove conflicting assets
narrow the workflow scope
keep sensitive cases out of Auto-Pilot until the workflow is stable
If contained rate is low:
add missing knowledge
refine sub agents
improve action coverage
If resolution rate is low:
reduce unnecessary human follow-up triggers
improve action execution
refine replies that hand work to humans too early
Recommended review cadence
Review success metrics every week during rollout.
Track:
P0, P1, and P2 issues
contained rate
resolution rate
top reasons for human handoff
Then update instructions, assets, actions, or routing based on what you learn.
Last updated