An AI agent is worth funding for back-office exception work when three things are true: the exceptions are frequent, resolving them takes judgment against documented rules, and the agent can reach your systems through the same APIs your staff use. If the work is fully predictable, simpler rule-based automation is cheaper. Two recent AWS case studies, from Worldwide Express and MoEngage, show what separates a sound project from an expensive experiment.
What does a well-scoped agent project look like?
Logistics company Worldwide Express had billing exceptions piling up: invoices that failed the standard three-way match between purchase order, goods receipt and carrier invoice. In less-than-truckload (LTL) freight, roughly 14 percent of invoices fail that match. Each one was researched and disputed by hand, carrier by carrier.
- 14%of less-than-truckload invoices fail the three-way match
- About halfof exceptions closed by the agents, six months in
Source: AWS Machine Learning
The company did not build one general assistant. It routes each exception to one of three specialized agents: one for invoice matching, one for pricing variances and one for unexpected accessorial fees. Each agent pulls that carrier’s billing rules and procedures before it recommends anything, then writes a plain-language note explaining what it found and did.
According to the AWS write-up, six months in the agents close about 10,000 exceptions a week, roughly half the total, with an error rate under 1 percent. The company reports that its backlog of active exceptions fell from $15 million to $4 million and that time-to-cash improved by four weeks.
Down 73%
The company also reports about 10,000 exceptions closed a week, with an error rate under 1 percent.
Source: AWS Machine Learning
Is your process a fit for an agent or for simpler automation?
The Worldwide Express team draws the line clearly. Rule-based process automation handles structured, predictable work. Exception handling needed contextual judgment: reading a carrier’s fee policy, comparing charges with shipment data and deciding whether a charge is valid. That variability is what made it a fit for agents.
Their advice for getting started is also worth copying. Automate the most repetitive exception type first, prove accuracy, then scale to the rest.
- Good fit: high volume, written rules that vary by customer or supplier, and a decision a trained person makes in minutes.
- Poor fit: steps that never vary, which plain automation already handles, or decisions with no written rules to check against.
What drives the cost and effort?
The model is rarely the hard part. Both case studies spend most of their effort on the work around it.
- Documented rules. Worldwide Express stores each carrier’s tariff rules, surcharge definitions, dispute steps and variance thresholds so every decision is grounded in that carrier’s own rules. If your rules live in people’s heads, writing them down is the first cost.
- Access to your systems. Their agents use the same APIs analysts use, so the AI sees exactly what a human reviewer sees. Systems without APIs add integration work.
- One integration layer, not many. MoEngage had about 400 internal APIs built by 25 teams. It rejected running more than 20 separate MCP servers and used one managed gateway that turns each team’s OpenAPI spec into agent tools.
- Review and measurement. Plan for an approval step, dashboards and a log of every rejected recommendation.
How should identity and permissions be handled?
This is where many pilots are weakest. MoEngage’s rule is that identity cannot be the agent’s decision. Tenant, user and role details travel as request headers added by the calling service, not as tool inputs, so the model cannot see or change them.
Permissions are also enforced on the server. Each agent has its own role and policy, and an agent that should only read never receives the write tools at all. MoEngage also kept every API off the public internet. Its write-up estimates that the approach saved six months of platform work and that onboarding a new API now takes hours instead of days.
Why keep a person in the loop at first?
Worldwide Express started with a reviewer approving every recommendation in its existing transportation management system. Only when accuracy exceeded 99 percent did bulk approval become the default. The team tracks automation rate, error rate, resolution time and how often people override the agent, and uses rejected recommendations to improve the instructions and knowledge base.
That design also kept financial controls in place, which matters when an agent’s output moves money.
What should you ask a vendor or partner?
Use these questions to compare proposals. A good partner will answer them in specifics for your systems.
- Which exception type do we automate first, and how will accuracy be measured?
- Where does a person approve results, and what accuracy lets us relax that step?
- How will the agent reach our systems: through our existing APIs, or new integrations?
- How is identity kept out of the model’s control, and can an agent see actions it should not take?
- What is logged when someone rejects a recommendation, and how does that improve the agent?
- Can we run a short pilot on our own data before committing?
| Approach | Best for | Watch out for |
|---|---|---|
| Rule-based automation | Predictable steps that never vary | Breaks when a decision needs judgment |
| AI agent with human approval | Frequent exceptions checked against written rules | Needs documented rules, API access and review time |
| AI agent with bulk approval | Exception types with proven, tracked accuracy | Only after accuracy is measured over time |
What should you do next?
Pull last quarter’s exceptions from one process, such as invoice disputes or order corrections, and sort them by type and volume. Pick the most repetitive type, check that its rules are written down and that your system exposes it through an API, then scope a pilot with human approval and clear accuracy targets. If you are weighing this for your business, see how we approach LLM integration and AI agent development, or book a meeting with our team.
Frequently asked questions
How long does a project like this take?
Worldwide Express moved from assessment to production in less than six months. Your timeline depends mostly on how well your rules are documented and whether your systems have APIs.
Do we need a separate MCP server for every system?
Not necessarily. MoEngage rejected running more than 20 MCP servers and used one managed gateway that generates agent tools from existing OpenAPI specs.
Should an AI agent make final decisions about money?
Not at first. Worldwide Express had a person approve every recommendation and moved to bulk approval only after accuracy exceeded 99 percent.
Sources
- How Worldwide Express automated freight exceptions with Amazon Bedrock, AWS Machine Learning, 2026-10-30
- How MoEngage turned over 400 APIs into agent tools with AgentCore Gateway, AWS Machine Learning, 2026-10-29
Researched from the sources above with AI assistance and checked by the Aranv team’s publishing standard.