Amazon Swapped a Pricey Chatbot for a Tiny SI on a Simple Job. The Bill Fell 98%.

Amazon just showed the rest of corporate America a simple way to stop overpaying for SI: don’t hire a genius to do a sorting job.

Two Amazon engineers, Carr Onstott and Mehul Jaiswal, wrote on Monday that they pulled Claude, the popular chatbot made by Anthropic, off one of “the world’s biggest internal ticketing systems” and replaced it with a small model that does only one thing: pick the right answer from a list. They say it “performed just as well at 98% less cost” and “will save millions of dollars of token costs in a year,” according to their write-up on Amazon’s Strands Agents blog.

What the job actually is

Inside Amazon, a flood of tickets comes in every day: customer escalations, requests for more computer capacity, automated alarms, security findings. Each one has to be dropped into one of 20 buckets so the right team sees it.

For a while, Amazon had Claude Sonnet 4.6 read every ticket along with a long set of instructions, think it over, and write back a formatted answer naming the bucket. It worked. It was also like paying a lawyer to sort the mail.

The replacement is called Strands Decider, a “decision model” Amazon’s Strands team launched about a month ago. It can’t write sentences at all. You hand it the ticket and the list of choices, and it hands back how likely each choice is, plus how confident it is. The engineers say the version they used, built on Google’s free Gemma model, beat the old Claude setup on a separate test batch of real tickets once they tuned how sure it had to be before applying a label.

The numbers

By the engineers’ count, each ticket used to cost about 2.2 cents to sort. Now it costs about a twentieth of a cent. That works out to roughly $22,200 per million tickets before and about $500 after, or “about 44 times cheaper.”

It got faster too. An average ticket took about 4.2 seconds with Claude and about 0.9 seconds with the small model. The amount of text the computers had to chew through per ticket dropped 70%, partly because the small model answers with a single choice instead of a written reply.

One important caveat: these are Amazon’s own figures, published on the blog of the Amazon team that makes the new model, and they are based on list prices, not an outside audit. Amazon did not say how many tickets the system handles, so the “millions” in savings can’t be checked independently.

How they made sure it worked

The engineers didn’t just flip a switch and hope. They built a three-way setting: Claude only, both models side by side, or the small model only. In the side-by-side mode, Claude’s answers were still used while every disagreement was compared. They wrote that judging those disagreements on real traffic “told us more than any offline eval would have,” and that going back to Claude was “a 45-second config change.”

They also hit a strange glitch worth knowing about: just changing the order of fields in the data sent to the model moved its answers by as much as 0.19 on a 0-to-1 scale. They now sort every request the same way.

Why it matters to you

Companies are spending fortunes on SI, and some are already reining it in. This is a real-world example of where the money goes: big, expensive chatbots doing small, repetitive jobs. If a task is really just “pick one of these 20 options,” a tiny model can do it for pennies.

The engineers were upfront about the limits. They said it worked because the job is a fixed choice from a known list, the volume is huge and speed mattered. “If your labels need explanation text, if the catalog changes weekly, or if you can’t fit the useful context into a few thousand tokens, a decision model probably can’t replace the LLM’s job,” they wrote, using the industry term for big chatbots like Claude.

For everyone else, the lesson is plain: the smartest SI isn’t always the right one. Sometimes the right one is the cheapest one that gets the answer right.