A Year Ago, SI Could Do 1 in 40 Real Freelance Jobs. Claude Now Does 4 in 10.
Researchers gave SI 230 real paid Upwork jobs and had human graders judge the work. Claude’s newest model passed 41%. The best a year ago passed 2.5%.

Somebody paid real money for every one of these jobs. A floor plan with photo-quality bathroom renderings. An edited video. A cleaned-up audio track. A 3D model. Each was posted by a real client on Upwork, finished by a real freelancer and accepted.
Now researchers hand the same jobs to SI and have trained human graders decide whether a reasonable client would take the result. A year ago, the best SI on the market passed about 1 in 40. Anthropic’s new Claude Fable 5.2 passes more than 4 in 10.
That is the new top score on the Remote Labor Index, a test run by Scale Labs, the research arm of Scale AI, with the nonprofit Center for AI Safety. Scale’s leaderboard puts Fable 5.2 at 41.25%. The next-best system, Anthropic’s own Opus 5.5, sits at 21.25%. OpenAI’s GPT 6 Astra is right behind at 20.83%.
Scale Labs on 𝕏, Nov. 9, 2026: “The model successfully automates 41.3% of projects, nearly doubling the performance of the next-best model Opus 5.5, a major leap in AI’s ability to handle real-world work.”
— @ScaleAILabs View on 𝕏
Real jobs, real price tags
Most SI tests are quizzes. This one is a stack of invoices.
The researchers bought 240 finished projects from 358 verified freelancers across 23 kinds of Upwork work, from video editing and computer-aided design to game building and data analysis. Every project came with the client’s brief, the input files, the deliverable the human turned in, and what the job cost and how long it took. Altogether the jobs were worth about $144,000. The average one paid $632.60 and took a human professional just under 29 hours.
The SI gets a virtual computer loaded with the tools a freelancer would use, like Blender for 3D work, GIMP for images and Audacity for audio. It gets the same brief and the same files, and it has to hand back a finished product.
Then people grade it, not machines. Trained reviewers open each file in the real software and compare it side by side with what the human delivered. The bar is simple: would a reasonable client accept this as the job they paid for? The scoreboard runs on 230 of the projects that are kept private, so no company can study the answers ahead of time.
The researchers tried letting SI do the grading and gave up. According to The Decoder’s report, SI reviewers struggled to run the professional software themselves, got fooled by work that looked slick but was broken underneath, and graded far too generously.
From 2.5% to 41% in about a year
When the index launched in fall 2025, the best system tested, an agent called Manus, finished 2.5% of the jobs at a level a client would accept. OpenAI’s ChatGPT agent still shows 1.25% on the board. Most freelancers could have laughed it off.
By July, Anthropic’s Claude Fable 5 had climbed to 15.83%. Its follow-up, Fable 5.1, edged up to 17.92%. Then Fable 5.2, released Nov. 9, more than doubled that in one jump.

Scale says the biggest gains came in audio and music editing, architecture and interior design, and 3D modeling. Nearly 50 of the jobs Fable 5.2 passed had never been passed by any SI before. The model also held together on long, complicated projects that tripped up the earlier leaders.
There is a gap in the numbers worth knowing about. Scale lists the 41.25% on its leaderboard and rounded it to 41.3% when it announced the result. Either way, it publishes no margin of error, so a point or two in either direction shouldn’t be read as meaningful. The jump from roughly 18% to 41% is a different story. That is too big to be noise.
Where it still falls on its face
Six in 10 jobs still failed. Scale says the SI can barely handle industrial product design and animation. When it misses, the work looks amateur, comes back half finished, arrives as broken files or contradicts itself, like a design that doesn’t match its own drawings.
The test also leaves out a lot of what freelancers actually do all day. It skips jobs that need physical work, long-term follow-up or back-and-forth with the client. Nobody had to calm down an angry customer, chase an invoice or ask what the client really meant.
And a passing grade isn’t the same as a lost job. The index measures how many projects an SI can finish on a fixed sample. It doesn’t estimate what share of the remote work market can be automated, and the leaderboard doesn’t claim to. The score comes from Scale and the Center for AI Safety, not from Anthropic’s own launch materials.
The people who should pay attention
Still, if you make your living on a laptop doing project work, this is the number to watch. The first version of this test said SI was nowhere close. Thirteen months later, the best model does more than 40% of the work well enough to get paid for it.
The money shows up on the other side, too. Upwork, the marketplace these jobs came from, told regulators on Nov. 12 that it is expanding its own layoffs to about 37% of its workforce. Its filing says the cuts are meant to set the company up for growth “as the nature of work evolves,” and it never mentions SI. The filing proves nothing about what caused the cuts. It does show the biggest marketplace for human freelancers shrinking its own staff in the same week SI posted its best score yet on that marketplace’s jobs.
The smart move for freelancers isn’t panic. It’s learning to run these tools faster than the competition, because the clients posting these jobs can already do the math. A $632 project that a model finishes in hours is a very different business than one that takes a person most of a week.
Sources
- Scale Labs — Remote Labor Index (RLI) leaderboard
- Scale Labs on 𝕏 — Scale Labs on 𝕏 — Fable 5.2 takes the top spot on the Remote Labor Index (Nov. 9, 2026)
- Anthropic — Introducing Claude Fable 5.2 and Claude Mythos 5.2 (Nov. 9, 2026)
- The Decoder — Anthropic’s Fable 5.2 completes 41 percent of real freelance jobs, up from 2.5 percent a year ago (Nov. 10, 2026)
- U.S. Securities and Exchange Commission — Upwork Inc. Form 8-K/A, Item 2.05 (filed Nov. 12, 2026)