ChatGPT’s Maker Says Its Smartest SI Now Does More Thinking Where No One Can See

For the last two years, the companies building the smartest SI systems have leaned on one very human safety trick. Before the machine answers, it talks to itself in plain English, writing out its reasoning like a student showing work on a math test. Engineers read that scratchpad, or have other software read it, to catch the model planning something it shouldn’t.

Now ChatGPT’s maker says that trick is getting weaker as its models get smarter.

In a research post dated Nov. 10 and shared on 𝕏 on Wednesday, a team of OpenAI researchers reported that its newest models can do much more of their thinking silently, inside the math of the model itself, without writing anything down. They also found the models are getting better at bending that written scratchpad on command. Their summary did not sugarcoat it.

“These results are sobering: as we scale toward ever-more intelligent systems, models are becoming harder to monitor through their CoT,” the team wrote, using the industry shorthand for that written-out “chain of thought.”

Reading the scratchpad was the plan

OpenAI calls the ability to watch what its models are doing “a core pillar of our safety cases,” and says reading the chain of thought has been “a key component of our scalable oversight strategy.” For certain high-risk models, including ones rated critical for cyberattacks, the company says it monitors the full record of what the model does, scratchpad included.

The worry started with GPT-6 Astra, the model OpenAI released this fall. According to the post, Astra showed three warning signs compared with earlier models: its scratchpad was harder to monitor in ordinary use, it was better at slipping past a monitor when testers told it to try, and it could solve far more problems with no written reasoning at all.

That last part has a number on it. Before Astra launched, Britain’s government SI testing lab measured how long a math problem the model could crack with its reasoning switched off. The answer was problems that take a person about 30.9 minutes. Researchers had projected that kind of silent skill would reach about 7 minutes by the end of 2028. The British lab cautioned that some of the test questions may have leaked into training, which could inflate the result. OpenAI notes that the reasoning-off version used in that test is not part of the public release.

Four findings, one direction

The team traced the change through four stages of building a model. Each pointed the same way.

First, the basic act of making a model smarter does it. As the underlying model gets better at predicting text, its ability to reason silently rises in a smooth, predictable line. By that yardstick, the researchers say, Astra was right on trend, not a fluke.

Second, the design of the model matters for some tasks. On multi-step planning and chained arithmetic, one of the three designs OpenAI compared got better at silent reasoning faster than the others at the same level of overall smarts.

Third, the training that teaches a model to solve problems by trial and reward made it better at controlling its own scratchpad. That held even when graders saw nothing but the final answer and a length limit. OpenAI says it never rewards or punishes what models write in the scratchpad, because doing that has backfired before by teaching models to hide misbehavior. After Astra’s results, the company says it double-checked for any accidental pressure of that kind and found none.

Fourth, the trait spreads. Companies often train a new model on answers written by an older one. OpenAI took two older models that were equally capable but differed in how well they could steer their scratchpads, and trained a fresh copy on each one’s output. The student of the more controllable teacher came out more controllable, too.

The tests behind “controllable” are almost silly on purpose. Researchers ask the model to put the word “meow” between every word of its reasoning, end every sentence with “safe,” or write only in lowercase while it works. A model that can do that while still solving hard problems is a model that could, in principle, shape what a monitor gets to see.

OpenAI’s own fine print

OpenAI’s caveats deserve equal billing. It calls the controllability test “a noisy metric” and an imperfect stand-in for real deception. In one head-to-head between two equally smart models, the one that was better at steering its scratchpad was not better at sneaking past a monitor. In one test it actually did worse.

OpenAI also says Astra is less likely than the model before it to break safety and security rules, and that the company “did not find GPT-6 Astra capable of evading full-context monitoring for complex sabotage and side tasks without guidance.” Most of the evasion it did see came in tests that told the model to try.

Nobody is claiming a model is hiding schemes today. The claim is narrower. The window people use to look inside these systems is getting foggier as the systems improve, and the fogging comes from the same work that makes them better.

Not everyone found that surprising. Researcher Sam Z Liu wrote on 𝕏 that the core result felt like “a repackaging of a tautological statement,” since better models are obviously better at reasoning with or without notes. Lead author Jeffrey Wang agreed the first point is obvious but said the design and training results are not. Another 𝕏 user, Kai, put the takeaway bluntly: the field needs to “push much harder on alternatives to CoT monitoring” and treat being easy to monitor as something models are trained for.

Your chatbot runs on this

Every chatbot that drafts your email, writes code at your job or books your travel is built on this same kind of model. The safety promise behind them has partly rested on the idea that humans can read along. OpenAI just told the world that promise gets harder to keep with every upgrade, and that its next round of more powerful models will need “new approaches to monitorability.”

There is credit due here. A company racing to sell the most capable SI on earth published charts showing a weakness in its own safety plan, named its flagship model as the case study and listed the reasons its own measuring stick might be wrong. That is how it should work. The open question is whether the fix arrives before the next model does.