AI
“AI researcher Nick Cammarata, formerly of OpenAI and now at AI startup Goodfire, published a nearly 26,000-word essay arguing that AI models are not impenetrable black boxes and that there may be a way to see how they actually 'think.'”
Plain restatementNick Cammarata, a former OpenAI interpretability researcher now at Goodfire, published an essay of roughly 26,000 words arguing that the internals of AI models are more readable than the prevailing "black box" view holds.
This one checks out in its essentials. Nick Cammarata, who worked on interpretability at OpenAI and is now at the startup Goodfire, really did publish a very long essay on 15 October 2026 titled "Neural Networks Are Already Clean," arguing that the inner workings of AI models are far more readable than the field assumed. The length, roughly 26,000 words, matches what multiple outlets and the essay's own 65-minute reading estimate suggest. What the post leaves out is that the essay calls itself a working hypothesis, describes its key measurements as preliminary, and was published on the website of a company that sells interpretability work, with a technical companion paper still to come. No independent group had reproduced the results as of 24 October 2026, and two named researchers, Neel Nanda at Google DeepMind and Leo Gao, have publicly argued it does not show that interpretability is solved or even much easier. The post's framing that this points to a way of solving AI alignment goes well past what the essay demonstrates, which is about whether individual neurons have readable jobs, not about detecting or stopping misbehaving AI systems.
[drifted from the evidence:] AI researcher Nick Cammarata, [drifted from the evidence:] formerly of OpenAI [drifted from the evidence:] and now at [drifted from the evidence:] AI startup Goodfire, published [drifted from the evidence:] a nearly 26,000-word essay arguing that AI models are [drifted from the evidence:] not impenetrable black [drifted from the evidence:] boxes and that there may be a way to see how they actually 'think.'
Nick Cammarata, [added by the neutral restatement:] a former OpenAI [added by the neutral restatement:] interpretability researcher now at Goodfire, published [added by the neutral restatement:] an essay [added by the neutral restatement:] of roughly 26,000 words arguing that [added by the neutral restatement:] the internals of AI models are [added by the neutral restatement:] more readable than the prevailing "black [added by the neutral restatement:] box" view holds.
Red-tinted words in the claim drifted from the evidence. Green-tinted words are what a neutral restatement needs.
The trace / claim to source
- The essay exists, is titled "Neural Networks Are Already Clean," was published 15 October 2026, and is bylined Nick Cammarata of Goodfire.
- Cammarata was previously at OpenAI. The 2020 Distill circuits work lists him with an OpenAI affiliation.
- The length description is close. Multiple independent reports give roughly 26,000 words, one unaffiliated reader estimates about 27,000 words including the appendix, and the essay itself advertises a 65-minute read.
- The thesis is described fairly. The essay argues that most neurons in models up to trillions of parameters can be understood and that the outlook for mechanistic interpretability may be far better than thought.
- The hedge in the claim, "there may be a way," matches the essay's own hedging rather than overstating it.
- The quoted reactions check out. Altman posted brief praise on Oct 16, the "major moment in scientific history" line traces to roon, and the Nanda quotation is reproduced consistently across several outlets.
- The post includes a named dissenting voice, which is a point in its favour relative to the surrounding framing.
- The essay labels itself a working hypothesis with preliminary audits and a technical companion still to come, and it flags one step of its own argument as highly speculative. The post slides, headed "EXPLOSIVE DOCUMENT CLAIMS THAT AI IS NOT WHAT WE THINK," carry none of that provisionality. A reader comes away thinking a result has landed when what landed is a hypothesis its author has asked the field to test.
- The post's images say that in the eyes of AI leadership this "means that there may be a way of 'solving' AI alignment," and that it would let people figure out why agents go rogue and put a stop to it. The essay's evidence concerns whether individual neurons have readable functions. Nothing in it demonstrates detecting or preventing a rogue agent or a guardrail break. Neuron legibility is a proxy for, not a demonstration of, behavioural control.
- The essay is published on the website of Goodfire, a commercial interpretability lab, and has not been peer reviewed or independently reproduced. The post presents it as a finding about the world rather than as a claim by an interested party awaiting outside checking.
- Benchmark cherry picking, in the comparator sense: the headline contrast is between the new essay and a 2023 OpenAI result that found a very small share of neurons interpretable. Those two numbers come from different models, different tools and different definitions of what counts as "one job," so the size of the apparent leap depends heavily on the baseline chosen.
- The claim and the post speak about "AI models" in general. The detailed evidence is concentrated in small vision models, and the headline language-model figure comes from a preliminary audit of one open model, Qwen3-32B.
- The post reports Nanda's dissent but omits the more technical published rebuttal from Leo Gao, which argues the essay's observation is real but far short of showing that interpretability is solved or vastly easier.
- Whether the roughly 90% legibility estimate survives independent replication. No outside reproduction had been published as of 24 October 2026, and the essay itself says the figure moves with thresholds and definitions.
- Whether the promised technical companion paper has been released. I found only references to it as forthcoming.
- The exact word count. "Nearly 26,000" is consistent with every report I found and with the essay's stated reading time, but estimates range from about 26,000 to about 27,000 depending on whether the appendix is counted.
- Whether the picture holds for frontier closed models. The strongest demonstrations concern older vision models and one open-weights language model.
The essay exists and matches the description. The page is titled "Neural Networks Are Already Clean," is bylined Nick Cammarata of Goodfire, and states that the cause of polysemanticity in early model organisms was not superposition but several ordinary, mostly fixable issues, that most neurons in models up to trillions of parameters can now be understood, and that the outlook for mechanistic interpretability may be far better than previously thought. The page labels itself "A working hypothesis · 65-minute read." Multiple outlets date it to 15 October 2026, and an independent reader working through it estimates about 27,000 words including the appendix, noting it is written as a working hypothesis rather than a paper. On the author's history: the 2020 Distill "Thread: Circuits" lists Nick Cammarata with an OpenAI affiliation, alongside Chris Olah and others, which supports the "formerly of OpenAI" description. On the reactions the post cites: Sam Altman's X page shows an Oct 16 post reading "Very happy to see this from Nick! @nickcammarata." The roon quote reads that he may be remembered as having lived through "a major moment in scientific history." Neel Nanda, who leads mechanistic interpretability at Google DeepMind, wrote on X that the work was interesting but "not a significant update to the big picture of interpretability," adding that he still considers superposition real. The essay also drew a substantive technical rebuttal that the post does not mention. Leo Gao published "Neural networks are not already clean" on LessWrong on 17 October 2026, agreeing that people have been too quick to write off neurons as polysemantic, while arguing it is "nowhere near a sufficient argument that we understand most of what neurons are doing, and these results do not show that interpretability is solved, or even that it is vastly easier than previously anticipated."
Complete reasoning
The reply is formatted for pasting into the thread where the claim is circulating.
Ask this case
Answers come only from the case file above; nothing is added.
Did Nick Cammarata really write a 26,000-word essay claiming AI models aren't black boxes?
Yes. He published an essay titled 'Neural Networks Are Already Clean' on Goodfire's website on 15 October 2026, running roughly 26,000 to 27,000 words depending on whether the appendix is counted, and it argues most neurons in models up to trillions of parameters can be understood.
Is Cammarata actually a former OpenAI researcher?
Yes. The 2020 Distill paper 'Thread: Circuits' lists him with an OpenAI affiliation alongside Chris Olah and others, supporting the description of him as formerly of OpenAI.
Does this essay prove that AI alignment can be solved?
No. The essay's evidence concerns whether individual neurons have readable functions, not whether researchers can detect or stop a misbehaving AI system. The case file notes that framing the essay as a path to 'solving' alignment goes well past what it actually demonstrates.
Has anyone pushed back on Cammarata's findings?
Yes. Neel Nanda of Google DeepMind called it interesting but not a significant update to the field and said he still considers superposition real. Leo Gao published a more detailed rebuttal on LessWrong arguing the results are real but fall far short of showing interpretability is solved or vastly easier.
Has this essay been independently verified?
The investigation did not establish this. As of 24 October 2026, no independent group had reproduced the results, the essay itself calls its findings a working hypothesis with preliminary audits, and a promised technical companion paper had not yet been released.