“Apple's paper 'The Illusion of Thinking' found that reasoning AI models such as Claude, Gemini, and GPT collapse to zero accuracy on hard puzzles even when given the exact algorithm to solve them, showing they are not really thinking but are instead very good at pattern matching.”

The paper exists, is by Apple researchers, and is not merely a preprint: Apple's own research site lists it as a main venue accepted paper at NeurIPS 2025, with a poster session on Wednesday, December 3, and the NeurIPS proceedings version is published. On the collapse finding, the paper's own words: "Through extensive experimentation across diverse puzzles, we show that frontier LRMs face a complete accuracy collapse beyond certain complexities." The paper describes three regimes, with low-complexity tasks where non-thinking models are more accurate and token-efficient, medium-complexity tasks where additional thinking shows an advantage, and high-complexity tasks where both model types collapse completely. On the algorithm experiment: "As shown in Figure 8, in the Tower of Hanoi and Checker Jumping puzzle environments, even when we provide the algorithm in the prompt, so that the model only needs to execute the prescribed steps, performance does not improve, and the observed collapse still occurs at roughly the same point on both puzzle environments." On which models were tested, the proceedings text states: "Our experiments evaluate five thinking models: o3-mini (medium and high configurations), DeepSeek-R1, DeepSeek-R1-Distill-Qwen-32B, and Claude-3.7-Sonnet", compared against non-thinking counterparts, specifically DeepSeek-R1/V3 and Claude 3.7 Sonnet thinking/non-thinking. Gemini appears only in the introduction's list of examples of this model class, alongside OpenAI's o1/o3, DeepSeek-R1 and Claude 3.7 Sonnet Thinking, not in the experiments. On the "pattern matching" framing, the paper poses it as an open question rather than a conclusion. The introduction asks whether these models are capable "of generalizable reasoning, or are they leveraging different forms of pattern matching", and the abstract states the results are "ultimately raising crucial questions about their true reasoning capabilities". The paper's substantive negative finding is narrower: these models "fail to develop generalizable problem-solving capabilities for planning tasks". The paper also states its own limits: "We acknowledge that our work has limitations. While our puzzle environments enable controlled experimentation with fine-grained control over problem complexity, they represent a narrow slice of reasoning tasks and may not capture the diversity of real-world or knowledge-intensive reasoning problems." Two published critiques contest the interpretation. The Opus and Lawsen comment argues the findings primarily reflect experimental design limitations rather than fundamental reasoning failures, raising output token limits that models explicitly acknowledge, an evaluation framework that does not distinguish reasoning failures from practical constraints, and River Crossing instances that are mathematically impossible for N greater than or equal to 6 due to insufficient boat capacity, yet scored as failures. When they changed the output format, asking for a Lua function instead of an exhaustive move list, they report high accuracy across Claude 3.7 Sonnet, Claude Opus 4, OpenAI o3 and Google Gemini 2.5 on Tower of Hanoi at N=15, in under 5,000 tokens, though they note budget constraints limited the testing. A separate replication using Gemini 2.5 Pro reports that by fixing boat capacity at 3 and increasing N beyond 5, the original experiments include unsolvable configurations, and that on valid instances even large cases such as N=100 with capacity 4, requiring up to 200 correct moves, are solved reliably. The authors responded in the later paper version. They state that restricting River Crossing to N below 6 "does not invalidate our core findings, as performance of models mostly collapse earlier from N=3, which requires only an 11-move solution", and that they ran the algorithm-provision test on an additional puzzle, Checker Jumping, which they chose because it requires fewer moves than Tower of Hanoi and appears to have less algorithmic familiarity, concluding that the results "suggest a more fundamental failure mode in reasoning models, rather than one specific to the Tower of Hanoi." On the post's Hanoi versus River Crossing point: the paper reports that for Claude 3.7 Sonnet thinking, in Tower of Hanoi the first error often occurs around move 100 for N=10, compared to River Crossing where the model produces a valid solution only until move 4, and offers the explanation that this "likely suggests that examples of River Crossing with N>2 are scarce on the web, meaning LRMs may not have frequently encountered or memorized such instances during training."