A recent paper, The Illusion of Thinking , seems to prove that AI is not thinking at all, but is a very sophisticated pattern recognition algorithm. We speak here of LRMs, Large Reasoning Models. Among the many findings are • We question the current evaluation paradigm of LRMs on established math benchmarks and design a controlled experimental testbed by leveraging algorithmic puzzle environments that enable controllable experimentation with respect to problem complexity. • We show that state-of-the-art LRMs (e.g., o3-mini, DeepSeek-R1, Claude-3.7-Sonnet-Thinking) still fail to develop generalizable problem-solving capabilities, with accuracy ultimately collapsing to zero beyond certain complexities across different environments. • We find that there exists a scaling limit in the LRMs’ reasoning effort with respect to problem complexity, evidenced by the counterintuitive decreasing trend in the thinking tokens after a complexity point. The full article i...
Random thoughts and used ideas