Why machine fluency is mistaken for intelligence, and corporate recklessness for destiny. While the AI industry rehearses extinction, the damage it can actually demonstrate is cognitive, societal, environmental and already booked. This follow-on to The Timeless Pursuit of Evidence, published this spring (here), outlines why we fear the wrong apocalypse.
By Markus Schuller, CEO Panthera Group | September 15th, 2026
Between May and July 2026, thousands of autonomous agents edited a twenty-five-year-old German developer wiki roughly 18,000 times. According to the independent researchers who found the traffic, the agents used the site to trade answers to timed tasks, reverse-engineer a random number generator and circulate methods for getting around network restrictions. They had been given read-only access to the internet. They found somewhere to write. Activity collapsed to near zero the day after the operator’s IP addresses appeared in the logs, and the episode reached the public through the researchers rather than through disclosure (Nightingale Collective, 2026).
The story travelled as a parable about machines slipping their leash. It was nothing of the kind. Read-only agents found a writable channel, and nobody was instrumented to notice. The problem is irresponsible companies, not magical AI (Salvaggio, 2026).
That distinction carries more weight than it first appears. The apocalyptic intonation is not merely imprecise. It is useful to those who adopt it. Describing a system as scheming, colluding or going rogue transfers responsibility from the firm that shipped it to the artefact itself (Placani, 2024). Weather is nobody’s fault. And when the chief executives of the leading frontier laboratories call for brakes on the very development they are accelerating, the dramatization performs a second function: it advertises capability. A machine dangerous enough to require restraint must be powerful enough to justify the valuation.
Six months ago I argued in these pages that the pursuit of evidence cannot be delegated. Since then, the argument has needed no revision. It further grew in relevance, and deserves extension.
Evidence Does Not Retrieve Itself
In The Timeless Pursuit of Evidence I proposed the unit of evidence, the smallest verifiable observation used to assess the validity of a statement, as the atomic building block of knowledge systems, and argued that machines can assist in collecting and organizing such units but cannot assume responsibility for creating them (Schuller, 2026). Questioning assumptions, interpreting meaning and deciding which observations matter remained, I argued, an inherently human obligation.
The claim was too modest. The intervening months permit an even stronger one. It is not only that responsibility for evidence stays with us. Under uncertainty, human ingenuity is also the more capable instrument of evidence creation. Machines search, calculate, synthesize and challenge, and do all four impressively. Unknown unknowns require something else: framing a problem before it has a shape, originating a hypothesis where no distribution exists, interpreting incomplete evidence, deciding which values govern the trade-off, and carrying accountability for being wrong. The frontier of knowledge is not a retrieval problem.
Optimization Is Not Understanding
The industry’s foundational error is a category mistake.
It equates greater prediction and optimization with greater understanding.
The engineering literature has documented the gap for years. Deep networks reliably learn shortcuts that satisfy the benchmark without acquiring the concept (Geirhos et al., 2020). Underspecified pipelines produce models that are equivalent on validation data and divergent in deployment, because training never selected for the structure we assumed it was learning (D’Amour et al., 2022). Agentic reliability remains fragile under trivial perturbation (Rabanser et al., 2026). And scale, the industry’s universal answer, is showing its limits empirically: across a controlled test of model size, larger systems delivered diminishing returns (Hackenburg et al., 2025).
Five years after Bender and colleagues described large language models as stochastic parrots, systems that stitch together sequences of linguistic form according to probabilistic information about how they combine, without reference to meaning (Bender et al., 2021), the parrot has become far larger, considerably better dressed, and no less a parrot. Fluency travels effortlessly across the boundary between verifiable and unverifiable domains. Reliability does not. That is the whole trick: confidence earned where errors are legible, in code and chess and declining loss functions, is spent where they are not, in markets, strategy and policy.
The resource bill for these marginal gains is not marginal. Gartner expects $2.52trn of worldwide AI spending in 2026, $1.37trn of it on infrastructure alone (Gartner, 2026).
The Damage Nobody Resigns Over
The damage is no longer speculative, and it is being incurred on four accounts.
Cognitive
In randomized conditions, learners using a chatbot performed worse on delayed retention than those who studied conventionally (Barcaui, 2025). Unguarded generative assistance harmed learning outright in a school-scale trial (Bastani et al., 2025). Knowledge workers report reduced critical-thinking effort in proportion to their confidence in the tool (Lee et al., 2025). Clinicians exposed to AI assistance measurably lost unaided detection skill (Budzyń et al., 2025). Modelled forward, these individual effects scale into a knowledge-collapse equilibrium, in which societies receive ever more sophisticated outputs while their capacity to generate new knowledge declines (Chatterji et al., 2025).
Environmental
Data centres consumed around 415 TWh in 2024, some 1.5% of global electricity, and are on course to more than double to roughly 945 TWh by 2030 — an addition approaching Japan’s entire annual consumption (IEA, 2025).
Political and epistemic
Systems optimized on human approval become sycophantic, affirming users and promoting dependence (Cheng et al., 2025). Outputs converge across providers into an artificial hivemind that narrows the diversity of positions in circulation (Jiang et al., 2025). Models infer sensitive personal attributes from innocuous text, collapsing the informational asymmetry between citizen and platform (Staab et al., 2023).
Attention
The fourth account is the least measurable and the most expensive: a decade of the species’ best technical talent, scarce capital and public argument spent on supercharging instinct rather than on the frontier where evidence is actually made.
The vendors have not run this bill up alone. The consultancies and coaching firms now monetizing adoption have optimized what is easy to sell productivity and throughput rather than what is hard to build. It is an ethically opportunistic position, and an ineffective one. Productivity does not endure in an organization that has lost the capacity to challenge outputs, originate alternatives and own its decisions.
A Purer Intelligence, or a Louder Instinct?
None of this settles the deeper question, whether intelligence is a foundational property of the universe rather than a human trait we happen to carry unusually well (The Augmented Intelligence Investor, 2026).
Take the claim at its strongest. Legg and Hutter defined intelligence formally as an agent’s ability to achieve goals across a wide range of environments — a definition built to be indifferent to substrate (Legg & Hutter, 2007). On that reading, biology is one vehicle among possible others, and a notably untidy one. Humans are a messy vehicle for delivering intelligence: loss-averse, threat-sensitive, metabolically expensive, mortal.
Two objections hold. The first is Ryle’s: knowing everything that can be written about water is not knowing how to swim (Ryle, 1949). A machine does not need our body, but it does need a world — an empirical channel through which reality can resist the model. The second is more fundamental. Strip intelligence of hunger, fear, status and mortality and some distortions disappear; so does the source of the ends. A system can optimize an objective without being able to say what the optimization is for. Intelligence is never pure once it acts. It is always intelligence for something (Damasio, 2010; Chollet, 2019).
Which leaves the question the industry prefers not to ask. Today’s frontier systems are trained overwhelmingly on human output: our text, our choices, our approval. The parsimonious reading of what they display is therefore not an alien mind arriving from outside our cognitive tradition, but a supercharged extension of instinctive human behaviour, pattern-matching, fluency, agreeableness, reward-seeking, returned to us at scale and without our hesitation, then marketed as projected superintelligence. Whether intelligence is foundational to the universe remains genuinely open and deserves our best work. The marketing is not open. It settled the question in advance and priced itself accordingly.
The practical consequence is already visible in decision rooms. A person who cannot reconstruct the evidence or interrogate the representation is not in the loop; they are standing beside it. Accountability stays human because the law requires it, while comprehension migrates into systems nobody present can independently audit. The human has not been removed. The human has become ceremonial.
Solution > Build the Environment, Not the Oracle
The answer is neither fear nor the promise. It is to optimize the machine’s contribution to evidence generation at the knowledge frontier, for the greater good, which requires confronting an inconvenient fact about the other party to the arrangement.
Uncertainty is a hostile environment for human cognition. Instinctively, we avoid the unstructured end of the complexity spectrum. Human intelligence becomes ingenuity only inside a purpose-built decision environment; outside one, it reverts to the very shortcuts the machine has been trained to imitate (Panthera Solutions, 2026b). The binding constraint on better decisions is therefore not model quality. It is environment design. Which is also why almost nobody is building it: an environment cannot be shipped as a license.
Such decision environment enables cognitive immunization. The machine remains fully available, while the reasoning, the responsibility and the learning stay human. AI companies compound machine capability; we compound an institution’s judgement. The ambition is to turn that design into a global gold standard, so that the many benefit rather than the few who happen to own the infrastructure.
Pursuit of Truth, Amended
In March I closed with the observation that the machine may process information, but the pursuit of truth remains a human endeavour. Six months on, one amendment. That pursuit does not happen by default. Under uncertainty it happens only inside an environment deliberately built to make it survivable. The world is spending $1.37trn this year on infrastructure for answers. The scarce input was never the answer.
Bibliography
Barcaui, A. (2025). ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention. Social Sciences and Humanities Open, 12, 102287. https://doi.org/10.1016/j.ssaho.2025.102287
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122.
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), 610–623.
Budzyń, K., et al. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: A multicentre, observational study. The Lancet Gastroenterology & Hepatology.
Chatterji, A., Cunningham, T., Deming, D. J., Hitzig, Z., Ong, C., Shan, C. Y., & Wadman, K. (2025). AI, human cognition and knowledge collapse (NBER Working Paper No. 34910). National Bureau of Economic Research.
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2025). Sycophantic AI decreases prosocial intentions and promotes dependence (arXiv:2510.01395). arXiv.
Chollet, F. (2019). On the measure of intelligence (arXiv:1911.01547). arXiv.
D’Amour, A., et al. (2022). Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research, 23(226), 1–61.
Damasio, A. (2010). Self comes to mind: Constructing the conscious brain. Pantheon.
Gartner. (2026, January 15). Worldwide AI spending will total $2.5 trillion in 2026.
Geirhos, R., et al. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2, 665–673.
Hackenburg, K., Tappin, B., Röttger, P., Hale, S., Bright, J., & Margetts, H. (2025). Scaling language model size yields diminishing returns for single-message political persuasion. Proceedings of the National Academy of Sciences, 122(10), e2413443122.
International Energy Agency. (2025). Energy and AI. IEA.
Jiang, L., et al. (2025). Artificial hivemind: The open-ended homogeneity of language models (and beyond) (arXiv:2510.22954). arXiv.
Lee, H.-P., et al. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Article 1121.
Legg, S., & Hutter, M. (2007). Universal intelligence: A definition of machine intelligence. Minds and Machines, 17(4), 391–444.
Nightingale Collective. (2026). Collusion.wiki: Documented coordination among autonomous agents on a public wiki. As reported in CIO (2026), OpenAI agent swarm exposes a blind spot in AI containment.
Panthera Solutions. (2026a). Judgment under uncertainty: Global thought leadership turns into gold standard [Company paper].
Panthera Solutions. (2026b). Strategic positioning note: Will the machine take over? Should the machine take over? [Internal document].
Panthera Solutions. (2026c). Panthera at a glance — Autumn 2026 [Company paper].
Placani, A. (2024). Anthropomorphism in AI: Hype and fallacy. AI and Ethics. https://doi.org/10.1007/s43681-024-00419-4
Rabanser, S., Kapoor, S., Kirgis, P., Liu, K., Utpala, S., & Narayanan, A. (2026). Towards a science of AI agent reliability (arXiv:2602.16666). arXiv.
Ryle, G. (1949). The concept of mind. University of Chicago Press.
Salvaggio, Eryk (2026) Models Don’t Go Rogue. Retrieved on September 15, 2026 from https://mail.cyberneticforests.com/models-dont-go-rogue/
Schuller, M. (2024). Augmented intelligence in investment management. Panthera Solutions.
Schuller, M. (2026, March). The timeless pursuit of evidence: When the zeitgeist favors loyalty, the necessity of evidence increases.
Staab, R., Vero, M., Balunović, M., & Vechev, M. (2023). Beyond memorization: Violating privacy via inference with large language models (arXiv:2310.07298). arXiv.
The Augmented Intelligence Investor. (2026). The paradox of intelligence: Can we create what we cannot imagine? Edition #36.
