Artificial intelligence models widely used to study brain function may be achieving their impressive results through mechanisms that bear little resemblance to actual human neural processing, according to research from York University. The discovery challenges a foundational assumption in both AI development and neuroscience research—with potentially significant implications for clinical applications ranging from autism studies to PTSD treatment development.

The One-Way Street Problem

Dr. Kohitij Kar, Assistant Professor and Canada Research Chair in Visual Neuroscience at York University, led a team that developed a novel "reverse predictivity testing" method to evaluate whether AI models truly operate like biological brains. The approach, published in Nature Machine Intelligence, goes beyond traditional validation by testing the relationship bidirectionally.

Working with co-author Dr. Sabine Muzellec, a Postdoctoral Fellow with York's Connected Minds program, the researchers tested AI models using 1,320 natural photographs and synthetic images featuring ten subjects—including bears, elephants, faces, apples, and cars—plus 300 additional images in various visual styles.

The results revealed a troubling asymmetry: while AI models successfully predicted neuronal responses to visual stimuli, recorded neurons were "much less effective" at predicting many internal AI model features. Critically, this one-way predictive relationship was not observed when comparing neuronal activity across different biological brains, suggesting the asymmetry reflects fundamental differences in how artificial and biological systems process information.

Clinical Research at a Crossroads

The implications extend far beyond theoretical neuroscience. Current clinical research on conditions including autism spectrum disorder and PTSD frequently relies on AI models as tools for understanding brain function. If these models achieve accurate predictions through non-biological mechanisms, findings derived from them may require additional validation.

The concern is particularly acute for York's autism research program, which depends on accurate models of neurotypical brain function as a baseline for comparison. "If the models don't work like the brain, then we need to be cautious about using them to study brain disorders," the researchers note.

Healthcare providers and clinical researchers should exercise caution when interpreting findings from AI-based brain modeling studies, as these tools may reach correct predictions through pathways that don't reflect actual neural mechanisms. This doesn't invalidate existing research but highlights the need for biological validation before clinical translation.

A New Standard for AI Validation

The research team has released a publicly available testing toolkit for AI developers to evaluate their models using the reverse predictivity approach. This represents both a challenge and opportunity for health AI companies: while "brain-like" marketing claims may require substantiation, developers can now build next-generation systems that more faithfully replicate biological mechanisms.

For companies developing AI systems for neurological diagnostics, cognitive assessment, or brain-computer interfaces, incorporating reverse predictivity testing into validation protocols may become essential for regulatory approval and clinical acceptance.

The research was conducted through York University's Centre for Vision Research and Centre for Integrative and Applied Neuroscience, marking a significant methodological advance in how we evaluate artificial intelligence systems designed to model or interact with human neural processing.

Sources

  1. Reverse predictivity for bidirectional comparison of neural networks and biological brains — Nature Machine Intelligence
  2. AI may not work like the human brain after all, scientists discover — ScienceDaily