AI's Attention Limitations Revealed by Psychology Tests

Although large language models (LLMs) have demonstrated impressive capabilities, a recent study published in PNAS Nexus reveals significant differences in cognitive function between these artificial intelligences and the human brain. Researchers discovered that current LLMs possess limited executive control over attention, a fundamental human ability essential for filtering distractions and focusing on relevant information to enable adaptive behavior. This finding indicates a gap in how AI processes information compared to biological intelligence.

The concept of attention has been a cornerstone of psychology since the late 19th century. Early pioneers like William James defined it as the deliberate act of concentrating on specific elements while disregarding others, emphasizing the importance of focused engagement over a "scatterbrained state." Decades later, psychologists Michael Posner and Steven Petersen proposed a model of attention comprising distinct subsystems for orienting, detecting, and alerting, further refined by the Attention Network Test (ANT) in 2002. These foundational understandings of human attention are now being applied to evaluate and guide the development of AI. While LLMs have rapidly advanced in capabilities since the public release of ChatGPT in 2022, understanding their cognitive limitations, especially concerning attention, is crucial for future advancements.

To assess the attentional control of AI, the researchers employed the Stroop test, a widely recognized psychological assessment for executive functioning. This test challenges participants to name the color of a word printed in a conflicting color (e.g., the word "blue" printed in red ink). The study evaluated several leading LLMs, including GPT-5, GPT-4o, Claude Opus 4.1, Claude Sonnet 4.5, Claude 3.5 Sonnet, and Gemini 2.5 Pro. The results showed that LLMs performed comparably to humans on short word lists, but their accuracy drastically declined with longer lists, unlike human performance. This "progressive degradation" suggests that LLMs struggle with maintaining goals under prolonged task demands, highlighting that the issue is not merely a memory deficiency but a need for improved conflict resolution in their attention mechanisms. Integrating more advanced cognitive control systems into AI architectures is suggested as a critical step toward achieving artificial general intelligence.

The insights gained from studying human attention systems provide a roadmap for developing more sophisticated AI. By incorporating mechanisms that mimic the brain's ability to prioritize and process information, future AI models can overcome current cognitive limitations. This interdisciplinary approach, bridging psychology and computer science, promises to unlock new frontiers in artificial intelligence, creating systems that are not only powerful but also adaptive, intelligent, and capable of nuanced understanding, ultimately contributing to beneficial technological advancements that enhance human lives and societal progress.