https://orcid.org/0000-0003-0853-8334
-
Views
-
Cite
Cite
Alejandro García-Rudolph, David Sanchez-Pinsach, Eloy Opisso, Evaluating AI Models: Performance Validation Using Formal Multiple-Choice Questions in Neuropsychology, Archives of Clinical Neuropsychology, Volume 40, Issue 1, February 2025, Pages 150–155, https://doi.org/10.1093/arclin/acae068
Close - Share Icon Share
Abstract
High-quality and accessible education is crucial for advancing neuropsychology. A recent study identified key barriers to board certification in clinical neuropsychology, such as time constraints and insufficient specialized knowledge. To address these challenges, this study explored the capabilities of advanced Artificial Intelligence (AI) language models, GPT-3.5 (free-version) and GPT-4.0 (under-subscription version), by evaluating their performance on 300 American Board of Professional Psychology in Clinical Neuropsychology-like questions. The results indicate that GPT-4.0 achieved a higher accuracy rate of 80.0% compared to GPT-3.5’s 65.7%. In the “Assessment” category, GPT-4.0 demonstrated a notable improvement with an accuracy rate of 73.4% compared to GPT-3.5’s 58.6% (p = 0.012).
The “Assessment” category, which comprised 128 questions and exhibited the highest error rate by both AI models, was analyzed. A thematic analysis of the 26 incorrectly answered questions revealed 8 main themes and 17 specific codes, highlighting significant gaps in areas such as “Neurodegenerative Diseases” and “Neuropsychological Testing and Interpretation.”
