Performance Validity Testing in Neuropsychology: Ethical Considerations and Current State of the Art
Summary Abstract:
This workshop aims to provide participants with up-to-date information about the application of ethical principles to performance validity testing in clinical neuropsychology. The workshop will include a brief history of how performance validity tests (PVTs) were originally developed and about the evolution of their use. PVT’s derive their utility from their perceived insensitivity to bona fide neurocognitive or memory impairment; however, the use of forced-choice recognition has been commonplace in memory-related cognitive neuroscience research. Cognitive processes purported to underlie forced-choice recognition will be reviewed. Current practice standards and current practices in both routine clinical and forensic settings will be discussed. The fundamental designs used to develop and validate PVTs will be reviewed with attention to: 1. Simulation designs; 2. Known groups designs based on other PVT results; 3. Known groups designs based on below chance performance; 4. Known groups designs based on other criteria for non-credible performance and symptom reporting (e.g., sub rosa or surveillance videos; contradictory statements and evidence); 5. Known groups designs based on studies of groups without known incentives or people for whom motivations for secondary gain have been resolved. The basic psychometric properties of PVTs will be considered, including the implications of examining performance and test score distributions that have severe negative skewness. We will consider the use of multiple PVTs, the principles of probability chaining, and the implications of exams with multiple correlated PVTs. We will review the base rates of non-credible performance in different assessment contexts, and the implications of these estimates for calculations of positive and negative predictive power. Recent literature will be surveyed that addresses challenges to invoking the concept of malingering based on PVT analyses. Following review of the conceptual issues, Dr. Rohling will illustrate how these principles are manifest in data obtained in real-world clinical settings. These analyses will highlight ways in which simulations may fail to reflect the complexities of data observed in the clinic, and the implications of the results for general clinical and forensic contexts. Finally, we will consider future directions and the potential applications of modern psychometric theory to PVTs, particularly the use of person-fit statistics to determine validity of scores on any neuropsychological test. Throughout the workshop, the focus will be on the implications for practice, and the ethical considerations involved in balancing the risks and benefits for both our patients and society in light of the APA ethical principles of psychologists. Particular attention will be paid to methods that avoid potential bias or discrimination against minoritized patient groups, patients with diverse linguistic and educational backgrounds, and patients who have lower levels of cognitive ability.
Number of Credit Hours: 3.0
Level of Instruction: Intermediate
Learning Objectives:
1. Participants are expected to learn how the core ethical principles of the American Psychological Association apply to the use of performance validity testing.
2. Participants are expected to learn what key aspects of experimental design and psychometric properties that are important in evaluating performance validity tests.
3. Participants are expected to feel comfortable evaluating the pros and cons of different performance validity tests with respect to the likelihood that their use will yield false positive or false negative indications of invalid performance.
Presenter(s):
Robert M Bilder, PhD
UCLA Semel Institute for Neuroscience & Human Behavior
Robert M. Bilder is the Tennenbaum Family Distinguished Professor of Psychiatry and Psychology, Chief of Psychology at UCLA Health, and Director of the Center for the Biology of Creativity at UCLA’s Semel Institute for Neuroscience and Human Behavior. He is a board-certified clinical neuropsychologist and directs training programs in Clinical Neuropsychology at UCLA. His research focuses on brain and behavior, with aims to eliminate artificial boundaries between mental health and illness, and between every day and exceptional creativity. His current NIH grants examine reward mechanisms and have established a National Neuropsychology Network to aggregate data on a large scale to help develop the next generation of neuropsychological assessment methods. He helps lead the INS Worldwide Initiative for Neuropsychological Data Sharing (WINDS) special interest group, which aims to promote international data sharing and data harmonization. He has a long-standing interest in promoting innovation and technology, served as a member of the expert panel at the Minnesota Update Conference for neuropsychology and heads the Disruptive Technology Initiative for the American Academy of Clinical Neuropsychology. He also recently completed the “Big C” project to examine brain function in exceptional creativity and now directs a National Endowment for the Arts Research Lab to measure impact of the arts on well-being. He lived and worked for 28 years in New York and is delighted to be returning for INS 2024 in the Big Apple!
Martin Rohling, PhD
University of South Alabama
Martin L. Rohling, PhD is a retired professor and former Director of Clinical Training of the Combined Clinical and Counseling Psychology Training Program at the University of South Alabama. Dr. Rohling earned his first college degree in Mechanical Engineering from the University of Illinois and worked as an engineer for 5 years prior to returning to school to earn his PhD in clinical psychology. He earned both his masters and doctoral degrees from the University of Alabama. He did his clinical internship at the VA Medical Center in Palo Alto, CA. He did his postdoctoral fellowship at the VA Medical Center in Portland, OR. He has authored 130 articles, book chapters, and abstracts. Much of his research work has focused on both performance validity and symptom validity, as well as the psychometrics of a neuropsychological assessment battery. He has created a statistical procedure for analyzing the results of a neuropsychological assessment battery for a single case entitled the Rohling Interpretive Method. Dr. Rohling has lectured extensively on the use of PVT and SVTs.
Russell Bauer, PhD
University of Florida
Russell M. Bauer, Ph.D., ABPP, is a board-certified clinical neuropsychologist with over 40 years of experience in neuropsychological assessment, research, and education and training. He has been a practicing neuropsychologist within the NFL’s Baseline Assessment Program and Monetary Assistance Fund, and has devoted consistent effort throughout his career to practicing and teaching forensic neuropsychology. His expertise in memory disorders and memory assessment provide expertise with which to evaluate the cognitive/memory demands of common PVT’s, and to understand their relationship with performance on commonly used memory and neurocognitive tests.
