NIH Toolbox Special Series — Guest Edited by Skye McDonald

PLEASE NOTE: The CE credits indicated may only be earned by reading ALL articles in the special issue. Below, you may review the learning objectives for the entire series as well as the titles, authors, and learning objectives for each article in the series.

Learning Objectives for the Series:

Discuss the rationale behind the creation of the NIH Toolbox for Neurological and Behavioral Function-Cognition Battery (NIHTB-CB) and the derived scores
Describe how tests of the NIHTB-CB were developed to assess cognitive constructs across the lifespan
Explain advantages of novel methods used in the design of the NIHTB-CB subtests
Analyze the cognitive constructs underlying the NIHTB-CB subtests.

Article Abstracts:

The Cognition Battery of the NIH Toolbox for Assessment of Neurological and Behavioral Function: Validation in an Adult Sample. Weintraub S, Dikmen SS, Heaton RK, Tulsky DS, Zelazo PD, Slotkin J, Carlozzi NE, Bauer PJ, Wallner-Allen K, Fox N, Havlik R, Beaumont JL, Mungas D, Manly JJ, Moy C, Conway K, Edwards E, Nowinski CJ, Gershon R.

This study introduces a special series on validity studies of the Cognition Battery (CB) from the U.S. National Institutes of Health Toolbox for the Assessment of Neurological and Behavioral Function (NIHTB) (Gershon, Wagster et al., 2013) in an adult sample. This first study in the series describes the sample, each of the seven instruments in the NIHTB-CB briefly, and the general approach to data analysis. Data are provided on test–retest reliability and practice effects, and raw scores (mean, standard deviation, range) are presented for each instrument and the gold standard instruments used to measure construct validity. Accompanying papers provide details on each instrument, including information about instrument development, psychometric properties, age and education effects on performance, and convergent and discriminant construct validity. One study in the series is devoted to a factor analysis of the NIHTB-CB in adults and another describes the psychometric properties of three composite scores derived from the individual measures representing fluid and crystallized abilities and their combination. The NIHTB-CB is designed to provide a brief, comprehensive, common set of measures to allow comparisons among disparate studies and to improve scientific communication. (JINS, 2014, 20, 1–12)

Factor Structure, Convergent Validity, and Discriminant Validity of the NIH Toolbox Cognitive Health Battery. Mungas D, Heaton R, Tulsky D, Zelazo PD, Slotkin J, Blitz D, Lai JS, Gershon R.

The objective of this study is to evaluate the construct validity of the NIH Neurobehavioral Toolbox Cognitive Health Battery (NIHTB-CHB) in adults. Confirmatory factor analysis was used to evaluate the dimensional structure underlying the NIHTB-CHB and Gold Standard tests chosen to serve as concurrent validity criteria for the NIHTB-CHB. These results were used to evaluate the convergent and discriminant validity of the NIHTB-CHB in adults ranging from 20 to 85 years of age. Five dimensions were found to explain the correlations among NIHTB-CHB and Gold Standard tests: Vocabulary, Reading, Episodic Memory, Working Memory and Executive Function/Processing Speed. NIHTB-CHB measures and their Gold Standard analogues defined factors in a pattern that broadly supported the convergent and discriminant validity of the NIHTB-CHB tests. This 5-factor structure was found to be invariant across 20–60 year old (N = 159) and 65–85 year old (N = 109) age groups that were included in the current validity study. Second order Crystallized Abilities (Vocabulary and Reading) and Fluid Abilities (Episodic Memory, Working Memory, Executive/Speed) factors parsimoniously explained correlations among the five first order factors. These results suggest that the NIHTB-CHB will provide both fine-grained and broad characterization of cognition across the adult age span. (JINS, 2014, 20, 1–9)

Reliability and Validity of Composite Scores from the NIH Toolbox Cognition Battery in Adults. Heaton RK, Akshoomoff N, Tulsky D, Mungas D, Weintraub S, Dikmen S, Beaumont J, Casaletto KB, Conway K, Slotkin J, Gershon R.

This study describes psychometric properties of the NIH Toolbox Cognition Battery (NIHTB-CB) Composite Scores in an adult sample. The NIHTB-CB was designed for use in epidemiologic studies and clinical trials for ages 3 to 85. A total of 268 self-described healthy adults were recruited at four university-based sites, using stratified sampling guidelines to target demographic variability for age (20–85 years), gender, education, and ethnicity. The NIHTB-CB contains seven computer-based instruments assessing five cognitive sub-domains: Language, Executive Function, Episodic Memory, Processing Speed, and Working Memory. Participants completed the NIHTB-CB, corresponding gold standard validation measures selected to tap the same cognitive abilities, and sociodemographic questionnaires. Three Composite Scores were derived for both the NIHTB-CB and gold standard batteries: “Crystallized Cognition Composite,” “Fluid Cognition Composite,” and “Total Cognition Composite” scores. NIHTB Composite Scores showed acceptable internal consistency (Cronbach’s alphas = 0.84 Crystallized, 0.83 Fluid, 0.77 Total), excellent test–retest reliability (r: 0.86–0.92), strong convergent (r: 0.78–0.90) and discriminant (r: 0.19–0.39) validities versus gold standard composites, and expected age effects (r = 0.18 crystallized, r = − 0.68 fluid, r = − 0.26 total). Significant relationships with self-reported prior school difficulties and current health status, employment, and presence of a disability provided evidence of external validity. The NIH Toolbox Cognition Battery Composite Scores have excellent reliability and validity, suggesting they can be used effectively in epidemiologic and clinical studies. (JINS, 2014, 20, 1–11)

NIH Toolbox Cognition Battery (NIHTB-CB): List Sorting Test to Measure Working Memory. Tulsky DS, Carlozzi N, Chiaravalloti ND, Beaumont JL, Kisala PA, Mungas D, Conway K, Gershon R.

The List Sorting Working Memory Test was designed to assess working memory (WM) as part of the NIH Toolbox Cognition Battery. List Sorting is a sequencing task requiring children and adults to sort and sequence stimuli that are presented visually and auditorily. Validation data are presented for 268 participants ages 20 to 85 years. A subset of participants (N=89) was retested 7 to 21 days later. As expected, the List Sorting Test had moderately high correlations with other measures of working memory and executive functioning (convergent validity) but a low correlation with a test of receptive vocabulary (discriminant validity). Furthermore, List Sorting demonstrates expected changes over the age span and has excellent test-retest reliability. Collectively, these results provide initial support for the construct validity of the List Sorting Working Memory Measure as a measure of working memory. However, the relationship between the List Sorting Test and general executive function has yet to be determined. (JINS, 2014, 20, 1-12).

Measuring Episodic Memory Across the Lifespan: NIH Toolbox Picture Sequence Memory Test. Dikmen SS, Bauer PJ, Weintraub S, Mungas D, Slotkin J, Beaumont JL, Gershon R, Temkin NR, Heaton RK.

Episodic memory is one of the most important cognitive domains that involves acquiring, storing and recalling new information. In this article, we describe a new measure developed for the NIH Toolbox, called the Picture Sequence Memory Test (PSMT) that is the first to examine episodic memory across the age range from 3 to 85. We describe the development of the measure and present validation data for ages 20 to 85. The PSMT involves presentation of sequences of pictured objects and activities in a fixed order on a computer screen and simultaneously verbally described, that the participant must remember and then reproduce over three learning trials. The results indicate good test-retest reliability and construct validity. Performance is strongly related to well-established “gold standard” measures of episodic memory and, as expected, much less well correlated with those of a measure of vocabulary. It shows clear decline with aging in parallel with a gold standard summary measure and relates to several other demographic factors and to self-reported general health status. The PSMT appears to be a reliable and valid test of episodic memory for adults, a finding similar to those found for the same measure with children. (JINS, 2014, 20, 1-9).

NIH Toolbox Cognition Battery (CB): Validation of Executive Function Measures in Adults. Zelazo PD, Anderson JE, Richler J, Wallner-Allen K, Beaumont JL, Conway KP, Gershon R, Weintraub S.

This study describes psychometric properties of the NIH Toolbox Cognition Battery (NIHTB-CB) executive function measures in an adult sample. The NIHTB-CB was designed for use in epidemiologic studies and clinical trials for ages 3 to 85. A total of 268 self-described healthy adults were recruited at four university-based sites, using stratified sampling guidelines to target demographic variability for age (20-85 years), gender, education and ethnicity. The NIHTB-CB contains two computer-based instruments assessing executive function: the Dimensional Change Card Sort (a measure of cognitive flexibility) and a flanker task (a measure of inhibitory control and selective attention). Participants completed the NIHTB-CB, corresponding gold standard convergent and discriminant measures, and sociodemographic questionnaires. A subset of participants (N=89) was retested 7 to 21 days later. Results reveal excellent sensitivity to age-related changes during adulthood, excellent test-retest reliability, and adequate to good convergent and discriminant validity. The NIH Toolbox EF measures can be used effectively in epidemiologic and clinical studies. (JINS, 2014, 20, 1-10).

NIH Toolbox Cognitive Battery (NIHTB-CB): The NIHTB Pattern Comparison Processing Speed Test. Carlozzi NE, Tulsky DS, Chiaravalloti ND, Beaumont JL, Weintraub S, Conway K, Gershon RC.

The NIH Toolbox (NIHTB) Pattern Comparison Processing Speed Test was developed to assess processing speed within the NIHTB for the Assessment of Neurological Behavior and Function Cognition Battery (NIHTB-CB). This study highlights validation data collected in adults ages 18-85 on this measure and reports descriptive data, test-retest reliability, construct validity, and preliminary work creating a composite index of processing speed. Results indicated good test-retest reliability. There was also evidence for both convergent and discriminant validity; the Pattern Comparison Processing Speed Test demonstrated moderate significant correlations with other processing speed tests (i.e., WAIS-IV Coding, Symbol Search and Processing Speed Index), small significant correlations with measures of working memory (i.e., WAIS-IV Letter-Number Sequencing and PASAT), and non-significant correlations with a test of vocabulary comprehension (i.e., PPVT-IV). Finally, analyses comparing and combining scores on the NIHTB Pattern Comparison Processing Speed Test with other measures of simple reaction time from the NIHTB-CB indicated that a Processing Speed Composite score performed better than any test examined in isolation. The NIHTB Pattern Comparison Processing Speed Test exhibits several strengths: it is appropriate for use across the lifespan (ages, 3-85 years), it is short and easy to administer, and it has high construct validity. (JINS, 2014, 20, 1-12).

Language Measures of the NIH Toolbox Cognition Battery. Gershon RC, Cook KF, Mungas D, Manly JJ, Slotkin J, Beaumont JL, Weintraub S.

Language facilitates communication and efficient encoding of thought and experience. Because of its essential role in early childhood development, in educational achievement and in subsequent life adaptation, language was included as one of the subdomains in the NIH Toolbox for the Assessment of Neurological and Behavioral Function Cognition Battery (NIHTB-CB). There are many different components of language functioning, including syntactic processing (i.e., morphology and grammar) and lexical semantics. For purposes of the NIHTB-CB, two tests of language-a picture vocabulary test and a reading recognition test-were selected by consensus based on literature reviews, iterative expert input, and a desire to assess in English and Spanish. NIHTB-CB’s picture vocabulary and reading recognition tests are administered using computer adaptive testing and scored using item response theory. Data are presented from the validation of the English versions in a sample of adults ages 20-85 years (Spanish results will be presented in a future publication). Both tests demonstrated high test-retest reliability and good construct validity compared to corresponding gold-standard measures. Scores on the NIH Toolbox measures were consistent with age-related expectations, namely, growth in language during early development, with relative stabilization into late adulthood. (JINS, 2014, 20, 1-10).