Aim, design, and setting
Aim is to develop and validate a cognitive impairment indicator that summarizes cognitive performance across a neuropsychological battery. Prospective cohort design, but cross-sectional analyses for the current project. Setting is community-based.
Participants
CLSA participants have been described elsewhere (1). Briefly, two stratified random samples from the Canadian population between the age of 45 and 85 years were selected. The Tracking cohort (N=21,241) was administered questionnaires over the telephone; including yes/no questions about having been diagnosed by a physician as having a chronic condition (34 conditions), four neuropsychological tests (see (35) for a description of the data collection protocol and tools). Participants in the Comprehensive cohort (N=30,097) were assessed in person either in their homes or at one of eleven data collection sites across Canada. The descriptions of the two cohorts are shown in Table 1.
Measures
Neuropsychological Tests
The neuropsychological tests used in CLSA are described in more detail elsewhere (36, 37), but included the following tests administered by telephone to the Tracking cohort: Rey Auditory Verbal Learning Test immediate recall (REY I) and five minute delayed recall (REY II), the Mental Alternation Test (MAT), and Animal Fluency (AF; we used AF2 scores that are consistent with scoring rules used clinically (14)). The Comprehensive Cohort completed testing in-person, including the above four tests, as well as the Controlled Oral Word Association Test (total score was used for the letters FAS) and the Victoria Stroop Test (Stroop Interference - interference ratio time it took to complete the task on the “color” task divided by performance on the “dot” task). For all but the Stroop test, higher scores indicated better cognitive performance; for the Stroop Interference score, lower scores indicated less interference. Summaries of the raw performances on these tests (i.e., not normed) for the whole sample in both cohorts are shown in the top of Table 2. To create normatively corrected scores (15) Stroop Interference scores were reversed, making interpretation of these scores consistent with the other neuropsychological tests, and for all analyses the norm corrected Stroop scores were used.
Chronic Conditions
Participants were asked to respond yes/no to the question: “Has a doctor ever told you that you have (the chronic condition)?” The list of conditions, and the number of participants who responded yes to each, is shown in Table 3. We examined each condition separately for its association with the CII, and we created three groupings based on our a priori hypotheses. One group labelled “Neurological” included participants who reported having a physician diagnosis of dementia or Alzheimer’s disease, memory problems, stroke, transient ischemic attack, multiple sclerosis, parkinsonism or Parkinson’s disease, or epilepsy; versus those who denied any neurological condition. A second group labelled “Risk Neurological” included participants with a self-reported physician diagnosis of at least one known risk factor for cognitive impairment: diabetes, hypertension, cardiac diseases, major depressive disorder, peripheral vascular disease, kidney disease, or thyroid dysfunction versus those reporting none of these conditions. To provide support for the CII with divergent validity, a third group was created with participants who had self-reported conditions for which we did not expect to see increased likelihood of cognitive impairment: allergies, arthritis, migraines, osteoporosis, history of cancer, ulcers, or back pain. We were unable to create a comparison group of persons who reported none of these conditions because too many participants in each cohort had at least one of these conditions. Consequently, the third grouped condition was modified to include arthritis, migraines, osteoporosis, history of cancer, or ulcers with the comparison group comprised of those reporting none of these conditions.
Analytic Approach
Derivation of the Cognitive Impairment Indicator (CII)
For each cohort, the derivation of the CII involved three steps. First, on each neuropsychological test, each participant’s raw score was transformed to a normed score based on comparisons with the neurologically healthy normative sample (14), and normed with respect to the participant’s age, sex and education within each language group (referred to hereafter as “normed scores”). In the second step, the participant’s normed score was used to obtain their low score indicator (impaired versus within normal limits) for each neuropsychological test by comparing the participant’s normed score to the cut-off point for abnormally low scores. The cut-off point was the mean from a bootstrapped distribution of scores from the normative sample corresponding to the 5th percentile for each test score. In the third step, the CII was determined for each participant based on her/his performance across the battery of neuropsychological tests. This classification into overall impaired versus non-impaired for the CII incorporated baserate of low scores. In particular, baserates of the expected proportions of a cognitively healthy population estimated to demonstrate cognitive impairment on any given test was determined using the algorithm created by Crawford and colleagues (12). The Crawford et al. (12) algorithm uses a Monte Carlo-based method to estimate the probability of obtaining a given number of abnormally low scores. The probability of abnormally low scores increases as the number of tests in the battery increases and is dependent on the test scores’ intercorrelations. This baserate algorithm is based on the intercorrelations of the neuropsychological tests in the cognitively healthy sub-sample (i.e., the normative sample). The likelihood of low scores also depends on the cut-off used, and for the algorithm we selected as the 5th percentile. The algorithm estimates the baserate for the frequency of test scores falling in the abnormally low range that would be expected to occur in a cognitively healthy population.
We used Crawford et al.’s (12) algorithm in our neurologically healthy norming samples to determine how frequently abnormally low scores would occur on the neuropsychological battery of four (Tracking) or six (Comprehensive) intercorrelated tests, separately for French- and English-speaking subsamples. Additionally, we completed this for the four tests given to both the Comprehensive cohort and the Tracking cohort to allow for more direct comparisons across the two. Abnormally low scores were defined as equal to or lower than the 5th percentile because these indicate relatively rare outcomes. For the CLSA Tracking cohort, the algorithm by Crawford and colleagues (12) estimated the percentage of a cognitively healthy population presenting with at least one abnormally low score on the four-test battery to be 15.9% of the English-speaking and 15.7% of the French-speaking subsamples, which in a clinical setting represents a relatively common outcome. In contrast, only 3.7% of the cognitively healthy population based on the English-speaking subsample and 3.8% of the cognitively healthy population based on the French-speaking subsample were estimated to present with at least two abnormally low scores, which represents a sufficiently rare baserate and, thus, likely indicates cognitive impairment.
For the Comprehensive Cohort, the baserate for at least one abnormally low score on the four-test battery was estimated at 15.6% of the English-speaking and 15.9% for the French-speaking subsample, again a relatively common occurrence, whereas at least two abnormally low scores would be expected to occur with a baserate of 3.5% for both the French- and English-speaking subsamples. We determined that two of the four tests presenting as abnormally low was sufficiently rare to indicate cognitive impairment for the four-test CII in the Comprehensive cohort.
For the six-test Comprehensive battery, the estimated percentage of the population presenting with at least one abnormally low score was 22.6% (22.56% in English and 22.60% in French), whereas 5.8% were estimated to present with two or more low scores (5.81% in English and 5.78% in French) and only 1.4% of the population were estimated to present with three or more abnormally low scores. The baserate of low scores for three or more tests is too low because these were estimated to occur in less than 2% of the population), so we chose to use the cut off of two or more tests as indicative of cognitive impairment.[1]
In summary, for both Tracking and Comprehensive cohorts, participants who obtained two or more abnormally low test scores, whether in the four-test or the six-test battery, were classified as overall cognitively impaired (CII=1); otherwise, they were classified as not cognitively impaired (CII=0). The CII was created for all participants in the CLSA who had complete cognitive data (i.e. four test scores in the Tracking cohort and six test scores in the Comprehensive cohort) and for whom normative comparisons were possible (i.e., they had complete data for age, sex, education level, and language of administration).
Concurrent and Discriminant Validity of the CII
To explore the validity of the CII, we used logistic regression analyses to assess whether individual or groups of chronic medical conditions were associated with CII as posited by our a priori hypotheses (see Chronic Conditions for the groupings). In the analyses for groups of chronic conditions, we used sampling weights (version 1.2) (38) that were adjusted for the Canadian population to explore if this impacted the associations with the CII. Sampling weights inflate the observations in the sample to the level of the population to minimize the sampling bias, allowing observations within the sample to be extrapolated to the population of origin.
For the odds ratio (OR) estimates from the logistic regressions, we used the descriptors of magnitude of OR provided by Chen et al. (39), based on a rate of cognitive impairment of 4% in a cognitively healthy group for the 4-test CII: OR = 1.0 to 1.49 as trivial to 1.5 as small, 1.6 to 2.7 as medium, and 2.8 to 5.0 as large (the six-test CII had a higher baserate of cognitive impairment, so OR = 1.5 was classified as small, OR = 2.7 was medium, OR = 4.6 was large). Finally, we calculated the prevalence of cognitive impairment in the Tracking and Comprehensive cohorts with and without sampling weights (38) using the CII based on the same four tests.
[1] Although not described here, a CII based on three or more tests abnormally low as a cut off are available as a derived variable from CLSA for researchers who wish to use the stricter criterion.