Evaluating Sensitivity, Specificity, and Optimal Cutoff Scores of the Montreal Cognitive Assessment Using U.S. National Alzheimer Coordinating Centre Uniform Data Set: A Retrospective Analysis of 16,309 Participants

doi:10.21203/rs.3.rs-4959701/v1

Download PDF

Research Article

Evaluating Sensitivity, Specificity, and Optimal Cutoff Scores of the Montreal Cognitive Assessment Using U.S. National Alzheimer Coordinating Centre Uniform Data Set: A Retrospective Analysis of 16,309 Participants

https://doi.org/10.21203/rs.3.rs-4959701/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Neurodegenerative diseases (NDDs) like Alzheimer's disease and Mild Cognitive Impairment are characterized by progressive cognitive decline, with limited effective treatments available. Several screening tools are available for diagnosing MCI and dementia, including the Rapid Cognitive Screen, the Montreal Cognitive Assessment (MoCA), the Mini-Mental State Examination (MMSE), and the Dementia Rating Scale (DRS).

Objective: This study aims to evaluate the sensitivity and specificity of MoCA to determine its suitability as a diagnostic tool in screening programs.

Methods: The study analyzed data from participants aged 55 and older, recruited from U.S. Alzheimer's Disease Research Centers (ADRCs), using a National Alzheimer Coordinating Center Uniformed Data Set (NACC-UDS). Participants had an initial diagnosis and complete records, focusing on MoCA and then was followed by evaluations to classify MCI, dementia, or no impairment. Receiver operating characteristic (ROC) analysis was used to calculate the area under the curve (AUC).

Results: The study utilized an initial dataset of 188,700 participant records from NACC. After applying inclusion criteria, 16,309 participants were included. The participants had complete diagnostic information, clinician-conducted cognitive assessments, and MoCA scores. The participants were categorized into three groups: 7,624 with no cognitive impairment (NoCI), 4,893 with MCI, and 3,792 with dementia. The study focused on MoCA scores, revealing significant differences among diagnostic groups (P< 0.001). ROC analysis demonstrated the MoCA's strong diagnostic capability, with AUC values significantly above 0.5 (P < 0.001). Sensitivity and specificity were calculated at the recommended cutoff scores of 26 and 21, while the "optimal" cutoff scores were identified as (< 24) for detecting MCI and (< 21) for dementia based on the Youden index in reference to individuals with no cognitive impairment. Although PPV was generally low, the high NPV across comparisons underscores the MoCA's effectiveness in ruling out cognitive impairment.

Conclusion: The study confirms MoCA as an effective tool for detecting dementia, showing 83% sensitivity and 82% specificity at a cutoff value of 21. With a high NPV of 94%, MoCA is particularly reliable for ruling out dementia. However, its ability to identify MCI is limited, with a sensitivity of 52.7%. Despite this, MoCA's 87% specificity helps reduce false positives for MCI, supporting its overall validity as a cognitive assessment tool.

Alzheimer's Disease (AD)

Mild Cognitive Impairment (MCI)

Montreal Cognitive Assessment (MoCA)

Diagnostic Tools

Neurodegenerative diseases (NDDs) are debilitating conditions marked by the gradual degeneration and irreversible damage to nervous tissue. Despite continuous medical research efforts to discover effective treatments for NDDs the results have been disappointing. Conditions like Alzheimer’s disease (AD) and Mild Cognitive Impairment (MCI) remain significant clinical concerns in most older adults [1–3].

Numerous studies have shown that participants with specific NDDs develop distinct deposits of proteins with altered physicochemical properties in their brain. However, NDDs share many underlying processes linked with neuronal dysfunction [4][5]. Because the brain controls numerous body functions, NDDs affect many areas of human functioning, making both simple and complex tasks more challenging. Although some therapies aim to alleviate symptoms, relieve pain, and restore movement and balance, most NDDs progress without remission [2][6].

Based on clinical presentation, anatomical regions, type of the affected cells, conformationally changed proteins involved in the pathogenesis of the disease, and the etiology, NDDs is nosologically classified [4].

Dementia is a medical condition characterized by a significant decline in a person's cognitive abilities, which interferes with their occupational, domestic, or social functioning. A traditional method of categorizing dementia involves dividing the condition into two major categories: neurodegenerative and non-neurodegenerative. An individual can, and often do, develop both neurodegenerative and non-neurodegenerative dementias. Parkinson's disease (PD), dementia with Lewy bodies, and AD are considered the most common neurodegenerative dementias among the elderly [7].

MCI is an intermediate period between the normal cognitive decline associated with healthy aging and the onset of dementia [8]. Various criteria of MCI have been suggested; although they differ, they all include a measurable deficit in at least one cognitive domain without the presence of dementia or affecting daily functioning [9][10]. There are many screening tools available, but the Saint Louis University Mental Status Examination (SLUMS), a 30-point questionnaire that takes less than 10 minutes to administer, is highly validated for diagnosing both MCI and dementia. The Rapid Cognitive Screen and the Montreal Cognitive Assessment (MoCA) also can be used but the Mini-Mental State Examination (MMSE) and the Dementia Rating Scale (DRS) are not recommended [11–14].

AD is a progressive neurodegenerative disorder characterized by significant decline in cognitive abilities [15][16]. It is currently estimated that around 50 million people worldwide have AD [16]. Nowadays, approximately 6.9 million Americans aged 65 and older are living with AD, which is the fifth-leading cause of death among this age group [17]. The neuropathology of AD is associated with the accumulation of β-amyloid plaques and neurofibrillary tangles which triggers many immune responses [16]. The first FDA-approved treatments for AD are those targeting Aβ. The three drugs—aducanumab, lecanemab, and donanemab—are each at different stages in the FDA approval process [18].

AD is complex and often requires expensive and invasive tests that are not widely accessible outside specialized clinical settings [19]. In 2020, the diagnosis of AD relied heavily on PET scans and the analysis of cerebrospinal fluid (CSF) proteins. Both methods showed similar accuracy and indicated that pathological changes begin around 20 years before symptoms appear. As a result, biomarkers were suggested to help with the early detection of AD. The two primary biomarkers for AD are β-amyloid (Aβ) protein and phosphorylated tau protein (p-tau), which can serve as confirmatory and prognostic indicators of the disease [20]. However, using biomarkers is not suitable for screening AD in primary care or community settings [19].

Accurate assessment of general cognitive ability in older adults is crucial to identify those at higher risk of developing dementia and to evaluate the efficacy of clinical trials conducted on individuals living with dementia. The Alzheimer's Disease Assessment Scale (ADAS), which includes cognitive (ADAS-Cog) and non-cognitive (ADAS-NoCog) subscales, is a widely used tool for screening older adults for AD and other forms of dementia. In 2010, it was considered the most widely used screening tool in clinical trials for AD and is still used today, although it requires further investigation [21][22].

The DRS is also a clinically valid psychometric test for detecting AD [23]. The MMSE is not the most accurate or efficient tool for assessing cognitive disorders, but it has established a standard against which all newer tools can be compared [24]. In 2007, a study evaluated the diagnostic value of the Clinical Dementia Rating (CDR) and found that the CDR global score showed strong agreement with the gold standard and demonstrated high diagnostic values [25].

MoCA is a neuropsychological screening tool designed to measure cognitive functioning and decline. It effectively distinguishes between normal cognitive function, MCI, and early-onset dementia and is standardized for individuals aged 55–85 [26][27]. It is scored on a 30-point scale and can be administered in 10 minutes. The MoCA evaluates short-term memory, visuospatial/executive functioning, attention, concentration, working memory, language, and orientation to place and time [26].

A new version of the MoCA, called MoCA-Basic (MoCA-B), was developed to overcome the limitations of the original MoCA for those with low education. Additionally, the newly introduced MoCA Memory Index Score helps clinicians predict which participants with MCI are most likely to advance to dementia [28].

In a Delphi Global Consensus conducted in 2023, the MoCA was ranked the top tool out of 53 for assessing cognitive abilities in clinical practice [29][30]. Studies compared the cognitive assessment capabilities of the MoCA and the MMSE, finding that MoCA is more effective at detecting dementia. In contrast to a 2015 study that considered the MMSE the best and most frequently used screening tool for evaluating cognitive impairment, other studies found that the MoCA is more effective at detecting dementia [27][31].

This study aims to evaluate the sensitivity and specificity of the Montreal Cognitive Assessment (MoCA) to determine its suitability as a diagnostic tool in screening programs.

3.1. Participants

Volunteers recruited from Alzheimer's Disease Research Centers (ADRCs) funded by the National Institute on Aging across the United States participated in the study. The National Alzheimer's Coordinating Center (NACC) maintains a comprehensive database of neuropathologic and clinical data for these participants. To be included in the analysis, participants needed to be 55 years or older and received a diagnosis at their initial visit. The publicly available dataset from NACC was utilized for the study. The inclusion criteria focused on initial visit evaluations and a complete dataset for diagnostic and clinical investigations, with particular emphasis on the MoCA.

3.2. Diagnostic Data

All participants underwent evaluation with the MoCA, conducted by a trained research assistant or psychiatric nurse practitioner independently of the diagnostic process. The MoCA covers several cognitive domains, such as executive function, visuospatial skills, naming, short-term memory, attention and working memory, language, concentration, verbal abstraction, and orientation. The test takes approximately 10 minutes to complete, with a maximum score of 30 indicating perfect cognitive performance. A score below 26 was the original recommended cutoff for identifying cognitive impairment (CI).

3.3. Reference Test

The final diagnosis was made during multidisciplinary team meetings involving an old age psychiatrist, neuropsychologist, and geriatrician. The team used a combination of clinical assessments, cognitive tests, and participant history to evaluate each case. Based on these assessments, they determined if the participant met the criteria for MCI, dementia, or showed no significant cognitive impairment. MCI was characterized by noticeable cognitive decline that did not majorly impact daily activities. In contrast, dementia involved more severe and persistent cognitive decline, affecting daily functioning and necessitating intervention. Participants with no observable impairment were classified as cognitively normal. This collaborative process ensured a comprehensive and accurate diagnosis, reflecting the complex nature of cognitive health.

3.4. Statistical Analysis

Statistical analyses were conducted using Jamovi 2.3.28, Microsoft Excel, and MedCalc. The chi-squared test was used to compare categorical variables such as sex, race, marital status, level of independence, and history of CI and depression over the past two years among the groups with dementia, MCI, or no cognitive impairment (NoCI). The Kruskal-Wallis’s test was employed to compare continuous variables such as age, years of education, GCDR, and MoCA scores, followed by post hoc test.

To evaluate the diagnostic accuracy of the MoCA, receiver operating characteristic (ROC) analysis was used to calculate the area under the curve (AUC). Different ROC curves were generated to distinguish between dementia and non-dementia cases and to identify CI in clinical settings. Positive predictive value (PPV) and negative predictive value (NPV) were calculated based on the "optimal" cutoff scores determined by the Youden J index.

4.1. Study Groups

The initial NACC dataset included a total of 188,700 longitudinal participant data. The selection process was based on inclusion criteria of; initial visit cohort and informed consent for research, which resulted in excluding 137,738 participant data being follow-up visits, and complete data cohort, which resulted in excluding 34,653 participant data due to missing values regarding either MoCA or definitive diagnosis. 16,309 participant data, who had complete records of diagnosis information, cognitive assessment by clinician and MoCA score, were included in the study. According to the diagnosis information available, the included participants were classified according to their diagnosis of cognition to; 7624 no cognitive impairment (NoCI), 4893 MCI and 3792 demented (Fig. 1). This final dataset provided a robust foundation for analyzing the clinical and cognitive characteristics of the study population.

4.2. Demographic Findings

Within the referred participants, there was a significant difference in age (P < .001) between the diagnostic groups, as expected. Gender distribution also showed a significant variance (P < .001), with the proportion of females differing between the groups. Educational years, marital status, and level of independence also showed a significant difference across the groups (P < .001). Regarding the diagnostic data, indicator of parents with cognitive impairment was high with statistically significant difference across diagnostic groups (P < .001). Global Clinical Dementia Rating (GCDR) also showed statistically significant difference between the groups (P < .001) [Table 1].

Table 1

Baseline characteristics of study participants.
	Total ^a N = 16309	NoCI ^b N = 7624	MCI ^c N = 4893	Dem ^d N = 3792	Statistical Sig. (p < 0.001)
Age, years (IQR)	70 (12)	69 (10)	71 (12)	70 (14)	Sig. (d > c > b)
Gender Female, N (%)	9380 (57.5)	4981 (65.33)	2527 (51.65)	1872 (49.4)	Sig.
Race White, N (%) African American, N (%) American Indian, N (%) Native Hawaiian, N (%) Asian, N (%) 5 Multiracial, N (%) Unknown, N (%)	12387 (76) 2555 (15.7) 119 (0.7) 13 (0.1) 474 (2.9) 468 (2.9) 293 (1.8)	5532 (72.6) 1343 (17.62) 79 (1.04) 6 (0.08) 252 (3.31) 266 (3.5) 146 (2)	3561 (72.8) 934 (19.1) 33 (0.8) 3 (0.06) 138 (3) 143 (3) 81 (1.7)	3294 (87) 278 (7.33) 7 (0.2) 4 (0.11) 84 (2.22) 59 (1.6) 66 (1.7)	Sig.
Education, years (IQR)	16 (4)	16 (4)	16 (4)	16 (5)	Sig. (b > c > d)
Marital Status Married, N (%) Widowed, N (%) Divorced, N (%) Separated, N (%) Never Married, N (%) Domestic Partner, N (%) Unknown, N (%)	10759 (66) 1712 (10.5) 2225 (13.6) 213 (1.3) 1025 (6.3) 299 (1.8) 76 (0.5)	4673 (61) 832 (11) 1200 (15.74) 114 (1.5) 615 (8.1) 146 (2) 44 (0.6)	3149 (64) 527 (11) 720 (15) 70 (1) 316 (6.5) 88 (2) 23 (0.5)	2937 (77.5) 353 (9.31) 305 (8.01) 29 (0.8) 94 (2.5) 65 (2) 9 (0.24)	Sig.
Level of Independence Independent, N (%) Requires assistance with complex activities, N (%) Requires assistance with basic activities, N (%) Dependent, N (%) Unknown, N (%)	12570 (77.1) 2696 (16.5) 792 (4.9) 134 (0.8) 117 (0.7)	7511 (98.52) 72 (0.94) 20 (0.26) 0 (0) 21 (0.28)	4063 (83.04) 707 (14.45) 79 (1.61) 4 (0.082) 40 (0.82)	996 (26.27) 1917 (51) 693 (18.3) 130 (3.42) 56 (1.5)	Sig.
Indicator of Mother with Impaired Cognition, N (%)	5492/16302 (33.7)	2771/4617 (60.02)	1600/4893 (32.7)	1121/3792 (29.6)	Sig.
Indicator of Father with Impaired Cognition, N (%)	2818/16302 (17.3)	1391/4617 (30.13)	788/4893 (16.1)	639/3792 (17)	Sig.
Active depression in the last two years, N (%)	3876/16045 (24.16)	1330/7517 (18)	1302/4828 (27)	1224/3700 (33.1)	Sig.
GCDR, (IQR)	0.5 (0.5)	0 (0)	0.5 (0.5)	1 (0.5)	Sig. (d > c > b)
MoCA, (IQR)	24 (7)	27 (3)	23 (5)	16 (9)	Sig. (b > c > d)
Data was represented using Median and Interquartile Range (IQR) or using frequency (percentage). All variables were compared between b, c, and d with either chi-square if categorical or Kruskal Walis if numerical. Abbreviations: MCI, mild cognitive impairment; MoCA, Montreal Cognitive Assessment; NoCI, no cognitive impairment; GCDR, Global Clinical Dementia Rating; Dem, Demented.

4.3. MoCA Outcome

The median MoCA scores differed significantly between groups; the differences in average MoCA scores between the individual referred groups were significant (P < .001).

The results of the ROC analysis, for clinical situations, are shown in (Fig. 2). Table 2 displays the AUCs of these and additional analyses, as well as their sensitivity and specificity at the literature-recommended cutoff scores of 26 and 21. All AUCs were significantly different from 0.5 (no diagnostic accuracy) (P < .001).

Table 2

represents the area under the curve, sensitivity and specificity of literature scores.
Groups			AUC	SE	Cutoff < 26		Cutoff < 21
Groups			AUC	SE	Sens	Spec	Sens	Spec
Dem	vs	NoDem	0.910	0.00266	.9839	.3712	.8291	.8228
Dem	vs	MCI	0.836	0.00433	.9839	.1286	.8291	.6421
NoCI	vs	MCI	0.800	0.00400	.5269	.8714	.9387	.3579
Note. Dem: dementia (n = 3792); NoDem: no dementia (MCI + NoCI; n = 12517); MCI: mild cognitive impairment (n = 4893); NoCI: referred patients no cognitive impairment (n = 7624). AUC: area under the curve. SE: standard error. Sens: sensitivity. Spec: specificity.

For the original suggested cutoff score of 26 to discriminate MCI from NoCI, the sensitivity and specificity are 53% and 87%, respectively (in the original article 90% and 87%) [32]. A cutoff score for diagnosing dementia is still under debate but is often set around 21 [33–35], which in our study results in a sensitivity of 83%. To find the “best” cutoff score for our population, the specificity and sensitivity were calculated for different scores of the MoCA [Table 3].

Table 3

represents MoCA scores with their sensitivity and specificity analyses.
Cutoff Value	Sensitivity %	Specificity %
Cutoff Value	Sensitivity %	MCI + NoCI No Dementia	MCI	NoCI
Dementia
18	65.48	93.33	85.12	65.48
19	71.84	90.85	80.01	71.84
20	77.74	87.27	73.10	77.74
21	82.91	82.28	64.21	82.91
22	88.61	76.18	53.51	88.61
23	92.67	68.61	42.47	92.67
24	95.31	59.33	31.35	95.31
25	97.26	48.77	21.30	97.26
26	98.39	37.12	12.86	98.39
MCI
18	98.60			14.88
19	97.81			19.99
20	96.35			26.90
21	93.87			35.79
22	90.73			46.49
23	85.39			57.53
24	77.28			68.65
25	66.40			78.70
26	52.69			87.14

Note. Dem: dementia (n = 3792); NoDem: no dementia (MCI + NoCI; n = 12517); MCI: mild cognitive impairment (n = 4893); NoCI: referred patients no cognitive impairment (n = 7624).

The “optimum” cutoff scores against NoCI as calculated by the Youden index were less than 24 for detecting MCI, sensitivity 77% (95% CI, 76-78), specificity 69% (95%CI, 67–69), cutoff scores against MCI as calculated by the Youden index were less than 19 for detecting dementia, sensitivity 72% (95% CI, 70–73), specificity 80% (95% CI, 79–81), and cutoff scores against NoCI as calculated by the Youden index were less than 21 for detecting dementia, sensitivity 83% (95% CI, 82–85), specificity 82% (95% CI, 81–83).

The PPV and NPV were calculated [Table 4] for the scores with the highest computed Youden index between each pair of groups. The PPV was low in almost all situations (except cutoff score of 19 between dementia and NoCI) whereas the NPV was high in all situations.

Table 4

represents cutoff scores with the highest Youden index.
Cutoff Value	No Dem		MCI		NoCI
Cutoff Value	PPV	NPC	PPV	NPV	PPV	NPV
	%	%	%	%	%	%
Dem
19	70	91	74	79	94	87
95% CI	(69-71)	(90-92)	(72-75)	(77-80)	(93-95)	(86-88)
21	59	94	64	82	87	91
95% CI	(58-60)	(93-95)	(63-65)	(81-83)	(86-88)	(89-92)
22	52	95	59	85	83	94
95% CI	(51-53)	(94-96)	(58-60)	(84-87)	(81-84)	(93-95)

MCI
24					79	66
95% CI					(78-80)	(65-67)

Note. Dem: dementia (n = 3792); NoDem: no dementia (MCI + NoCI; n = 12517); MCI: mild cognitive impairment (n = 4893); NoCI: referred patients no cognitive impairment (n = 7624). Data marked in bold indicates scores with the highest computed Youden index for each pair of groups.

Starting with brief assessments and progressing to extensive psychological examinations, physicians use different methods and tools to evaluate the cognitive functions. One of the widely used tools in this realm is the MoCA, an evaluation tool designed to assess many aspects of cognitive functions. Accordingly, this study aims to measure the sensitivity and specificity of the MoCA to test its validity as a screening tool.

This study is characterized by notable strengths, such as the large sample size which greatly enhances the statistical power, using an established valid assessment tool (MoCA), which adds credibility to the study, using a retrospective analysis which allows employing of a large data set with no need for additional data collection and focusing on the effect of key variables like depression and marital status on the cognitive health. However, it is important to acknowledge certain weaknesses such as the limitations of retrospective analysis as it is limited by the quality of existing data.

The study findings revealed that there was a slight statistically significant difference between the 3 groups in age (P < 0.001), with dementia participants category exhibiting the highest age. It was found that the prevalence of dementia rises as individuals become older, as demonstrated by a comprehensive review that found that about one in four adults between 90 and 91 years old are affected [36]. According to a German study, dementia rates increase between ages 70 and 94 and reach its maximum level (45%) in 95 years old participants [37].

Female participants constituted a larger portion in each category. This finding is strongly supported by a study done in the UK which reported that women are at higher risk of getting cognitive problem due to different factors such as surgical menopause and hysterectomy [38]. In contrast, an American study revealed that males are more prone to MCI and dementia [39].

The study demonstrated that, over the past two years, active depression levels were elevated in all groups which was statistically significant (p < 0.001), with highest levels observed in individuals with dementia (33%). This result indicates that there is a close connection between them, emphasizing the strong ties between emotions and cognitive health. This finding contrasts with some research as one study reported that cognitive defects may continue even after the active phase of depression has remitted [40]. On the other hand, a Chinese study showed that there is an association between depression and MCI [41]. Furthermore, an observational study revealed that within five years, more than 70% of old participants with pseudodementia developed overt dementia, while only 18% of intact participants did. This clearly illustrates that cognitive impairment in old individuals with depression is a significant predictor of subsequent dementia [42].

According to the current study, the MoCA scale had a sensitivity of 83% for detecting mild dementia (< 21). This clearly indicates a high level of effectiveness in identifying mild dementia. This result aligns with those of two other studies, both indicating that MoCA scale is capable of recognizing mild dementia and, as a result, it is a useful screening tool for it as noted in a meta-analysis done on 67 studies that MoCA has a sensitivity of 93% [43].

Additionally, in PD screening for PDD and PD-MCI, MoCA is better than MMSE due to its better detection of early cognitive changes [44]. MoCA also exhibited high specificity (82%) in distinguishing ones with dementia from those without which enhances accuracy.

Regarding identification of MCI participants, MoCA showed limited ability to detect MCI with sensitivity of 52.7% (< 26). In contrast, a meta-analysis of 35 studies demonstrated that MoCA detects MCI with an effectiveness rate of 83% [45], whereas a Chinese study reported a higher sensitivity of 88% [11]. These findings underscore its ability to detect MCI accurately. What is even more is that MoCA excelled in minimizing false positive MCI results with specificity of 87%. Therefore, MoCA is considered as a preferable and efficient tool for screening of MCI as some studies hinted [46][47].

Although MoCA appears to be an effective screening tool for mild dementia and MCI, its diagnostic ability is slightly limited as it showed PPV of only 59% for mild dementia (< 21). This finding is in agreement with results of other studies which also demonstrates that MoCA exhibits low PPV (50%) for dementia diagnosis as shown in a cohort study done on elderly participants [48]. Another study that assessed 260 participants with dementia and MCI using MoCA scale and MACE (Mini-Addenbrooke’s Cognitive Examination) supports this finding. It showed PPV of 22% for detecting dementia [49].

This could be a result of low prevalence as when prevalence of a condition decreases, the PPV of a screening test also decreases. For conditions with initially high prevalence, even small drops in prevalence can lead to significant drops in PPV until prevalence falls below a certain level so this effect becomes pronounced [50]. This effect is clearly shown in the study that used MoCA and MACE as the prevalence of dementia was only 17% [49].

It is important to note that although the MoCA scale showed limited ability to diagnose dementia, it showed great ability to exclude it (≥ 21) with NPV of 94%. This result is greatly supported by a Singaporean study done on 3780 participants which indicated that the NPV of MoCA for dementia diagnosis is as high as 99% highlighting its effectiveness in ruling out dementia [51]. Thus, MoCA can be confidently used to exclude dementia due to its high NPV and its significant accuracy [52].

One of the key findings of the current study was that a cutoff score of 21 appeared the optimal threshold for detecting dementia with the highest Youden index. Equally, a Chinese cross-section study done on 7445 participants to determine cutoff scores of MoCA-P (Peking Medical Union College Hospital version of the MoCA) demonstrated that the best cutoff value to detect dementia in age group (80–89) years old was also 21 [53]. As expected, using this cutoff value succeeded as shown in a cross-section study done on 710 participants [52]. However, other studies reported different cutoff values. For instance, the optimal cutoff value for dementia detecting was 20 according to a Chinese study [46]. Furthermore, evidence from British research showed 24 as the optimal cutoff value with a Youden index of 0.62 [54]. It is important to note that this variation can happen based on the population, setting and population's features that widely vary.

The study also exhibited that 24 is the optimal cutoff value to detect MCI with maximum Youden index reflecting high effectiveness. It also showed valuable PPV (79%) and NPV (66%) indicating significant ability to diagnose and rule out MCI. Evidence from two further studies support this result indicating that 24 is the optimal cutoff value to detect MCI caused by Alzheimer’s disease [55][56]. Another study that used the Chinese version of MoCA took a different approach by suggesting that a range from 18 to 24 was considered the best to detect MCI [57]. An alternative perspective from some studies indicates that a high cutoff value of 26 could be more effective for detecting MCI. Despite that, many studies are done to discover the effectiveness of the cutoff value of 26, as shown in a 4758 participants-based study which revealed higher misidentification rate for black participants compared to white ones [58]. Thus, these values need further investigation to ensure its accuracy and reliability and therefore researchers remain committed to explore every facet.

This study has several limitations. Its retrospective design depends on the quality and completeness of existing data, potentially introducing selection bias due to the specific inclusion criteria of participants from ADCs. The generalizability of findings is limited, as the study population may not reflect broader or more diverse groups. A large portion of the initial dataset was excluded, affecting the robustness of results. Final diagnoses, based on multidisciplinary team assessments, could introduce variability. Potential confounding factors, such as socioeconomic status and comorbidities, were not fully accounted for. The identified MoCA cutoff scores may not be universally applicable, and the low positive predictive value (PPV) of the MoCA for mild dementia suggests limited diagnostic utility in similar prevalence settings. Exclusion of follow-up data prevents assessment of cognitive changes over time, and cultural and ethnic differences in cognitive performance were not addressed, further limiting the generalizability of the findings.

This study concludes that the MoCA is an effective tool for detecting dementia, with a sensitivity of 83% and a specificity of 82% at cutoff value of 21 and can be used to exclude dementia due to its high NPV (94%). These findings are consistent with the literature, where MoCA is frequently validated as a reliable screening tool for cognitive impairments. And its ability to identify mild cognitive impairment (MCI) is limited, showing a sensitivity of 52.7%. Despite this, MoCA's specificity of 87% minimizes false positives for MCI.

Taking the above results into account, one can conclude that MoCA can be useful in clinical settings to identify individuals requiring further diagnostic evaluation and confirm normal cognitive functioning. Further research is necessary to enhance its sensitivity for MCI and reduce false positives in dementia diagnosis.

Aβ

Amyloid beta

AUC

Area Under the Curve

ADAS

Alzheimer's Disease Assessment Scale

ADAS

Cog-Alzheimer's Disease Assessment Scale-Cognitive subscale

ADAS

NoCog-Alzheimer's Disease Assessment Scale-Non-Cognitive subscale

Alzheimer's Disease

ADRCs

Alzheimer's Disease Research Centers

CDR

Clinical Dementia Rating

Cognitive Impairment

CSF

Cerebrospinal Fluid

DRS

Dementia Rating Scale

FDA

Food and Drug Administration

GCDR

Global Clinical Dementia Rating

MACE

Mini-Addenbrooke’s Cognitive Examination

MCI

Mild Cognitive Impairment

MMSE

Mini-Mental State Examination

MoCA

Montreal Cognitive Assessment

MoCA

B-Montreal Cognitive Assessment-Basic

NACC

National Alzheimer's Coordinating Center

NDDs

neurodegenerative diseases

NoCI

No Cognitive Impairment

NPV

Negative Predictive Value

Parkinson's Disease

PDD

Parkinson’s Disease Dementia

PET

Positron Emission Tomography

PPV

Positive Predictive Value

ROC

Receiver Operating Characteristic

SLUMS

Saint Louis University Mental Status Examination

8.1. Ethical Approval and Consent to Participate: The study utilized de-identified data from the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set (UDS). The original data collection was approved by the Institutional Review Boards (IRBs) of each contributing Alzheimer’s Disease Research Centers (ADRCs). All participants provided informed consent for their data to be used in research. As this study involved secondary analysis of de-identified data, no additional IRB approval was required.

8.2. Ethical Declaration: The research was conducted in accordance with the ethical standards set forth in the Declaration of Helsinki, ensuring adherence to the principles of medical research involving human subjects.

8.3. Consent for Publication: Not Applicable.

8.4. Availability of Data and Material: All data generated or analyzed during this study are included in this published article.

8.5. Competing Interest: The authors declare that they have no competing interests.

8.6. Funding: No funding was received for this research.

8.7. Authors’ Contribution:

Y.A.I. conceptualized the study, designed the methodology, and performed the primary statistical analysis. H.A.A. contributed to the data curation, analysis, and interpretation, as well as the writing of the original draft. S.A.S. assisted in the literature review, data interpretation, and contributed to the manuscript's revision. N.M.A. was involved in the data collection, managed the project, and provided critical feedback on the manuscript. All authors reviewed and approved the final version of the manuscript, with Y.A.I. and H.A.A. sharing equal contributions to this work.

8.8. Acknowledgments:

The NACC database is funded by NIA/NIH Grant U24 AG072122. NACC data are contributed by the NIA-funded ADRCs: P30 AG062429 (PI James Brewer, MD, PhD), P30 AG066468 (PI Oscar Lopez, MD), P30 AG062421 (PI Bradley Hyman, MD, PhD), P30 AG066509 (PI Thomas Grabowski, MD), P30 AG066514 (PI Mary Sano, PhD), P30 AG066530 (PI Helena Chui, MD), P30 AG066507 (PI Marilyn Albert, PhD), P30 AG066444 (PI John Morris, MD), P30 AG066518 (PI Jeffrey Kaye, MD), P30 AG066512 (PI Thomas Wisniewski, MD), P30 AG066462 (PI Scott Small, MD), P30 AG072979 (PI David Wolk, MD), P30 AG072972 (PI Charles DeCarli, MD), P30 AG072976 (PI Andrew Saykin, PsyD), P30 AG072975 (PI David Bennett, MD), P30 AG072978 (PI Neil Kowall, MD), P30 AG072977 (PI Robert Vassar, PhD), P30 AG066519 (PI Frank LaFerla, PhD), P30 AG062677 (PI Ronald Petersen, MD, PhD), P30 AG079280 (PI Eric Reiman, MD), P30 AG062422 (PI Gil Rabinovici, MD), P30 AG066511 (PI Allan Levey, MD, PhD), P30 AG072946 (PI Linda Van Eldik, PhD), P30 AG062715 (PI Sanjay Asthana, MD, FRCP), P30 AG072973 (PI Russell Swerdlow, MD), P30 AG066506 (PI Todd Golde, MD, PhD), P30 AG066508 (PI Stephen Strittmatter, MD, PhD), P30 AG066515 (PI Victor Henderson, MD, MS), P30 AG072947 (PI Suzanne Craft, PhD), P30 AG072931 (PI Henry Paulson, MD, PhD), P30 AG066546 (PI Sudha Seshadri, MD), P20 AG068024 (PI Erik Roberson, MD, PhD), P20 AG068053 (PI Justin Miller, PhD), P20 AG068077 (PI Gary Rosenberg, MD), P20 AG068082 (PI Angela Jefferson, PhD), P30 AG072958 (PI Heather Whitson, MD), P30 AG072959 (PI James Leverenz, MD).

Choonara YE, Pillay V, Du Toit LC, Modi G, Naidoo D, Ndesendo VM, Sibambo SR. Trends in the molecular pathogenesis and clinical therapeutics of common neurodegenerative disorders. Int J Mol Sci. 2009;10:2510–57. 10.3390/ijms10062510.
Lamptey RNL, Chaulagain B, Trivedi R, Gothwal A, Layek B, Singh J. A Review of the Common Neurodegenerative Disorders: Current Therapeutic Approaches and the Potential Role of Nanotherapeutics. Int J Mol Sci. 2022;23(3):1851. 10.3390/ijms23031851. Published 2022 Feb 6.
Martin JB. Molecular basis of the neurodegenerative disorders. N Engl J Med. 1999;340:1970–80. 10.1056/NEJM199906243402507.
Kovacs GG. Current concepts of neurodegenerative diseases. EMJ Neurol. 2014;1(1):10–1.
Dugger BN, Dickson DW. Pathology of Neurodegenerative Diseases. Cold Spring Harb Perspect Biol. 2017;9(7):a028035. Published 2017 Jul 5. 10.1101/cshperspect.a028035
Gitler AD, Dhillon P, Shorter J. Neurodegenerative disease: models, mechanisms, and a new hope. Dis Model Mech. 2017;10(5):499–502. 10.1242/dmm.030205.
Gale SA, Acar D, Daffner KR, Dementia. Am J Med. 2018;131(10):1161–9. 10.1016/j.amjmed.2018.01.022.
Anderson ND. State of the science on mild cognitive impairment (MCI). CNS Spectr. 2019;24:78–87. 10.1017/S1092852918001347.
Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999;56(3):303–8.
Delbari A, Tabatabaei FS, Ghasemi H, Azimi A, Bidkhori M, Saatchi M, et al. Prevalence and associated factors of mild cognitive impairment among middle-aged and older adults: Results of the first phase of Ardakan Cohort Study on Aging. Health Sci Rep. 2024;7(1):e1827.
Sanford AM. Mild Cognitive Impairment. Clin Geriatr Med. 2017;33(3):325–37. 10.1016/j.cger.2017.02.005.
Espino DV, Lichtenstein MJ, Palmer RF, Hazuda HP. Ethnic differences in Mini-Mental State Examination (MMSE) scores: where you live makes a difference. J Am Geriatr Soc. 2001;49(5):538–48.
Matallana D, de Santacruz C, Cano C, et al. The relationship between education level and Mini-Mental State Examination domains among older Mexican Americans. J Geriatr Psychiatry Neurol. 2011;24(1):9–18.
Jongsiriyanyong S, Limpawattana P. Mild Cognitive Impairment in Clinical Practice: A Review Article. Am J Alzheimers Dis Other Demen. 2018;33(8):500–7. 10.1177/1533317518791401.
Contador I, Bermejo-Pareja F, Infante MT, Bajo-Guerra S, Lora-Pablos D, Rami L et al. Charting Alzheimer’s disease and dementia: epidemiological insights, risk factors and prevention pathways. J Clin Med. 2024;13(14).
Chaudhary RK, Raj K, Gupta V, Kumar A, Verma P, Aggarwal A. Alzheimer’s disease: epidemiology, neuropathology, and neurochemistry. In: Kumar A, Kato N, editors. Computational and experimental studies in Alzheimer’s disease. Boca Raton (FL): CRC; 2024. pp. 1–14.
2024 Alzheimer’s disease facts and figures. Alzheimers Dement. 2024;20(5):3708–3821. 10.1002/alz.13809
Beveridge J, Kaniecki E, Naidu A, Silverglate BD, Grossberg G. How promising are the latest monoclonal antibodies targeting amyloid-β for the treatment of early Alzheimer’s disease? Expert Opin Emerg Drugs. 2024;29(1):35–43. 10.1080/14728214.2024.2304059.
Cheung CY, Ran AR, Wang S, et al. A deep learning model for detection of Alzheimer's disease based on retinal photographs: a retrospective, multicentre case-control study. Lancet Digit Health. 2022;4(11):e806–15. 10.1016/S2589-7500(22)00169-8.
Khan S, Barve KH, Kumar MS. Recent Advancements in Pathogenesis, Diagnostics and Treatment of Alzheimer’s Disease. Curr Neuropharmacol. 2020;18(11):1106–25. 10.2174/1570159X18666200528142429.
Tsatali M, Moraitou D, Gialaouzidis M, et al. Discriminant Potential of the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog) in Greek Older Adults with Subjective Cognitive Decline and Mild Cognitive Impairment. J Alzheimers Dis Rep. 2024;8(1):543–54. 10.3233/ADR-230151. Published 2024 Mar 19.
Cano SJ, Posner HB, Moline ML, Hurt SW, Swartz J, Hsu T, Hobart JC. The ADAS-cog in Alzheimer’s disease clinical trials: Psychometric evaluation of the sum and its parts. J Neurol Neurosurg Psychiatry. 2010;81:1363–8.
Monsch AU, Bondi MW, Salmon DP, Butters N, Thal LJ, Hansen LA, et al. Clinical validity of the Mattis Dementia Rating Scale in detecting dementia of the Alzheimer type: a double cross-validation and application to a community-dwelling sample. Arch Neurol. 1995;52(9):899–904.
Mitchell AJ. The Mini-Mental State Examination (MMSE): an update on its diagnostic validity for cognitive disorders. In: Cognitive Screening Instruments: A Practical Approach. 2013. pp. 15–46.
Chaves MLF, Camozzato AL, Godinho C, Kochhann R, Schuh A, De Almeida VL, et al. Validity of the clinical dementia rating scale for the detection and staging of dementia in Brazilian patients. Alzheimer Dis Assoc Disord. 2007;21(3):210–7.
Geller R, Slicer K. Montreal Cognitive Assessment (MoCA). Clinical Integration of Neuropsychological Test Results. CRC; 2024. pp. 191–204.
Pinto TCC, Machado L, Bulgacov TM, et al. Is the Montreal Cognitive Assessment (MoCA) screening superior to the Mini-Mental State Examination (MMSE) in the detection of mild cognitive impairment (MCI) and Alzheimer’s Disease (AD) in the elderly? Int Psychogeriatr. 2019;31(4):491–504. 10.1017/S1041610218001370.
Julayanont P, Nasreddine ZS. Montreal Cognitive Assessment (MoCA): concept and clinical review. In: Cognitive screening instruments: A practical approach. 2017. P. 139–195.
Ellison TS, Cappa SF, Garrett D, et al. Outcome measures for Alzheimer’s disease: a global inter-societal Delphi consensus. Alzheimers Dement. 2023;19(6):2707–29. 10.1002/alz.12945.
Klil-Drori S, Bodenstein KC, Sun S, et al. Montreal Cognitive Assessment (MoCA) XpressO: Validation of a digital self-administered cognitive prescreening tool. J Am Geriatr Soc Published online April. 2024;1. 10.1111/jgs.18902.
Arevalo-Rodriguez I, Smailagic N, Roqué I, Figuls M, Ciapponi A, Sanchez-Perez E, Giannakou A et al. Mini-Mental State Examination (MMSE) for the detection of Alzheimer’s disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev. 2015;2015(3):CD010783. Published 2015 Mar 5. 10.1002/14651858.CD010783.pub2
Nasreddine ZS, Phillips NA, Bédirian V et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment [published correction appears in J Am Geriatr Soc. 2019;67(9):1991. doi: 10.1111/jgs.15925]. J Am Geriatr Soc. 2005;53(4):695–699. 10.1111/j.1532-5415.2005.53221.x
17, Thissen AJAM, van Bergen F, de Jonghe JFM, Kessels RPC. Dautzenberg, PLJ (n.d.) Bruikbaarheid En Validiteit van de Nederlandse Versie van de Montreal Cognitive Assessment (MoCA-D) Bij Het Diagnosticeren van Mild Cognitive Impairment Applicability and Validity of the Dutch Version of the Montreal Cognitive Assessment (MoCA‐D) in Diagnosing MCI.
18, Davis DHJ, Creavin ST, Yip JLY, Noel-Storr AH, Brayne C, Cullum S. Montreal Cognitive Assessment for the diagnosis of Alzheimer's disease and other dementias. Cochrane Database Syst Rev. 2015;(10):CD010775 Published 2015 Oct 29. 10.1002/14651858.CD010775.pub2
Waldron-Perrine B, Axelrod BN. Determining an appropriate cutting score for indication of impairment on the Montreal Cognitive Assessment. Int J Geriatr Psychiatry. 2012;27(11):11891194. http://ovidsp.ovid.com/ovidweb.cgi?&T=JS&PAGE=reference&MODE=ovidclassic&CSC=y&NEWS=n&D=mesx,prem,mesz,medp&SEARCH=22228412.ui.10.1002/gps.3768
Yang Z, Slavin MJ, Sachdev PS. Dementia in the oldest old. Nat Rev Neurol. 2013;9(7):382–93. 10.1038/nrneurol.2013.105.
Wernicke TF, Reischies FM. Prevalence of dementia in old age: clinical diagnoses in subjects aged 95 years and older. Neurology. 1994;44(2):250–3. 10.1212/wnl.44.2.250.
Lindseth LRS, de Lange AG, van der Meer D, et al. Associations between reproductive history, hormone use, APOE ε4 genotype and cognition in middle- to older-aged women from the UK Biobank. Front Aging Neurosci. 2023;14:1014605. 10.3389/fnagi.2022.1014605. Published 2023 Jan 19.
Wang G, Li W. Sex as a Risk Factor for Developing Cognitive Impairments in National Alzheimer's Coordinating Center Participants. J Alzheimers Dis Rep. 2021;5(1):1–6. 10.3233/ADR-200275. Published 2021 Jan 11.
Weisenbach SL, Boore LA, Kales HC. Depression and cognitive impairment in older adults. Curr Psychiatry Rep. 2012;14(4):280–8. 10.1007/s11920-012-0278-7.
Song D, Li PWC, Yu DSF. The association between depression and mild cognitive impairment: A cross-sectional study. Int J Geriatr Psychiatry. 2018;33(4):672–4. 10.1002/gps.4798.
Perini G, Cotta Ramusino M, Sinforiani E, Bernini S, Petrachi R, Costa A. Cognitive impairment in depression: recent advances and novel treatments. Neuropsychiatr Dis Treat. 2019;15:1249–58. 10.2147/NDT.S199746. Published 2019 May 10.
Wang X, Li F, Gao Q, et al. Evaluation of the Accuracy of Cognitive Screening Tests in Detecting Dementia Associated with Alzheimer's Disease: A Hierarchical Bayesian Latent Class Meta-Analysis. J Alzheimers Dis. 2022;87(1):285–304. 10.3233/JAD-215394.
Kasten M, Bruggemann N, Schmidt A, Klein C. Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology. 2010;75(5):478–9. 10.1212/WNL.0b013e3181e7948a.
Kawada T. Montreal Cognitive Assessment (MoCA) and its memory tasks for detecting mild cognitive impairment. Neurol Sci. 2019;40(3):633. 10.1007/s10072-018-3616-7.
Tsai JC, Chen CW, Chu H, et al. Comparing the Sensitivity, Specificity, and Predictive Values of the Montreal Cognitive Assessment and Mini-Mental State Examination When Screening People for Mild Cognitive Impairment and Dementia in Chinese Population. Arch Psychiatr Nurs. 2016;30(4):486–91. 10.1016/j.apnu.2016.01.015.
Abd Razak MA, Ahmad NA, Chan YY, et al. Validity of screening tools for dementia and mild cognitive impairment among the elderly in primary health care: a systematic review. Public Health. 2019;169:84–92. 10.1016/j.puhe.2019.01.001.
Dautzenberg G, Lijmer J, Beekman A. Clinical value of the Montreal Cognitive Assessment (MoCA) in patients suspected of cognitive impairment in old age psychiatry. Using the MoCA for triaging to a memory clinic. Cogn Neuropsychiatry. 2021;26(1):1–17. 10.1080/13546805.2020.1850434.
Larner AJ. MACE versus MoCA: equivalence or superiority? Pragmatic diagnostic test accuracy study. Int Psychogeriatr. 2017;29(6):931–7. 10.1017/S1041610216002210.
Balayla J. Prevalence threshold (ϕe) and the geometry of screening curves. PLoS ONE. 2020;15(10):e0240215. 10.1371/journal.pone.0240215. Published 2020 Oct 7.
Pang T, Xia B, Zhao X, et al. Cost-benefit and discriminant validity of a stepwise dementia case-finding approach in an Asian older adult community. Gen Psychiatr. 2023;36(5):e101049. 10.1136/gpsych-2023-101049. Published 2023 Oct 30.
Dautzenberg G, Lijmer J, Beekman A. Diagnostic accuracy of the Montreal Cognitive Assessment (MoCA) for cognitive screening in old age psychiatry: Determining cutoff scores in clinical practice. Avoiding spectrum bias caused by healthy controls. Int J Geriatr Psychiatry. 2020;35(3):261–9. 10.1002/gps.5227.
Tan JP, Li N, Gao J, et al. Optimal cutoff scores for dementia and mild cognitive impairment of the Montreal Cognitive Assessment among elderly and oldest-old Chinese population. J Alzheimers Dis. 2015;43(4):1403–12. 10.3233/JAD-141278.
Larner AJ. Optimising the cutoffs of cognitive screening instruments in pragmatic diagnostic accuracy studies: maximising accuracy or the Youden index? Dement Geriatr Cogn Disord. 2015;39(3–4):167–75. 10.1159/000369883.
Pugh EA, Kemp EC, van Dyck CH, Mecca AP, Sharp ES. Alzheimer's Disease Neuroimaging Initiative. Effects of Normative Adjustments to the Montreal Cognitive Assessment. Am J Geriatr Psychiatry. 2018;26(12):1258–67. 10.1016/j.jagp.2018.09.009.
Cecato JF, Martinelli JE, Izbicki R, Yassuda MS, Aprahamian I. A subtest analysis of The Montreal Cognitive Assessment (MoCA): which subtests can best discriminate between healthy controls, mild cognitive impairment and Alzheimer's disease? Int Psychogeriatr. 2017;29(4):701. 10.1017/S104161021600212X.
Yang C, Wang L, Hu H, Dong X, Wang Y, Yang F. Montreal Cognitive Assessment: Seeking a Single Cutoff Score May Not Be Optimal. Evid Based Complement Alternat Med. 2021;2021:9984419. 10.1155/2021/9984419. Published 2021 Sep 25.
Ratcliffe LN, McDonald T, Robinson B, Sass JR, Loring DW, Hewitt KC. Classification statistics of the Montreal Cognitive Assessment (MoCA): Are we interpreting the MoCA correctly? Clin Neuropsychol. 2023;37(3):562–76. 10.1080/13854046.2022.2086487.

No competing interests reported.

Download PDF

Editorial decision: Revision requested
28 Aug, 2024
Editor assigned by journal
22 Aug, 2024
Submission checks completed at journal
22 Aug, 2024
First submitted to journal
22 Aug, 2024

You are reading this latest preprint version

Evaluating Sensitivity, Specificity, and Optimal Cutoff Scores of the Montreal Cognitive Assessment Using U.S. National Alzheimer Coordinating Centre Uniform Data Set: A Retrospective Analysis of 16,309 Participants

Status:

Version 1

Abstract

Figures

INTRODUCTION

METHODS

3.1. Participants

3.2. Diagnostic Data

3.4. Statistical Analysis

RESULTS

4.1. Study Groups

4.2. Demographic Findings

4.3. MoCA Outcome

DISCUSSION

CONCLUSION

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1