You are currently viewing a beta version of our website. If you spot anything unusual, kindly let us know.
Preprint
Article
Cognitive Screening in Older People Using Free-Cog and Mini-Addenbrooke’s Cognitive Examination (MACE)
Altmetrics
Downloads
268
Views
69
Comments
0
This version is not peer-reviewed
Abstract
Ageing is the strongest known risk factor for many of the neurodegenerative diseases causing cognitive decline and dementia. Identification of cognitive impairment may be a prelude to appropriate treatment, hopefully disease-modifying. Use of cognitive screening instruments may be an equitable way to identify cognitive impairment. This study examined the use of two such instruments, Free-Cog and Mini-Addenbrooke’s Cognitive Examination (MACE), in patient cohorts referred to a dedicated cognitive disorders clinic based at a tertiary neurosciences centre. Results showed that: (1) specificity and positive predictive value increased with patient age for both tests with some loss of sensitivity and negative predictive value. (2) In the oldest age groups (≥75 and ≥70 years respectively) where specificity was at maximum, a positive test result (i.e. below the specified test cut-off) rules in the diagnosis of cognitive impairment. (3) Values of an “Efficiency Index” for each test indicated qualitatively a moderate change in the probability of correct diagnosis and quantitatively an approximately 15-25% increase in the probability of correct diagnosis. These findings show that both Free-Cog and MACE may be used with confidence for the identification of cognitive impairment and dementia in older patient cohorts. These findings may have implications for public health policies directed to case-finding in clinical practice as opposed to population-based screening.
Keywords:
Subject: Medicine and Pharmacology - Neuroscience and Neurology
1. Introduction
Ageing is the strongest known risk factor for many of the neurodegenerative diseases of the brain which give rise to cognitive decline, sometimes sufficient to meet diagnostic criteria for dementia. The development of age-related cognitive impairment and dementia is recognised to constitute one of the most profound global challenges to public health in the 21st century, in part as a consequence of the ageing of populations around the world.
Nevertheless, screening for cognitive impairment and dementia in older adults remains a contentious issue. Such screening is predicated on the idea that identification of cognitive impairment will be the prelude to early interventions to improve outcomes. Whilst various screening instruments have been shown capable of detecting cognitive impairment in older adults, and multiple practice guidelines and consensus statements on the detection of mild cognitive impairment (MCI) have been published [1], evidence for benefit in terms of improved patient or caregiver outcomes is currently lacking [2,3]. Hence the criteria set down in the World Health Organisation (WHO) guidelines for screening [4] are not currently fulfilled for dementia and cognitive impairment.
However, therapeutic interventions for early cognitive impairment may now possibly be within sight [5] (although their delivery in terms of costs and logistics remains daunting [6]). If such should prove to be the case, then screening for cognitive impairment may become an accepted and appropriate management strategy, consistent with WHO screening criteria. Investigations to detect neurodegenerative pathology leading to cognitive impairment based around disease biomarkers (e.g. neuroimaging, CSF, and ultimately blood [7]) hold out great promise in this area. However, since these investigations may not be readily scaled up to universal availability, the use of cognitive screening instruments is likely to remain an integral part of clinical assessment in the immediate future. Equity of global health care provision may be ensured by the relative ease of accessibility of such screening tests (unlike biomarker neuroimaging, CSF, and blood tests).
Multiple cognitive screening instruments are available and many have been specifically assessed for the detection of cognitive impairment in older adult populations. For example, the test accuracy of the Mini-Addenbrooke’s Cognitive Examination (MACE) [8] for the identification of dementia and MCI in a cohort of older patients has previously been reported in a neurology-led dedicated cognitive disorders clinic [9]. MACE test accuracy has also been compared to another short (and copyright-free) cognitive screening instrument, Free-Cog [10,11], in both standard and abridged forms suitable respectively for face-to-face [12] and remote (telephone, video) [10,13] testing.
The aims of the current study were threefold: firstly, to undertake the same analysis of age-related test accuracy for Free-Cog as previously reported for MACE [9]; secondly, to extend the age-related test accuracy analysis for MACE to produce a more fine-grained examination of the performance of this instrument with patient age; and thirdly, in both cases to extend the range of test outcome measures to include the “Efficiency Index,” a recently described unitary test outcome measure which permits ready communication of the risk of whether a test outcome results in correct diagnosis rather than misdiagnosis [14].
2. Materials and Methods
2.1. Participants
Datasets from two pragmatic prospective screening test accuracy studies examining MACE (June 2014-December 2018 inclusive) [15] and Free-Cog (November 2017-October 2018 inclusive) [12] respectively were re-interrogated. Both studies recruited consecutive new patients referred to a neurology-led dedicated cognitive function clinic based in a regional neuroscience centre. The clinic does not operate any age-related exclusion criteria.
2.2. Procedures
Patient assessment comprised semi-structured history enquiring about cognitive symptoms and functional performance, with collateral history from a reliable and knowledgeable informant where possible. All patients underwent neuroradiological examination with brain computed tomography (CT), with interval magnetic resonance (MR) brain imaging in some cases. Formal neuropsychological assessment was pursued in some cases where there was remaining diagnostic uncertainty.
Administration of Free-Cog or MACE occurred on the same day as, but separate from, the cross-sectional assessment. Free-Cog is a hybrid screening instrument of cognition and executive function (item content shown in Table 1, left hand column), purposely designed to have a conversational quality in order to put patients at their ease and hence maximise performance [10]. Free-Cog was used in its standard, unitary formulation, rather than as separate cognitive and executive function tests formulated either sequentially using Boolean logical operators or as a stepwise decision tree, as examined elsewhere [16]. MACE (item content shown in Table 1, right hand column) is a brief cognitive screening instrument derived from the longer Addenbrooke’s Cognitive Examination in its third iteration, ACE-III [17]. Both Free-Cog and MACE are scored out of 30 points, with higher scores better.
Patients were diagnosed with dementia or MCI according to standard diagnostic criteria (DSM-IV, Petersen, respectively), as previously used in this clinic [12,15]. Criterion diagnosis of dementia, MCI, or subjective memory complaint (SMC), was by the judgment of an experienced clinician based on the specified diagnostic criteria but did not use either Free-Cog or MACE score in order to avoid review bias. STARDdem guidelines for reporting diagnostic test accuracy studies in dementia were observed [18].
2.3. Analyses
For Free-Cog, a hybrid screening instrument of cognition and executive function, data from patient cohorts aged ≥65 and ≥75 years, as well as the whole cohort, were examined (as in a previous study of MACE [9]). A cut-off of ≤22/30, defined from the study data by maximal Youden index [12], was used (a test cut-off was not specified in the index study of Free-Cog [10]).
For MACE (item content shown in Table 1, right hand column), the original analysis [9] was extended such that data were examined in 5-year cohorts between the ages of 50 and 79, as well as in catch-all groups <50 and ≥80 years. A MACE cut-off of ≤20/30, defined from the study data by maximal Youden index [15], was used, as opposed to the two MACE cut-offs (≤25/30, more sensitive, and ≤21/30, more specific) defined in the index study [8] and used in the previous age-related analysis of MACE [8].
Test accuracy outcomes at the chosen cut-offs were expressed in terms of sensitivity (Sens) and specificity (Spec), positive and negative predictive values (PPV, NPV), and overall test accuracy (or correct classification accuracy, Acc).
Sens and Spec results were interpreted in terms of the heuristic “SnNOut” and “SpPin” rules: with a highly sensitive test, a negative result rules out the diagnosis, whereas with a highly specific test, a positive result rules in the diagnosis [19].
The Efficiency Index (EI) [14] of each test in each age cohort was calculated on the basis of the values for Acc and its complement, Inaccuracy (Inacc), defined as:
EI = Acc/(1 – Acc)
= Acc/Inacc
This metric has a range 0-∞, where higher scores are better, with an inflection point at 1 (EI<1 favours misdiagnosis, EI>1 favours correct diagnosis, hence EI values >>1 are desired). EI values were classified qualitatively and semi-quantitatively as for likelihood ratios, as previously shown [14].
3. Results
3.1. Free-Cog
For the Free-Cog cohort (N = 141; demographics shown in Table 2), the prevalence of cognitive impairment increased with cohort age, as was anticipated.
Free-Cog test outcomes (Table 3) showed that test Sens and NPV declined with cohort age but that Spec and PPV increased, both reaching maximal values in the ≥75 years cohort. Following the “SpPin” rule [19], as a highly specific test, a positive Free-Cog test result (i.e. ≤22/30) likely rules in the diagnosis of cognitive impairment.
The values for both Acc and EI for Free-Cog were higher in the older patient cohorts compared to the whole cohort (Table 3). The EI values in the different cohorts examined ranged between ≈2-4, which indicates a qualitatively moderate change in the probability of correct diagnosis and quantitatively an approximately 15-25% increase in the probability of correct diagnosis [14].
3.2. MACE
For the MACE cohort (N = 755; demographics shown in Table 4), the prevalence of cognitive impairment increased with cohort age, as was anticipated.
MACE test outcomes (Table 5) showed that test Spec and PPV increased with cohort age, both reaching maximal values in the patient cohorts aged ≥70 years. NPV declined whereas Sens remained relatively stable across the age cohorts. Following the “SpPin” rule [19], as a highly specific test, a positive MACE result (i.e. ≤20/30) likely rules in the diagnosis of cognitive impairment.
The values for both Acc and EI for MACE remained relatively stable across the age cohorts (Table 5). The EI values in the different cohorts examined ranged between ≈2-4, which indicates a qualitatively moderate change in the probability of correct diagnosis and quantitatively an approximately 15-25% increase in the probability of correct diagnosis [14].
4. Discussion
Reanalysis of the dataset from the Free-Cog study [12] showed improved values for Spec and PPV with increasing patient age. Better Spec indicated fewer false positive results and hence, in accordance with the heuristic “SpPin” rule [19], a positive result (Free-Cog ≤22/30) likely rules in the diagnosis of cognitive impairment. This is the combination of outcomes that would be desirable for a screening test for cognitive impairment in older patient cohorts. This outcome might possibly reflect the hybrid nature of Free-Cog as a test of both cognitive and executive functions, since other studies have suggested such a combination may improve detection of neurocognitive disorder [20], although fragmenting the Free-Cog into separate tests of cognitive and executive functions did not suggest better overall performance than the unitary Free-Cog [16]. Other analyses of Free-Cog have previously shown that it fares well in comparison to other short cognitive screening instruments in so-called “metrics of limitation” such as misclassification rate and net harm/net benefit ratio [21].
Extending the reanalysis of the dataset from the MACE study [15] showed improved values for PPV with increasing patient age, as in the original study [9], but also improved Spec which was not apparent in the original study, perhaps related to the broader age cohorts used therein. In the current analysis, Spec was at maximum in patient cohorts aged ≥70 years, hence in accordance with the heuristic “SpPin” rule [19] a positive result (MACE ≤20/30) rules in the diagnosis of cognitive impairment.
These findings – improved values of both Spec and PPV for both Free-Cog and MACE with increasing patient age, reaching maximal values in the patient cohorts aged ≥75 years and ≥70 years respectively – are of particular interest in view of the characterisation of both Free-Cog [12] and MACE [15,22] as high sensitivity low specificity screeners on the basis of their overall test performance. Of course, the potential shortcomings of Sens and Spec and predictive value metrics are well-recognised, prompting the search for other outcome measures by which to judge test utility.
In this context the calculation of a novel, recently described test metric, the Efficiency Index [14], a measure of the likelihood of correct diagnosis versus misdiagnosis, proved easy to use. For Free-Cog, it suggested an improved EI in older patient cohorts compared to the whole cohort, and for MACE it suggested relative stability of EI across the examined age cohorts. Qualitatively the EI values suggested a moderate change in the probability of correct diagnosis of disease and semi-quantitatively an approximately 15-25% increase in the probability of correct diagnosis [14].
Other, more stringent, formulations of EI, taking into account disease prevalence and/or test cut-off, are available [23] and might be examined in future studies of screening instruments. Other priorities for future study include further examination of whether screening of cognitive domains alone or in combination with executive functions offers the optimal strategy for case-finding. The canonical definitions of dementia (DSM-IV) or major neurocognitive disorder (DSM-5) encompass deficits in both cognition and function, unlike MCI or minor neurocognitive disorder. The current analyses did not examine head-to-head test data, but in a previous study Free-Cog was more specific than MACE for diagnosis of both dementia and MCI [12].
How might the findings of this study fit more broadly into the existing (and possibly future) public health policies for dementia screening or identification? Currently there is no indication for population-based screening for dementia. The UK National Screening Committee on Screening for Dementia has been explicit in its statements recommending against screening healthy individuals aged 65 years and over (2015 and 2019; at time of writing an update of this review is anticipated). Likewise, the US Preventive Services Task Force has found no benefit in terms of improved outcomes for dementia screening [2]. However, targeted screening of populations known to be at higher risk of dementia might have a role, such as those individuals with subjective memory complaints, or those referred to memory clinics, or with conditions which predispose to dementia (such as diabetes mellitus), as these groups are known to have a higher prevalence of dementia and cognitive impairment than the general population. Another suggested dementia identification strategy is case-finding in clinical practice, which may be a viable alternative to screening [24]. One of the proposed stages of this approach, which in some ways reflects the idiom of clinical practice, involves cognitive assessment, but the authors found evidence for this stage to be lacking [24]. The current findings might contribute evidence in support of such an approach using either of these cognitive screening instruments. Certainly both Free-Cog and MACE were found to be acceptable to patients in the respective test accuracy studies [12,15]. This approach to case-finding might also be extended to remote assessment methods by using appropriate modifications of these instruments (Tele-Free-Cog, Tele-MACE) [10,13].
5. Limitations
The limitations of this study are those familiar in any clinic-based study. The selected study population had a relatively high prevalence of dementia and MCI compared to patient cohorts in community-based (e.g. primary care or population-based) cohorts. The Free-Cog study cohort was quite small (<150), meaning that confidence intervals for test metrics were wide, particularly in the age ≥75 years cohort. This was reflective of the relatively young age of the patients seen in neurology-based cognitive clinics (median age in Free-Cog study = 62 years [12]; in MACE study = 60 years [15]), unlike the case mix in old age psychiatry memory clinics. Use of cross-sectional clinical diagnoses as reference standard is idiomatic of day-to-day practice but potentially liable to error without delayed verification (e.g. no neuropathological data were available). All these factors might limit the generalizability of the current study findings.
6. Conclusions
Both Free-Cog and MACE may be of value in ruling in a diagnosis of cognitive impairment in older patient cohorts presenting with memory complaints. They might therefore be instrumental in implementing any policy of cognitive screening or case-finding in older adult populations.
Funding
The author received no specific funding for this work.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Committee on Human Research of the Walton Centre for Neurology and Neurosurgery (Approval: N 310)
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request.
Conflicts of Interest
The author declares no conflict of interest.
References
- Chen, Y.-X.; Liang, N.; Li, X.-L.; Yang, S.-H.; Wang, Y.-P.; Shi, N.-N. Diagnosis and treatment for mild cognitive impairment: a systematic review of clinical practice guidelines and consensus statements. Front. Neurol. 2021, 12, 719849. [Google Scholar] [CrossRef] [PubMed]
- US Preventive Services Task Force; Owens, D.K.; Davidson, K.W.; Krist, A.H.; Barry, M.J.; Cabana, M.; Caughey, A.B.; Doubeni, C.A.; Epling Jr, J.W.; Kubik, M.; Landefeld, C.S.; Mangione, C.A.; Pbert, L.; Silverstein, M.; Simon, M.A.; Tseng, C.-W.; Wong, J.B. Screening for cognitive impairment in older adults: US Preventive Services Task Force Recommendation Statement. JAMA 2020, 323, 757–763. [Google Scholar]
- Patnode, C.D.; Perdue, L.A.; Rossom, R.C.; Rushkin, M.C.; Redmond, N.; Thomas, R.G.; Lin, J.S. Screening for cognitive impairment in older adults: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA 2020, 323, 764–785. [Google Scholar] [CrossRef] [PubMed]
- Wilson, J.M.G.; Jungner, G. Principles and practice of screening for disease. Public health paper No. 34; World Health Organisation: Geneva, 1968. [Google Scholar]
- van Dyck, C.H.; Swanson, C.J.; Aisen, P.; Bateman, R.J.; Chen, C.; Gee, M.; Kanekiyo, M.; Li, D.; Reyderman, L.; Cohen, S.; Froelich, L.; Katayama, S.; Sabbagh, M.; Vellas, B.; Watson, D.; Dhadda, S.; Irizarry, M.; Kramer, L.D.; Iwatsubo, T. Lecanemab in early Alzheimer’s disease. N. Engl. J. Med. 2023, 388, 9–21. [Google Scholar] [CrossRef] [PubMed]
- Schott, J. It’s just a first step, but this new Alzheimer’s drug could be a huge breakthrough. Guardian 2022, www.theguardian.com/commentisfree/2022/nov/30/alzheimers-drug-lecanemab-trials-nhs (accessed 05/01/2023).
- Zetterberg, H.; Schott, J.M. Blood biomarkers for Alzheimer’s disease and related disorders. Acta Neurol. Scand. 2022, 146, 51–55. [Google Scholar] [CrossRef] [PubMed]
- Hsieh, S.; McGrory, S.; Leslie, F.; Dawson, K.; Ahmed, S.; Butler, C.R.; Rowe, J.B.; Mioshi, E.; Hodges, J.R. The Mini-Addenbrooke’s Cognitive Examination: a new assessment tool for dementia. Dement. Geriatr. Cogn. Disord. 2015, 39, 1–11. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Mini-Addenbrooke’s Cognitive Examination (MACE): a useful cognitive screening instrument in older people? Can. Geriatr. J. 2020, 23, 199–204. [Google Scholar] [CrossRef] [PubMed]
- Burns, A.; Harrison, J.R.; Symonds, C.; Morris, J. A novel hybrid scale for the assessment of cognitive and executive function: the Free-Cog. Int. J. Geriatr. Psychiatry 2021, 36, 566–572. [Google Scholar] [CrossRef]
- Burns, A.; Larner, A.J. Do we need yet another cognitive test? Free-Cog, a novel, hybrid, cognitive screening instrument. J. Neurol. Neurosurg. Psychiatry 2021, 92, 1359–1360. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Free-Cog: pragmatic test accuracy study and comparison with Mini-Addenbrooke’s Cognitive Examination. Dement. Geriatr. Cogn. Disord. 2019, 47, 254–263. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Cognitive testing in the COVID-19 era: Can existing screeners be adapted for telephone use? Neurodegener. Dis. Manag. 2021, 11, 77–82. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Communicating risk: developing an “Efficiency Index” for dementia screening tests. Brain Sci. 2021, 11, 1473. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. MACE for diagnosis of dementia and MCI: examining cut-offs and predictive values. Diagnostics (Basel) 2019, 9, E51. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J.; Burns, A. Free-Cog reformulated: analyses as independent or stepwise tests of cognitive and executive function. medRxiv 2023, https://medrxiv.org/cgi/content/short/2023.01.29.23285153v1.
- Hsieh, S.; Schubert, S.; Hoon, C.; Mioshi, E.; Hodges, J.R. Validation of the Addenbrooke’s Cognitive Examination III in frontotemporal dementia and Alzheimer’s disease. Dement. Geriatr. Cogn. Disord. 2013, 36, 242–250. [Google Scholar] [CrossRef] [PubMed]
- Noel-Storr, A.H.; McCleery, J.M.; Richard, E.; Ritchie, C.W.; Flicker, L.; Cullum, S.J.; Davis, D.; Quinn, T.J.; Hyde, C.; Rutjes, A.W.S.; Smailagic, N.; Marcus, S.; Black, S.; Blennow, K.; Brayne, C.; Fiorivanti, M.; Johnson, J.K.; Köpke, S.; Schneider, L.S.; Simmons, A.; Mattsson, N.; Zetterberg, H.; Bossuyt, P.M.M.; Wilcock, G.; McShane, R. Reporting standards for studies of diagnostic test accuracy in dementia: the STARDdem Initiative. Neurology 2014, 83, 364–373. [Google Scholar] [CrossRef] [PubMed]
- Strauss, S.E.; Glasziou, P.; Richardson, W.S.; Haynes, R.B. Evidence-based medicine. How to practice and teach EBM. 5th edition. Edinburgh: Elsevier; 2019, 165-166.
- Larner, A.J.; Hancock, P. Does combining cognitive and functional scales facilitate the diagnosis of dementia? Int. J. Geriatr. Psychiatry 2012, 27, 547–548. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Cognitive screening instruments for dementia: comparing metrics of test limitation. Dement. Neuropsychol. 2021, 15, 458–463. [Google Scholar] [CrossRef] [PubMed]
- Beishon, L.C.; Batterham, A.P.; Quinn, T.J.; Nelson, C.P.; Panerai, R.B.; Robinson, T.; Haunton, V.J. Addenbrooke’s Cognitive Examination III (ACE-III) and mini-ACE for the detection of dementia and mild cognitive impairment. Cochrane Database Syst Rev. 2019, 12, CD013282. [Google Scholar] [PubMed]
- Larner, A.J. Evaluating binary classifiers: extending the Efficiency Index. Neurodegener. Dis. Manag. 2022, 12, 185–194. [Google Scholar] [CrossRef] [PubMed]
- Ranson, J.M.; Kuźma, E.; Hamilton, W.; Lang, I.; Llewellyn, D.J. Case-finding in clinical practice: an appropriate strategy for dementia identification? Alzheimers Dement (N Y). 2018, 4, 288–296. [Google Scholar] [CrossRef] [PubMed]
Table 1.
Item content of Free-Cog and MACE.
| Free-Cog [10] | Mini-Addenbrooke’s Cognitive Examination (MACE) [8] | |
|---|---|---|
| “Cognitive Function” | ||
| General Knowledge | 1 | - |
| Orientation: Time | 3 | 4 |
| Orientation: Place | 3 | - |
| Registration | 0 (5 words) |
7 (7-item name and address, scored on third presentation) |
| Calculation | 3 | - |
| Attention/Concentration | 2 | - |
| Memory recall: | 5 (Recall of previously presented 5 words) |
7 (Recall of previously presented 7-item name and address) |
| Verbal fluency in 1 minute | 1 (semantic: animals) |
7 (phonemic: P words) |
| Language: Naming | 2 | - |
| Language: Repetition | 1 | - |
| Language: write a sentence | 1 | - |
| Visuospatial abilities: Wire (Necker) cube | - | - |
| Visuospatial abilities: Clock drawing test | 3 | 5 |
| “Executive Function” | ||
| 5 (questions relating to social function, travel, home, emergency, and care function) | - | |
| Total Score | 30 | 30 |
Table 2.
Demographics: Free-Cog cohort.
| Cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC) | |
|---|---|
| Whole cohort: | |
| N (dementia plus MCI vs SMC) | 141 (60 vs 81) |
| F:M (%F) | 61:80 (43.3%) |
| Prevalence (P = pre-test probability) |
Dementia plus MCI 0.426 |
| Pre-test odds (= P/1 – P) |
Dementia plus MCI 0.741 |
| Cohort aged ≥65 years: | |
| N (dementia plus MCI vs SMC) | 65 (44 vs 21) |
| F:M (%F) | 31:34 (47.7%) |
| Prevalence (P = pre-test probability) |
Dementia plus MCI 0.677 |
| Pre-test odds (= P/1 – P) |
Dementia plus MCI 2.095 |
| Cohort aged ≥75 years: | |
| N (dementia plus MCI vs SMC) | 25 (22 vs 3) |
| F:M (%F) | 14:11 (56.0%) |
| Prevalence (P = pre-test probability) |
Dementia plus MCI 0.880 |
| Pre-test odds (= P/1 – P) |
Dementia plus MCI 7.333 |
Table 3.
Free-Cog for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤22/30 in whole cohort versus cohorts of older patients (aged ≥65 and ≥75 years).
Table 3.
Free-Cog for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤22/30 in whole cohort versus cohorts of older patients (aged ≥65 and ≥75 years).
| Whole cohort | Older patients aged ≥65 years |
Older patients aged ≥75 years |
|
|---|---|---|---|
| N | 141 | 65 | 25 |
| Age:Free-Cog correlation | -0.28 | -0.31 | -0.38 |
| Sensitivity (Sens) |
1.00 | 0.73 (0.60-0.86) |
0.73 (0.56-0.91) |
| Specificity (Spec) |
0.67 (0.59-0.76) |
0.95 (0.86-1.00) |
1.00 |
| Positive Predictive Value (PPV) | 0.27 (0.15-0.38) |
0.97 (0.63-1.00) |
1.00 |
| Negative Predictive Value (NPV) | 1.00 | 0.63 (0.46-0.79) |
0.33 (0.03-0.64) |
| Correct classification accuracy (Acc) | 0.71 (0.63-0.78) |
0.8 (0.70-0.90) |
0.76 (0.59-0.93) |
| Efficiency Index (EI) | 2.45 | 4.0 | 3.17 |
Table 4.
Demographics: MACE cohort.
| Age cohort (years) |
N | Gender F:M (% female) | Prevalence (P) | Pre-test odds | Age:MACE correlation |
|---|---|---|---|---|---|
| <50 | 159 | 80:79 (50.3%) | 0.088 | 0.097 | -0.11 |
| 50-54 | 99 | 50:49 (50.5%) | 0.232 | 0.303 | -0.00 |
| 55-59 | 97 | 46:51 (47.4%) | 0.381 | 0.617 | -0.04 |
| 60-64 | 113 | 41:72 (36.3%) | 0.434 | 0.766 | -0.05 |
| 65-69 | 101 | 37:64 (36.6%) | 0.515 | 1.061 | -0.08 |
| 70-74 | 67 | 29:38 (43.3%) | 0.776 | 3.467 | 0.02 |
| 75-79 | 60 | 38:22 (63.3%) | 0.967 | 29.0 | -0.00 |
| ≥80 | 59 | 31:28 (52.5%) | 0.898 | 8.833 | 0.01 |
| Whole | 755 | 352:403 (46.6%) | 0.151 | 0.178 | -0.29 |
Table 5.
MACE for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤20/30 in different age cohorts versus whole cohort.
Table 5.
MACE for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤20/30 in different age cohorts versus whole cohort.
| Age cohort | Sens | Spec | PPV | NPV | Acc | EI |
|---|---|---|---|---|---|---|
| <50 | 0.64 (0.39-0.89) |
0.81 (0.75-0.88) |
0.25 (0.11-0.39) |
0.96 (0.92-0.99) |
0.80 (0.74-0.86) |
3.97 |
| 50-54 | 0.78 (0.61-0.95) |
0.80 (0.71-0.89) |
0.55 (0.38-0.72) |
0.92 (0.86-0.99) |
0.80 (0.72-0.88) |
3.80 |
| 55-59 | 0.73 (0.59-0.87) |
0.78 (0.68-0.89) |
0.68 (0.53-0.82) |
0.82 (0.73-0.92) |
0.76 (0.68-0.85) |
3.22 |
| 60-64 | 0.61 (0.48-0.75) |
0.83 (0.74-0.92) |
0.73 (0.60-0.87) |
0.74 (0.63-0.84) |
0.73 (0.65-0.82) |
2.77 |
| 65-69 | 0.50 (0.36-0.64) |
0.98 (0.94-1.00) |
0.96 (0.89-1.00) |
0.65 (0.54-0.76) |
0.73 (0.65-0.82) |
2.74 |
| 70-74 | 0.67 (0.55-0.80) |
1.00 |
1.00 |
0.47 (0.30-0.64) |
0.75 (0.64-0.85) |
2.94 |
| 75-79 | 0.67 (0.55-0.79) |
1.00 |
1.00 |
0.10 (0.00-0.22) |
0.68 (0.57-0.80) |
2.16 |
| ≥80 | 0.77 (0.66-0.89) |
1.00 |
1.00 | 0.33 (0.12-0.55) |
0.80 (0.69-0.90) |
3.92 |
| Whole cohort |
0.67 (0.62-0.72) |
0.84 (0.80-0.87) |
0.77 (0.72-0.82) |
0.76 (0.72-0.80) |
0.76 (0.73-0.79) |
3.19 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Submitted:
11 April 2023
Posted:
12 April 2023
You are already at the latest version
Alerts
This version is not peer-reviewed
Submitted:
11 April 2023
Posted:
12 April 2023
You are already at the latest version
Alerts
Abstract
Ageing is the strongest known risk factor for many of the neurodegenerative diseases causing cognitive decline and dementia. Identification of cognitive impairment may be a prelude to appropriate treatment, hopefully disease-modifying. Use of cognitive screening instruments may be an equitable way to identify cognitive impairment. This study examined the use of two such instruments, Free-Cog and Mini-Addenbrooke’s Cognitive Examination (MACE), in patient cohorts referred to a dedicated cognitive disorders clinic based at a tertiary neurosciences centre. Results showed that: (1) specificity and positive predictive value increased with patient age for both tests with some loss of sensitivity and negative predictive value. (2) In the oldest age groups (≥75 and ≥70 years respectively) where specificity was at maximum, a positive test result (i.e. below the specified test cut-off) rules in the diagnosis of cognitive impairment. (3) Values of an “Efficiency Index” for each test indicated qualitatively a moderate change in the probability of correct diagnosis and quantitatively an approximately 15-25% increase in the probability of correct diagnosis. These findings show that both Free-Cog and MACE may be used with confidence for the identification of cognitive impairment and dementia in older patient cohorts. These findings may have implications for public health policies directed to case-finding in clinical practice as opposed to population-based screening.
Keywords:
Subject: Medicine and Pharmacology - Neuroscience and Neurology
1. Introduction
Ageing is the strongest known risk factor for many of the neurodegenerative diseases of the brain which give rise to cognitive decline, sometimes sufficient to meet diagnostic criteria for dementia. The development of age-related cognitive impairment and dementia is recognised to constitute one of the most profound global challenges to public health in the 21st century, in part as a consequence of the ageing of populations around the world.
Nevertheless, screening for cognitive impairment and dementia in older adults remains a contentious issue. Such screening is predicated on the idea that identification of cognitive impairment will be the prelude to early interventions to improve outcomes. Whilst various screening instruments have been shown capable of detecting cognitive impairment in older adults, and multiple practice guidelines and consensus statements on the detection of mild cognitive impairment (MCI) have been published [1], evidence for benefit in terms of improved patient or caregiver outcomes is currently lacking [2,3]. Hence the criteria set down in the World Health Organisation (WHO) guidelines for screening [4] are not currently fulfilled for dementia and cognitive impairment.
However, therapeutic interventions for early cognitive impairment may now possibly be within sight [5] (although their delivery in terms of costs and logistics remains daunting [6]). If such should prove to be the case, then screening for cognitive impairment may become an accepted and appropriate management strategy, consistent with WHO screening criteria. Investigations to detect neurodegenerative pathology leading to cognitive impairment based around disease biomarkers (e.g. neuroimaging, CSF, and ultimately blood [7]) hold out great promise in this area. However, since these investigations may not be readily scaled up to universal availability, the use of cognitive screening instruments is likely to remain an integral part of clinical assessment in the immediate future. Equity of global health care provision may be ensured by the relative ease of accessibility of such screening tests (unlike biomarker neuroimaging, CSF, and blood tests).
Multiple cognitive screening instruments are available and many have been specifically assessed for the detection of cognitive impairment in older adult populations. For example, the test accuracy of the Mini-Addenbrooke’s Cognitive Examination (MACE) [8] for the identification of dementia and MCI in a cohort of older patients has previously been reported in a neurology-led dedicated cognitive disorders clinic [9]. MACE test accuracy has also been compared to another short (and copyright-free) cognitive screening instrument, Free-Cog [10,11], in both standard and abridged forms suitable respectively for face-to-face [12] and remote (telephone, video) [10,13] testing.
The aims of the current study were threefold: firstly, to undertake the same analysis of age-related test accuracy for Free-Cog as previously reported for MACE [9]; secondly, to extend the age-related test accuracy analysis for MACE to produce a more fine-grained examination of the performance of this instrument with patient age; and thirdly, in both cases to extend the range of test outcome measures to include the “Efficiency Index,” a recently described unitary test outcome measure which permits ready communication of the risk of whether a test outcome results in correct diagnosis rather than misdiagnosis [14].
2. Materials and Methods
2.1. Participants
Datasets from two pragmatic prospective screening test accuracy studies examining MACE (June 2014-December 2018 inclusive) [15] and Free-Cog (November 2017-October 2018 inclusive) [12] respectively were re-interrogated. Both studies recruited consecutive new patients referred to a neurology-led dedicated cognitive function clinic based in a regional neuroscience centre. The clinic does not operate any age-related exclusion criteria.
2.2. Procedures
Patient assessment comprised semi-structured history enquiring about cognitive symptoms and functional performance, with collateral history from a reliable and knowledgeable informant where possible. All patients underwent neuroradiological examination with brain computed tomography (CT), with interval magnetic resonance (MR) brain imaging in some cases. Formal neuropsychological assessment was pursued in some cases where there was remaining diagnostic uncertainty.
Administration of Free-Cog or MACE occurred on the same day as, but separate from, the cross-sectional assessment. Free-Cog is a hybrid screening instrument of cognition and executive function (item content shown in Table 1, left hand column), purposely designed to have a conversational quality in order to put patients at their ease and hence maximise performance [10]. Free-Cog was used in its standard, unitary formulation, rather than as separate cognitive and executive function tests formulated either sequentially using Boolean logical operators or as a stepwise decision tree, as examined elsewhere [16]. MACE (item content shown in Table 1, right hand column) is a brief cognitive screening instrument derived from the longer Addenbrooke’s Cognitive Examination in its third iteration, ACE-III [17]. Both Free-Cog and MACE are scored out of 30 points, with higher scores better.
Patients were diagnosed with dementia or MCI according to standard diagnostic criteria (DSM-IV, Petersen, respectively), as previously used in this clinic [12,15]. Criterion diagnosis of dementia, MCI, or subjective memory complaint (SMC), was by the judgment of an experienced clinician based on the specified diagnostic criteria but did not use either Free-Cog or MACE score in order to avoid review bias. STARDdem guidelines for reporting diagnostic test accuracy studies in dementia were observed [18].
2.3. Analyses
For Free-Cog, a hybrid screening instrument of cognition and executive function, data from patient cohorts aged ≥65 and ≥75 years, as well as the whole cohort, were examined (as in a previous study of MACE [9]). A cut-off of ≤22/30, defined from the study data by maximal Youden index [12], was used (a test cut-off was not specified in the index study of Free-Cog [10]).
For MACE (item content shown in Table 1, right hand column), the original analysis [9] was extended such that data were examined in 5-year cohorts between the ages of 50 and 79, as well as in catch-all groups <50 and ≥80 years. A MACE cut-off of ≤20/30, defined from the study data by maximal Youden index [15], was used, as opposed to the two MACE cut-offs (≤25/30, more sensitive, and ≤21/30, more specific) defined in the index study [8] and used in the previous age-related analysis of MACE [8].
Test accuracy outcomes at the chosen cut-offs were expressed in terms of sensitivity (Sens) and specificity (Spec), positive and negative predictive values (PPV, NPV), and overall test accuracy (or correct classification accuracy, Acc).
Sens and Spec results were interpreted in terms of the heuristic “SnNOut” and “SpPin” rules: with a highly sensitive test, a negative result rules out the diagnosis, whereas with a highly specific test, a positive result rules in the diagnosis [19].
The Efficiency Index (EI) [14] of each test in each age cohort was calculated on the basis of the values for Acc and its complement, Inaccuracy (Inacc), defined as:
EI = Acc/(1 – Acc)
= Acc/Inacc
This metric has a range 0-∞, where higher scores are better, with an inflection point at 1 (EI<1 favours misdiagnosis, EI>1 favours correct diagnosis, hence EI values >>1 are desired). EI values were classified qualitatively and semi-quantitatively as for likelihood ratios, as previously shown [14].
3. Results
3.1. Free-Cog
For the Free-Cog cohort (N = 141; demographics shown in Table 2), the prevalence of cognitive impairment increased with cohort age, as was anticipated.
Free-Cog test outcomes (Table 3) showed that test Sens and NPV declined with cohort age but that Spec and PPV increased, both reaching maximal values in the ≥75 years cohort. Following the “SpPin” rule [19], as a highly specific test, a positive Free-Cog test result (i.e. ≤22/30) likely rules in the diagnosis of cognitive impairment.
The values for both Acc and EI for Free-Cog were higher in the older patient cohorts compared to the whole cohort (Table 3). The EI values in the different cohorts examined ranged between ≈2-4, which indicates a qualitatively moderate change in the probability of correct diagnosis and quantitatively an approximately 15-25% increase in the probability of correct diagnosis [14].
3.2. MACE
For the MACE cohort (N = 755; demographics shown in Table 4), the prevalence of cognitive impairment increased with cohort age, as was anticipated.
MACE test outcomes (Table 5) showed that test Spec and PPV increased with cohort age, both reaching maximal values in the patient cohorts aged ≥70 years. NPV declined whereas Sens remained relatively stable across the age cohorts. Following the “SpPin” rule [19], as a highly specific test, a positive MACE result (i.e. ≤20/30) likely rules in the diagnosis of cognitive impairment.
The values for both Acc and EI for MACE remained relatively stable across the age cohorts (Table 5). The EI values in the different cohorts examined ranged between ≈2-4, which indicates a qualitatively moderate change in the probability of correct diagnosis and quantitatively an approximately 15-25% increase in the probability of correct diagnosis [14].
4. Discussion
Reanalysis of the dataset from the Free-Cog study [12] showed improved values for Spec and PPV with increasing patient age. Better Spec indicated fewer false positive results and hence, in accordance with the heuristic “SpPin” rule [19], a positive result (Free-Cog ≤22/30) likely rules in the diagnosis of cognitive impairment. This is the combination of outcomes that would be desirable for a screening test for cognitive impairment in older patient cohorts. This outcome might possibly reflect the hybrid nature of Free-Cog as a test of both cognitive and executive functions, since other studies have suggested such a combination may improve detection of neurocognitive disorder [20], although fragmenting the Free-Cog into separate tests of cognitive and executive functions did not suggest better overall performance than the unitary Free-Cog [16]. Other analyses of Free-Cog have previously shown that it fares well in comparison to other short cognitive screening instruments in so-called “metrics of limitation” such as misclassification rate and net harm/net benefit ratio [21].
Extending the reanalysis of the dataset from the MACE study [15] showed improved values for PPV with increasing patient age, as in the original study [9], but also improved Spec which was not apparent in the original study, perhaps related to the broader age cohorts used therein. In the current analysis, Spec was at maximum in patient cohorts aged ≥70 years, hence in accordance with the heuristic “SpPin” rule [19] a positive result (MACE ≤20/30) rules in the diagnosis of cognitive impairment.
These findings – improved values of both Spec and PPV for both Free-Cog and MACE with increasing patient age, reaching maximal values in the patient cohorts aged ≥75 years and ≥70 years respectively – are of particular interest in view of the characterisation of both Free-Cog [12] and MACE [15,22] as high sensitivity low specificity screeners on the basis of their overall test performance. Of course, the potential shortcomings of Sens and Spec and predictive value metrics are well-recognised, prompting the search for other outcome measures by which to judge test utility.
In this context the calculation of a novel, recently described test metric, the Efficiency Index [14], a measure of the likelihood of correct diagnosis versus misdiagnosis, proved easy to use. For Free-Cog, it suggested an improved EI in older patient cohorts compared to the whole cohort, and for MACE it suggested relative stability of EI across the examined age cohorts. Qualitatively the EI values suggested a moderate change in the probability of correct diagnosis of disease and semi-quantitatively an approximately 15-25% increase in the probability of correct diagnosis [14].
Other, more stringent, formulations of EI, taking into account disease prevalence and/or test cut-off, are available [23] and might be examined in future studies of screening instruments. Other priorities for future study include further examination of whether screening of cognitive domains alone or in combination with executive functions offers the optimal strategy for case-finding. The canonical definitions of dementia (DSM-IV) or major neurocognitive disorder (DSM-5) encompass deficits in both cognition and function, unlike MCI or minor neurocognitive disorder. The current analyses did not examine head-to-head test data, but in a previous study Free-Cog was more specific than MACE for diagnosis of both dementia and MCI [12].
How might the findings of this study fit more broadly into the existing (and possibly future) public health policies for dementia screening or identification? Currently there is no indication for population-based screening for dementia. The UK National Screening Committee on Screening for Dementia has been explicit in its statements recommending against screening healthy individuals aged 65 years and over (2015 and 2019; at time of writing an update of this review is anticipated). Likewise, the US Preventive Services Task Force has found no benefit in terms of improved outcomes for dementia screening [2]. However, targeted screening of populations known to be at higher risk of dementia might have a role, such as those individuals with subjective memory complaints, or those referred to memory clinics, or with conditions which predispose to dementia (such as diabetes mellitus), as these groups are known to have a higher prevalence of dementia and cognitive impairment than the general population. Another suggested dementia identification strategy is case-finding in clinical practice, which may be a viable alternative to screening [24]. One of the proposed stages of this approach, which in some ways reflects the idiom of clinical practice, involves cognitive assessment, but the authors found evidence for this stage to be lacking [24]. The current findings might contribute evidence in support of such an approach using either of these cognitive screening instruments. Certainly both Free-Cog and MACE were found to be acceptable to patients in the respective test accuracy studies [12,15]. This approach to case-finding might also be extended to remote assessment methods by using appropriate modifications of these instruments (Tele-Free-Cog, Tele-MACE) [10,13].
5. Limitations
The limitations of this study are those familiar in any clinic-based study. The selected study population had a relatively high prevalence of dementia and MCI compared to patient cohorts in community-based (e.g. primary care or population-based) cohorts. The Free-Cog study cohort was quite small (<150), meaning that confidence intervals for test metrics were wide, particularly in the age ≥75 years cohort. This was reflective of the relatively young age of the patients seen in neurology-based cognitive clinics (median age in Free-Cog study = 62 years [12]; in MACE study = 60 years [15]), unlike the case mix in old age psychiatry memory clinics. Use of cross-sectional clinical diagnoses as reference standard is idiomatic of day-to-day practice but potentially liable to error without delayed verification (e.g. no neuropathological data were available). All these factors might limit the generalizability of the current study findings.
6. Conclusions
Both Free-Cog and MACE may be of value in ruling in a diagnosis of cognitive impairment in older patient cohorts presenting with memory complaints. They might therefore be instrumental in implementing any policy of cognitive screening or case-finding in older adult populations.
Funding
The author received no specific funding for this work.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Committee on Human Research of the Walton Centre for Neurology and Neurosurgery (Approval: N 310)
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request.
Conflicts of Interest
The author declares no conflict of interest.
References
- Chen, Y.-X.; Liang, N.; Li, X.-L.; Yang, S.-H.; Wang, Y.-P.; Shi, N.-N. Diagnosis and treatment for mild cognitive impairment: a systematic review of clinical practice guidelines and consensus statements. Front. Neurol. 2021, 12, 719849. [Google Scholar] [CrossRef] [PubMed]
- US Preventive Services Task Force; Owens, D.K.; Davidson, K.W.; Krist, A.H.; Barry, M.J.; Cabana, M.; Caughey, A.B.; Doubeni, C.A.; Epling Jr, J.W.; Kubik, M.; Landefeld, C.S.; Mangione, C.A.; Pbert, L.; Silverstein, M.; Simon, M.A.; Tseng, C.-W.; Wong, J.B. Screening for cognitive impairment in older adults: US Preventive Services Task Force Recommendation Statement. JAMA 2020, 323, 757–763. [Google Scholar]
- Patnode, C.D.; Perdue, L.A.; Rossom, R.C.; Rushkin, M.C.; Redmond, N.; Thomas, R.G.; Lin, J.S. Screening for cognitive impairment in older adults: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA 2020, 323, 764–785. [Google Scholar] [CrossRef] [PubMed]
- Wilson, J.M.G.; Jungner, G. Principles and practice of screening for disease. Public health paper No. 34; World Health Organisation: Geneva, 1968. [Google Scholar]
- van Dyck, C.H.; Swanson, C.J.; Aisen, P.; Bateman, R.J.; Chen, C.; Gee, M.; Kanekiyo, M.; Li, D.; Reyderman, L.; Cohen, S.; Froelich, L.; Katayama, S.; Sabbagh, M.; Vellas, B.; Watson, D.; Dhadda, S.; Irizarry, M.; Kramer, L.D.; Iwatsubo, T. Lecanemab in early Alzheimer’s disease. N. Engl. J. Med. 2023, 388, 9–21. [Google Scholar] [CrossRef] [PubMed]
- Schott, J. It’s just a first step, but this new Alzheimer’s drug could be a huge breakthrough. Guardian 2022, www.theguardian.com/commentisfree/2022/nov/30/alzheimers-drug-lecanemab-trials-nhs (accessed 05/01/2023).
- Zetterberg, H.; Schott, J.M. Blood biomarkers for Alzheimer’s disease and related disorders. Acta Neurol. Scand. 2022, 146, 51–55. [Google Scholar] [CrossRef] [PubMed]
- Hsieh, S.; McGrory, S.; Leslie, F.; Dawson, K.; Ahmed, S.; Butler, C.R.; Rowe, J.B.; Mioshi, E.; Hodges, J.R. The Mini-Addenbrooke’s Cognitive Examination: a new assessment tool for dementia. Dement. Geriatr. Cogn. Disord. 2015, 39, 1–11. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Mini-Addenbrooke’s Cognitive Examination (MACE): a useful cognitive screening instrument in older people? Can. Geriatr. J. 2020, 23, 199–204. [Google Scholar] [CrossRef] [PubMed]
- Burns, A.; Harrison, J.R.; Symonds, C.; Morris, J. A novel hybrid scale for the assessment of cognitive and executive function: the Free-Cog. Int. J. Geriatr. Psychiatry 2021, 36, 566–572. [Google Scholar] [CrossRef]
- Burns, A.; Larner, A.J. Do we need yet another cognitive test? Free-Cog, a novel, hybrid, cognitive screening instrument. J. Neurol. Neurosurg. Psychiatry 2021, 92, 1359–1360. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Free-Cog: pragmatic test accuracy study and comparison with Mini-Addenbrooke’s Cognitive Examination. Dement. Geriatr. Cogn. Disord. 2019, 47, 254–263. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Cognitive testing in the COVID-19 era: Can existing screeners be adapted for telephone use? Neurodegener. Dis. Manag. 2021, 11, 77–82. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Communicating risk: developing an “Efficiency Index” for dementia screening tests. Brain Sci. 2021, 11, 1473. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. MACE for diagnosis of dementia and MCI: examining cut-offs and predictive values. Diagnostics (Basel) 2019, 9, E51. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J.; Burns, A. Free-Cog reformulated: analyses as independent or stepwise tests of cognitive and executive function. medRxiv 2023, https://medrxiv.org/cgi/content/short/2023.01.29.23285153v1.
- Hsieh, S.; Schubert, S.; Hoon, C.; Mioshi, E.; Hodges, J.R. Validation of the Addenbrooke’s Cognitive Examination III in frontotemporal dementia and Alzheimer’s disease. Dement. Geriatr. Cogn. Disord. 2013, 36, 242–250. [Google Scholar] [CrossRef] [PubMed]
- Noel-Storr, A.H.; McCleery, J.M.; Richard, E.; Ritchie, C.W.; Flicker, L.; Cullum, S.J.; Davis, D.; Quinn, T.J.; Hyde, C.; Rutjes, A.W.S.; Smailagic, N.; Marcus, S.; Black, S.; Blennow, K.; Brayne, C.; Fiorivanti, M.; Johnson, J.K.; Köpke, S.; Schneider, L.S.; Simmons, A.; Mattsson, N.; Zetterberg, H.; Bossuyt, P.M.M.; Wilcock, G.; McShane, R. Reporting standards for studies of diagnostic test accuracy in dementia: the STARDdem Initiative. Neurology 2014, 83, 364–373. [Google Scholar] [CrossRef] [PubMed]
- Strauss, S.E.; Glasziou, P.; Richardson, W.S.; Haynes, R.B. Evidence-based medicine. How to practice and teach EBM. 5th edition. Edinburgh: Elsevier; 2019, 165-166.
- Larner, A.J.; Hancock, P. Does combining cognitive and functional scales facilitate the diagnosis of dementia? Int. J. Geriatr. Psychiatry 2012, 27, 547–548. [Google Scholar] [CrossRef] [PubMed]
- Larner, A.J. Cognitive screening instruments for dementia: comparing metrics of test limitation. Dement. Neuropsychol. 2021, 15, 458–463. [Google Scholar] [CrossRef] [PubMed]
- Beishon, L.C.; Batterham, A.P.; Quinn, T.J.; Nelson, C.P.; Panerai, R.B.; Robinson, T.; Haunton, V.J. Addenbrooke’s Cognitive Examination III (ACE-III) and mini-ACE for the detection of dementia and mild cognitive impairment. Cochrane Database Syst Rev. 2019, 12, CD013282. [Google Scholar] [PubMed]
- Larner, A.J. Evaluating binary classifiers: extending the Efficiency Index. Neurodegener. Dis. Manag. 2022, 12, 185–194. [Google Scholar] [CrossRef] [PubMed]
- Ranson, J.M.; Kuźma, E.; Hamilton, W.; Lang, I.; Llewellyn, D.J. Case-finding in clinical practice: an appropriate strategy for dementia identification? Alzheimers Dement (N Y). 2018, 4, 288–296. [Google Scholar] [CrossRef] [PubMed]
Table 1.
Item content of Free-Cog and MACE.
| Free-Cog [10] | Mini-Addenbrooke’s Cognitive Examination (MACE) [8] | |
|---|---|---|
| “Cognitive Function” | ||
| General Knowledge | 1 | - |
| Orientation: Time | 3 | 4 |
| Orientation: Place | 3 | - |
| Registration | 0 (5 words) |
7 (7-item name and address, scored on third presentation) |
| Calculation | 3 | - |
| Attention/Concentration | 2 | - |
| Memory recall: | 5 (Recall of previously presented 5 words) |
7 (Recall of previously presented 7-item name and address) |
| Verbal fluency in 1 minute | 1 (semantic: animals) |
7 (phonemic: P words) |
| Language: Naming | 2 | - |
| Language: Repetition | 1 | - |
| Language: write a sentence | 1 | - |
| Visuospatial abilities: Wire (Necker) cube | - | - |
| Visuospatial abilities: Clock drawing test | 3 | 5 |
| “Executive Function” | ||
| 5 (questions relating to social function, travel, home, emergency, and care function) | - | |
| Total Score | 30 | 30 |
Table 2.
Demographics: Free-Cog cohort.
| Cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC) | |
|---|---|
| Whole cohort: | |
| N (dementia plus MCI vs SMC) | 141 (60 vs 81) |
| F:M (%F) | 61:80 (43.3%) |
| Prevalence (P = pre-test probability) |
Dementia plus MCI 0.426 |
| Pre-test odds (= P/1 – P) |
Dementia plus MCI 0.741 |
| Cohort aged ≥65 years: | |
| N (dementia plus MCI vs SMC) | 65 (44 vs 21) |
| F:M (%F) | 31:34 (47.7%) |
| Prevalence (P = pre-test probability) |
Dementia plus MCI 0.677 |
| Pre-test odds (= P/1 – P) |
Dementia plus MCI 2.095 |
| Cohort aged ≥75 years: | |
| N (dementia plus MCI vs SMC) | 25 (22 vs 3) |
| F:M (%F) | 14:11 (56.0%) |
| Prevalence (P = pre-test probability) |
Dementia plus MCI 0.880 |
| Pre-test odds (= P/1 – P) |
Dementia plus MCI 7.333 |
Table 3.
Free-Cog for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤22/30 in whole cohort versus cohorts of older patients (aged ≥65 and ≥75 years).
Table 3.
Free-Cog for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤22/30 in whole cohort versus cohorts of older patients (aged ≥65 and ≥75 years).
| Whole cohort | Older patients aged ≥65 years |
Older patients aged ≥75 years |
|
|---|---|---|---|
| N | 141 | 65 | 25 |
| Age:Free-Cog correlation | -0.28 | -0.31 | -0.38 |
| Sensitivity (Sens) |
1.00 | 0.73 (0.60-0.86) |
0.73 (0.56-0.91) |
| Specificity (Spec) |
0.67 (0.59-0.76) |
0.95 (0.86-1.00) |
1.00 |
| Positive Predictive Value (PPV) | 0.27 (0.15-0.38) |
0.97 (0.63-1.00) |
1.00 |
| Negative Predictive Value (NPV) | 1.00 | 0.63 (0.46-0.79) |
0.33 (0.03-0.64) |
| Correct classification accuracy (Acc) | 0.71 (0.63-0.78) |
0.8 (0.70-0.90) |
0.76 (0.59-0.93) |
| Efficiency Index (EI) | 2.45 | 4.0 | 3.17 |
Table 4.
Demographics: MACE cohort.
| Age cohort (years) |
N | Gender F:M (% female) | Prevalence (P) | Pre-test odds | Age:MACE correlation |
|---|---|---|---|---|---|
| <50 | 159 | 80:79 (50.3%) | 0.088 | 0.097 | -0.11 |
| 50-54 | 99 | 50:49 (50.5%) | 0.232 | 0.303 | -0.00 |
| 55-59 | 97 | 46:51 (47.4%) | 0.381 | 0.617 | -0.04 |
| 60-64 | 113 | 41:72 (36.3%) | 0.434 | 0.766 | -0.05 |
| 65-69 | 101 | 37:64 (36.6%) | 0.515 | 1.061 | -0.08 |
| 70-74 | 67 | 29:38 (43.3%) | 0.776 | 3.467 | 0.02 |
| 75-79 | 60 | 38:22 (63.3%) | 0.967 | 29.0 | -0.00 |
| ≥80 | 59 | 31:28 (52.5%) | 0.898 | 8.833 | 0.01 |
| Whole | 755 | 352:403 (46.6%) | 0.151 | 0.178 | -0.29 |
Table 5.
MACE for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤20/30 in different age cohorts versus whole cohort.
Table 5.
MACE for diagnosis of cognitive impairment (dementia plus MCI) versus no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% confidence intervals) using test cut-off ≤20/30 in different age cohorts versus whole cohort.
| Age cohort | Sens | Spec | PPV | NPV | Acc | EI |
|---|---|---|---|---|---|---|
| <50 | 0.64 (0.39-0.89) |
0.81 (0.75-0.88) |
0.25 (0.11-0.39) |
0.96 (0.92-0.99) |
0.80 (0.74-0.86) |
3.97 |
| 50-54 | 0.78 (0.61-0.95) |
0.80 (0.71-0.89) |
0.55 (0.38-0.72) |
0.92 (0.86-0.99) |
0.80 (0.72-0.88) |
3.80 |
| 55-59 | 0.73 (0.59-0.87) |
0.78 (0.68-0.89) |
0.68 (0.53-0.82) |
0.82 (0.73-0.92) |
0.76 (0.68-0.85) |
3.22 |
| 60-64 | 0.61 (0.48-0.75) |
0.83 (0.74-0.92) |
0.73 (0.60-0.87) |
0.74 (0.63-0.84) |
0.73 (0.65-0.82) |
2.77 |
| 65-69 | 0.50 (0.36-0.64) |
0.98 (0.94-1.00) |
0.96 (0.89-1.00) |
0.65 (0.54-0.76) |
0.73 (0.65-0.82) |
2.74 |
| 70-74 | 0.67 (0.55-0.80) |
1.00 |
1.00 |
0.47 (0.30-0.64) |
0.75 (0.64-0.85) |
2.94 |
| 75-79 | 0.67 (0.55-0.79) |
1.00 |
1.00 |
0.10 (0.00-0.22) |
0.68 (0.57-0.80) |
2.16 |
| ≥80 | 0.77 (0.66-0.89) |
1.00 |
1.00 | 0.33 (0.12-0.55) |
0.80 (0.69-0.90) |
3.92 |
| Whole cohort |
0.67 (0.62-0.72) |
0.84 (0.80-0.87) |
0.77 (0.72-0.82) |
0.76 (0.72-0.80) |
0.76 (0.73-0.79) |
3.19 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Triple Test Plus Rapid Cognitive Screening Test: A Combination of Clinical Signs and A Tool for Cognitive Assessment in Older Adults
Saadet Koc Okudur
et al.
Diagnostics,
2019
The Persian Version of the Quick Mild Cognitive Impairment Screen (Qmci-Pr): Psychometric Properties among Middle-Aged and Older Iranian Adults
Mohammad Rezaei
et al.
IJERPH,
2021
Comparing the Diagnostic Accuracy of Two Cognitive Screening Instruments in Different Dementia Subtypes and Clinical Depression
Rónán O’Caoimh
et al.
Diagnostics,
2019
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated