Reliability, Validity, and Cutoff Point of the Chinese Version of the Chelsea Critical Care Physical Assessment Tool in Critically Ill Patients

doi:10.21203/rs.3.rs-69289/v1

Download PDF

Research

Reliability, Validity, and Cutoff Point of the Chinese Version of the Chelsea Critical Care Physical Assessment Tool in Critically Ill Patients

https://doi.org/10.21203/rs.3.rs-69289/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Purpose: Translation and cross-cultural adaptation of the Chelsea Critical Care Physical Assessment Tool (CPAx) into a Chinese version of CPAx (“CPAx-Chi”), test the reliability and validity of CPAx-Chi, and verify the cutoff point for the diagnosis of intensive care unit-acquired weakness (ICU-AW)

Material and methods: Translation and cross-cultural adaptation of CPAx into CPAx-Chi was based on the Brislin model. Participants were recruited from the general ICU of five third-grade class-A hospitals in western China. Adult patients (n = 200) 48 h after receiving intensive care were included (median age, 53 years; 64% males). Patients were assessed by two assessment scales: Medical Research Council Muscle Score (MRC-Score) and CPAx-Chi.

Results: The item-level content validity was 0.889. The scale-level content validity was 0.955. Taking the MRC-Score scale as standard, the criterion validity of CPAx-Chi was r = 0.758 (p < 0.001) for Researcher A, and r = 0.65 (p < 0.001) for Researcher B. Cronbach’s ɑ was 0.939. The inter-rater reliability was 0.902 (p < 0.001). The AUC of CPAx-Chi for diagnosing ICU-AW based on MRC-Score ≤48 was 0.899 (95%CI 0.862–1.025) and 0.874 (0.824–0.925) for Researcher B. The maximum value of the Youden Index was 0.643, and the best cutoff point for CPAx-Chi for the diagnosis of ICU-AW was 31.5. The sensitivity was 87% and specificity was 77% for Researcher A, whereas it was 0.621, 31.5, 75%, and 87% for Researcher B, respectively. The consistency was high when taking CPAx-Chi ≤31 and MRC-Score ≤48 as the cutoff points for the diagnosis of ICU-AW. Also, kappa = 0.845 (p = 0.02) in Researcher A and 0.839 (p = 0.04) for Researcher B.

Conclusions: CPAx-Chi had good content validity, criterion-related validity and reliability. CPAx-Chi showed the best accuracy in assessment of patients at risk for ICU-AW with good sensitivity and specificity at a recommended cutoff of 31.

Critical Care & Emergency Medicine

ICU-acquired weakness

Reliability

Validity

Cutoff point

Intensive care unit-acquired weakness (ICU-AW) is a severe and debilitating complication in critically ill patients. The prevalence of ICU-AW in patients receiving mechanical ventilation for more than 4–7 days has been reported to be 33–82%^[1–5]. The prevalence of ICU-AW in sepsis patients is 100%^[3–5]. Early identification, assessment and active prevention are crucial to reduce ICU-AW risk because the pathophysiological mechanism of ICU-AW is not clear, and efficacious pharmacotherapy is lacking^{[1, 6]}.

A “gold standard” for ICU-AW is not available. The Medical Research Council Muscle Score (MRC-Score) is the most widely used diagnostic tool for ICU-AW^[7]. The MRC-Score evaluates the strength subjectively in three muscle groups of all four limbs according to the Oxford Muscle Strength Grading Scale. The latter is not only affected by several factors, it also cannot evaluate respiratory function. Several studies have shown that diaphragmatic dysfunction is correlated significantly with ICU-AW^[8–10], and that the function of respiratory muscles may be related to the occurrence and development of ICU-AW.

Chelsea Critical Care Physical Assessment Tool (CPAx) could be the optimal tool for predicting and evaluating ICU-AW. CPAx not only includes physical function, mobility and grip strength, it also includes respiratory function and cough ability^[11–13]. CPAx has been translated into several languages for use in the UK, Sweden, Denmark and other countries^[14–15]. However, a Chinese version of CPAx, or the cutoff point of CPAx for the diagnosis of ICU-AW, is lacking. Therefore, we undertook translation and cross-cultural adaptation of CPAx into “CPAx-Chi”, tested its reliability and validity, and verified the cutoff point for the diagnosis ICU-AW.

2.1. Ethical approval of the study protocol

The study protocol was approved by the Ethics Committee of the First Hospital of Lanzhou University (LDYYLL2019-232) in Lanzhou, China. This institution stated that written informed consent was not required.

2.2. Translation, cross-cultural adaptation and pre-testing

The translation of the original CPAx tool into Chinese was completed with the consent and assistance of the original author (EJ Corner). Translation, cross-cultural adaptation and pre-testing were done based on the model described by Brislin and colleagues^[16–17].

2.3. Translation and back-translation

Three bilingual authors with Chinese as their native language undertook the forward translation of CPAx from English to Chinese. One was a physician experienced within the specialty of critical illness; one was a nurse experienced within the specialty of critical illness; one was a graduate student in nursing with College English Test 6 certification unfamiliar with clinical medicine. A seminar was conducted to discuss and synthesize the results of the three translators. Different opinions were resolved through group consultation, and then integrated into the Chinese version of CPAx, which was named “CPAx-Chi-Forward”.

Three bilingual translators with English as their native language translated CPAx-Chi-Forward back into English. One was a doctoral student in nursing based in the UK; one was a doctoral student in physiotherapy based in Canada; one was a certified English linguist. They were unfamiliar with and blinded to the original CPAx version. A seminar was conducted to discuss and compare CPAx-Chi-Forward with the original CPAx. Discrepancies between the three translations were discussed until consensus was reached, and then the final synthesized back-translated English version was named “CPAx-Eng-Back”. The researchers provided a final report that included the annotations from translators about their rationale for translation, choices, and linguistic considerations to the author of the original CPAx.

2.4. Cross-cultural adaptation

Nine experts revised the items of CPAx-Chi-Forward based on their theoretical knowledge, practical experience, subjective feelings, and expression in the Chinese language. Two were specialists in critical care medicine, five were nursing specialists in critical care, one was a respiratory therapist, and one was a physiotherapist. During the process some words were rephrased or adjusted due to linguistic, grammatical, terminological or cultural differences between English and Chinese. Changes from the original CPAx version to the synthesized back-translated English version were discussed with and accepted by the original author.

2.5. Pre-testing

Forty ICU nurses from the First Hospital of Lanzhou University applied CPAx-Chi-Forward to assess ICU patients. Meanwhile, a dichotomous method was used to assess if the written expression in CPAx-Chi-Forward was easy to understand, clear, and based on Chinese expressions, and suggestions could be made. CPAx-Chi-Forward had good cross-cultural adaptation and there were no significant differences in sex, nationality, professional title, or time working in the ICU (p > 0.05 for all). Adjustments were not deemed necessary, and the final Chinese version of CPAx (CPAx-Chi) was created.

2.6. Verification of CPAx-Chi

2.6.1. Participants and sample size

Adult critically ill patients were recruited pragmatically from the general ICU of five third-grade class-A hospitals in western China from September 2019 to June 2020.

The inclusion criteria were: (i) critically ill and seriously ill patients eligible for ICU admission; (ii) age ≥ 18 years; (iii) duration of ICU stay ≥ 48 h; (iv) Glasgow Coma Scale (GCS) score ≥ 11; (v) volunteered to participate in our study.

The exclusion criteria were: (i) unstable fracture, limb deformity or limb dysfunction; (ii) myasthenia gravis or neuromuscular dysfunction.

2.6.2. Study design

This was a cross-sectional observational study, and the flowchart is shown in Fig. 1.. Two investigators simultaneously and independently assessed eligible patients using the MRC-Score and CPAx-Chi.

2.7. Statistical analyses

SPSS 22.0 (IBM, Armonk, NY, USA) was employed for statistical analyses. Frequency and percentages were used for dichotomous variables. The mean ± standard deviation was used for continuous variables. Content validity and criterion-related validity were employed to test the validity of CPAx-Chi. Cronbach’s ɑ coefficient and inter-rater reliability were used to test the reliability of CPAx-Chi. The MRC-Score was taken as the standard to calculate the receiver operating characteristic (ROC) curve and area under the ROC curve (AUC) of CPAx-Chi. The cutoff point of CPAx-Chi was determined by the maximum value of the Youden Index (YI). The kappa test was used to test the consistency of the MRC-Score and CPAx-Chi. p < 0.05 was considered significant.

3.1. Characteristics of participants

Two-hundred critically ill patients participated in this study (128 (64%) males and 72 (36%) females; mean age, 53.24 ± 15.06 years). The Acute Physiology and Chronic Health Evaluation score was 15.04 ± 6.70. The mean duration of ICU stay was 9.04 ± 6.15 days. The mean duration of hospital stay was 20.79 ± 11.84 days. The duration of mechanical ventilation was 3.55 ± 5.19 days. The principal diagnosis of participants was: craniocerebral injury (16, 8%), respiratory failure (22, 11%), surgical complications (68, 34%), hepatobiliary disease (42, 21%), cardiovascular disease (20, 10%), shock (14, 7%), and other (18, 9%). Also, 190 (95%) patients were transferred to other departments, two (1%) patients recovered, two (1%) patients were transferred to other hospitals, and six (3%) patients died.

3.2. Validity

3.2.1. Content validity

The nine specialists mentioned above were invited to evaluate the importance and applicability of items. The median age of specialists was 38 (interquartile range (IQR) 33–50) years. The median time the specialists had been working in the ICU was 13 (IQR 6–23) years. There were 9 specialists that included 1(11.11%) undergraduate, 4(44.44%) masters, and 4(44.44%)doctors; 4(44.44%) intermediate title and 5(55.55%)senior title.

The Content Validity Index of Items (I-CVI) was from 0.889 to 1. The Content Validity Index of Scale (S-CVI), which is the average of I-CVI, was 0.955. The Median Expert Authority Coefficient was 0.85 (IQR 0.75–0.95). The Kendall Synergy Coefficient was 0.061 (p = 0.842), and a significant difference was not detected in the degree of expert coordination.

3.2.2. Criterion validity

The correlation coefficient for ICU-AW assessment by Researcher A between the MRC-Score and CPAx-Chi was 0.60 (p < 0.001). The correlation coefficient for ICU-AW assessment by Researcher B between the MRC-Score and CPAx-Chi was 0.65 (p < 0.001) (Table 1).

Table 1

Criterion validity (n = 200)
Researcher	Criterion	Mean ± SD	r
A	CPAx-Chi	32.46 ± 8.83	0.60
A	MRC-Score	50.15 ± 10.42	0.60
B	CPAx- Chi	33.43 ± 9.08	0.65
B	MRC-Score	50.81 ± 10.50	0.65

3.3. Reliability

The internal consistency of CPAx-Chi was good (Cronbach’s α = 0.939). The correlation coefficient between Researcher A and Researcher B in the items of CPAx-Chi was between 0.668 and 0.992 (p < 0.001). The correlation coefficient between Researcher A and Researcher B in CPAx-Chi total score was 0.902 (p < 0.001) (Table 2).

Table 2

Inter-rater reliability (n = 200)
Physical parameter	Researcher	Mean ± SD	r	p
Respiratory function	A	3.62 ± 0.79	0.965	< 0.001
Respiratory function	B	3.64 ± 0.76	0.965	< 0.001
Cough	A	4.10 ± 0.99	0.715	< 0.001
Cough	B	4.21 ± 0.77	0.715	< 0.001
Moving within a bed (e.g., rolling)	A	4.06 ± 1.02	0.798	< 0.001
Moving within a bed (e.g., rolling)	B	4.08 ± 1.02	0.798	< 0.001
Supine to sitting on the bed edge	A	3.13 ± 1.06	0.766	< 0.001
Supine to sitting on the bed edge	B	3.24 ± 1.26	0.766	< 0.001
Dynamic sitting	A	3.69 ± 1.17	0.701	< 0.001
Dynamic sitting	B	3.66 ± 1.08	0.701	< 0.001
Standing balance	A	2.87 ± 1.15	0.766	< 0.001
Standing balance	B	1.97 ± 1.21	0.766	< 0.001
Sit to stand (starting position:≤90° hip flexion)	A	2.76 ± 1.11	0.763	< 0.001
Sit to stand (starting position:≤90° hip flexion)	B	2.67 ± 1.20	0.763	< 0.001
Transferring from bed to chair	A	2.61 ± 1.08	0.853	< 0.001
Transferring from bed to chair	B	2.94 ± 1.16	0.853	< 0.001
Stepping	A	1.95 ± 1.21	0.775	< 0.001
Stepping	B	2.33 ± 1.47	0.775	< 0.001
Grip strength	A	3.76 ± 1.35	0.992	< 0.001
Grip strength	B	3.76 ± 1.36	0.992	< 0.001
CPAx-Chi score	A	32.46 ± 8.83	0.902	< 0.001
CPAx-Chi score	B	33.59 ± 9.44	0.902	< 0.001

3.4. Best cutoff point for the diagnosis of ICU-AW using CPAx-Chi

The ROC curve for ICU-AW diagnosis with CPAX-Chi was drawn taking MRC-Score ≤ 48 as the standard for the diagnosis of ICU-AW. An MRC-Score ranging from 0 to 48 was termed “1” (ICU-AW group). An MRC-Score > 48 was termed “0” (non-ICU-AW group).

The AUC for Researcher A was 0.899 (95% confidence interval (CI) 0.862–1.025). The AUC for Researcher B was 0.874 (95%CI 0.824–0.925) (Fig. 2). The best cutoff point was determined by the maximum value of the YI. The maximum YI for Researcher A was 0.643, the cutoff point was 31.5, the sensitivity was 87%, and specificity was 77%. The maximum YI for Researcher B was 0.621, the cutoff point was 31.5, the sensitivity was 75%, and specificity was 87%.

3.5. MRC-Score and CPAx-Chi were consistent for the diagnosis of ICU-AW

We took 31 as the best cutoff point to diagnose ICU-AW using CPAx-Chi. Hence, if the total score of CPAx-Chi ranged from 0 to 31, it was marked as 1 (ICU-AW group), and if the total score of CPAx-Chi ranged from 32 to 50, it was marked as 0 (non-ICU-AW group). We found no significant difference in the total score of the ICU-AW group and non-ICU-AW group for Researcher A (F = 4.53, p = 0.035) or Researcher B (F = 6.51, p = 0.011). The test for consistency suggested that, taking CPAx-Chi ≤ 31.5 and MRC-Score ≤ 48 as the best cutoff points for the diagnosis of ICU-AW, then kappa was 0.845 (p = 0.02) for Researcher A, and kappa = 0.839 (p = 0.04) for Researcher B (Table 3) .

Table 3

MRC-Score and CPAx-Chi were consistent for the diagnosis of ICU-AW (n = 200)
Researcher	Scale	Group	MRC-score		Kappa	p
			ICU-AW (n)	Non-ICU-AW (n)
A	CPAx-Chi	ICU-AW	74	12	0.845	0.038
		Non-ICU-AW	3	111
B		ICU-AW	66	6	0.839	0.04
		Non-ICU-AW	9	119

4.1. Translation

The present study is the first to translate CPAx from English to Chinese using the Brislin model to guarantee sufficient equivalency^[16–17]. We not only included a multi-disciplinary committee to remedy content variance, but also included two Chinese nurses with English certifications studying, respectively, in the UK and Canada, as well as being native speakers of Chinese. In addition, we undertook tests for criterion validity and reliability for the completed translation.

4.2. Validity of CPAx-Chi

Validity is the degree that a measured result reflects the measured content. The more consistent the measured result is with the measured content, the higher is the validity^[18–19]. According to the guide of the scale compilation, when the number of experts is more than 5, the good standard of I-CVI is more than 0.78, and the experts must be authoritative and coordinated^[19–20].

The present study involved nine ICU multidisciplinary experts with deep theoretical knowledge and clinical experience. The Expert Authority Coefficient ranged from 0.75 to 0.95. The Kendall Synergy Coefficient was 0.061(p = 0.842) and I-CVI ranged from 0.889 to 1. Therefore, CPAx-Chi had good content validity^[21–22].

Corner and colleagues demonstrated that the CVI of CPAx was 1 (p < 0.05)^[11–12]. They also showed that CPAx has good predictive validity, and that the CPAx score could be used as an alternative indicator of functional prognosis in critically ill patients by analyzing the relationship between the CPAx score and patient outcomes^[13]. Other colleagues demonstrated the criterion validity of CPAx taking the scores for the MRC, Short Form (SF)-36, Sequential Organ Failure Assessment (SOFA), and GCS as a standard^[23]. They found that the correlation coefficient between the CPAx score and MRC-Score was 0.65 (p < 0.001). The correlation coefficient for the right upper limb, left upper limb, right lower limb, and left lower limb with the CPAx score was, respectively, 0.69, 0.64, 0.69 and 0.67. The correlation coefficient between the CPAx score and SOFA score was 0.68 (p < 0.001). The correlation coefficient between the CPAx score and GCS was 0.74 (p < 0.001). The correlation coefficient between the physical-function item of SF-36 and the CPAx score was 0.72 (p = 0.013). The correlation coefficient between the mental-function component of SF-36 and the CPAx score was 0.024 (p = 0.95). In the present study, the correlation coefficient between the CPAx-Chi score and the items of the MRC-Score ranged from 0.60 to 0.65 (p < 0.001). Therefore, CPAx-Chi had good validity.

4.3. Reliability of CPAx-Chi

Cronbach’s α mainly reflects the internal consistency of a scale^[18–19]. In general, Cronbach's α should be > 0.7; a value < 0.6 indicates that the items of scale must be revised. From the perspective of psychometrics, the “ideal” Cronbach’s α should be > 0.8^[24–26]. The inter-rater reliability mainly demonstrates the consistency of evaluation results among different evaluators, and the stability of scales used among different evaluators^[27–28]. An inter-rater correlation coefficient > 0.7 indicates that the inter-rater reliability is good. The inter-rater correlation coefficient ranging from 0.8 to 0.9 indicates that the inter-rater reliability is high^{[14, 18–20, 28]}. In the present study, Cronbach’s ɑ for CPAx-Chi was 0.939, and the inter-rater reliability of the CPAx-Chi score was 0.902 (p < 0.001). The inter-rater correlation coefficient was > 0.8 for the items of respiratory function, transfer from bed to chair, and grip strength. The inter-rater correlation coefficient of other items of CPAx-Chi were all > 0.7. Therefore, CPAx-Chi had good reliability.

4.4. Best cutoff point, sensitivity and specificity of CPAx-Chi

Typically, evaluation of diagnostic performance is based on the ROC curve and AUC. If the AUC of a certain scale is 1, then it is considered to be a “perfect” diagnostic tool, but the perfect tool does not exist in the real world. Hence, if the AUC of one scale ranges from 0.85 to 0.95, then the measurement effect of the scale is very good. If the AUC of one scale ranges from 0.5 to 0.7, then the measurement effect of the scale is considered to be undesirable. If the AUC of one scale is 0.5, then the measurement effect of the scale is barely functional^[29–31]. Our experts regarded an MRC-Score ≤ 48 as the standard to diagnose ICU-AW. First, some studies have demonstrated the value of diagnostic ICU-AW using the Barthel Index^[32], grip strength^[33], ICU Mobility Scale^[34], de Morton Mobility Index^[35], and the Physical Function Intensive Care Test^[36] using MRC-Score ≤ 48 as the standard. Second, the best cutoff point, sensitivity and specificity of neuromuscular ultrasound, electrophysiological recordings, electromyography, and other objective diagnostic methods used to diagnose ICU-AW have been verified using MRC-Score ≤ 48 as the criterion^[23,37−39]. Third, scholars have constructed several models of early prediction of ICU-AW by taking MRC-Score ≤ 48 as a diagnostic criterion^[40–42]. In the present study, the best cutoff point for the diagnosis of ICU-AW with CPAx-Chi was 31 points. This was verified by taking MRC-Score ≤ 48 as the criterion, and the sensitivity and specificity were good.

The kappa statistic quantifies inter-rater reliability for ordinal and nominal measures. In general, a kappa value between 0.40 and 0.60 indicates “moderate” agreement, 0.61 and 0.80 denotes “substantial” agreement, and > 0.81 reflects “excellent” agreement; a negative value for kappa represents disagreement^[43–44]. The concordance of the kappa value was high when taking the MRC-Score ≤ 48 and CPAx-Chi ≤ 31 as the best cutoff points to diagnose ICU-AW for Researcher A and Researcher B.

4.5. Strengths of our study

First, two researchers assessed and collected data independently, which improved the reference value of the validation data. Second, the best cutoff point for the diagnosis of ICU-AW using CPAx-Chi was determined to be 31 points according MRC-Score ≤ 48.

4.6. Weaknesses of our study

First, our findings were limited by use of a non-randomized pool of participants chosen primarily by their availability during the study period: this may have reduced the generalizability of our findings. Second, there were specific exclusion criteria that may have stopped the potential “ceiling and floor” effects of CPAx-Chi to be tested. Therefore, to further confirm the clinical value of CPAx in assessing and diagnosing ICU-AW, it must be applied together with the MRC-Score, ultrasound, electrophysiology, and electromyography. Also, multicenter, large-sample, and randomized trials are needed to verify the best cutoff point for CPAx.

We showed that CPAx-Chi had high criterion validity and reliability for assessing ICU-AW in adult patients in the ICU. CPAx-Chi showed good sensitivity and specificity in assessment of patients at risk of ICU-AW at a recommended cutoff of 31 points.

Acknowledgements

The authors gratefully acknowledge all the participants.

Funding

This study was based on a project supported by The Natural Science Foundation of Gansu Province (17JR5RA266). The funding source had no involvement in data collection/analyses, or manuscript preparation.

Conflict of interest

The authors have declared that no competing interests exist

Consent for publication

Manuscript is approved by all authors for publication

Ethical Approval and Consent to participate

Author contributions

ZGZ and YCW conceived of the study, and participated in its design and coordination and helped to draft the manuscript. GQW, HPW and BTJ performed the experiment and investigation. NND and JHT participated in the design of the study and performed the statistical analysis. BL participated in the project administration. WGY and JHT participated in the manuscript editing and review. All authors read and approved the final manuscript.

Data Availability Statement

Data can be requested from the Ethics Committee(contact via the First Hospital of Lanzhou University, Lanzhou, Gansu, China, email: [email protected]) for researchers who meet the criteria for access to confidential data.

Piva S, Fagoni N, Latronico N. Intensive care unit-acquired weakness: unanswered questions and targets for future research[J]. F1000Research. 2019;8. DOI:10.12688/f1000research. 17376.1
Farhan H, Moreno-Duarte I, Latronico N, et al. Acquired Muscle Weakness in the Surgical Intensive Care Unit: Nosology, Epidemiology, Diagnosis, and Prevention[J]. Anesthesiology. 2016;124(1):207-234. DOI: 10.1097/ALN.0000000000000874.
Qiu Y, Jiang L, Xi X. Early incidence and prognosis of ICU-acquired weakness in mechanical ventilation patients[J]. Chinese critical care medicine. 2019;31(7):821-826. DOI: 10.3760/cma. j.issn. 2095-4352.2019.07.005
Diaz Ballve L P, Dargains N, Urrutia Inchaustegui J G, et al. Weakness acquired in the intensive care unit. Incidence, risk factors and their association with inspiratory weakness. Observational cohort study[J]. Rev Bras Ter Intensiva, 2017, 29(4): 466-475. DOI: 10.5935 /0103-507X.20170063.
Appleton R T, Kinsella J, Quasim T. The incidence of Intensive care unit-acquired weakness syndromes: A systematic review[J]. J Intensive Care Soc, 2014,16(2):126-136. DOI: 10.1177/1751143714563016.
Shepherd SJ, Newman R, Brett SJ, et al. Pharmacological Therapy for the Prevention and Treatment of Weakness After Critical Illness: A Systematic Review[J]. Critical care medicine. 2016;44(6):1198-1205. DOI: 10.1097/CCM.0000000000001652.
Fan E, Cheek F, Chlan L, et al. An official American Thoracic Society clinical practice guideline: the diagnosis of intensive care unit-acquired weakness in adults [J]. Am J Respir Crit Care Med, 2014, 190 (12): 1437-1446. DOI: 10.1164/rccm.201411-2011ST.
Saccheri C, Morawiec E, Delemazure J, et al. ICU-acquired weakness, diaphragm dysfunction and long-term outcomes of critically ill patients[J]. Annals of intensive care. 2020;10(1):1. DOI:10.1186/s13613-019-0618-4.
Jung B, Moury PH, Mahul M, et al. Diaphragmatic dysfunction in patients with ICU-acquired weakness and its impact on extubation failure[J]. Intensive care medicine. 2016;42(5):853-61. DOI:10.1007/s00134-015-4125-2
Supinski GS, Morris PE, Dhar S, et al. Diaphragm Dysfunction in Critical Illness[J]. Chest. 2018;153(4):1040-51.DOI:10.1016/j.chest.2017.08.1157
Corner E J, Wood H, Englebretsen C, et al. The Chelsea Critical Care Physical Assessment Tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study[J]. Physiotherapy, 2013, 99(1): 33-41.DOI: 10.1016/j.physio.2012.01.003
Corner E J , Soni N , Handy JM , et al. Construct validity of the Chelsea critical care physical assessment tool: an observational study of recovery from critical illness[J]. Critical Care, 2014, 18(2):R55.DOI: 10.1186/cc1380
Corner EJ, Hichens LV, Attrill KM, et al. The responsiveness of the Chelsea Critical Care Physical Assessment tool in measuring functional recovery in the burns critical care population: an observational study[J]. Burns. 2015,41(2):241-247. DOI: 10.1016/j.burns. 2014.12.002.
Holdar U, Eriksson F, Siesage K, et al. Cross-cultural adaptation and inter-rater reliability of the Swedish version of the Chelsea critical care assessment tool (CPAX-Swe) in critically ill patients[J]. Disabil Rehabil. 2019:1-5. DOI: 10.1080/09638288.2019.1668971.
Astrup K, Corner EJ, Hansen MG, et al. Translation and cross-cultural adaptation of the Chelsea Critical Care Physical Assessment tool into Danish[J]. Physiother Theory Pract. 2018:1-8. DOI:10.1080/09593985.2018.1548048.
Valmi D S, Wilaiporn R, Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guidline[J]. J Eval Clin Pract, 2011,17(2):268-274
Abu Sharour L. Translation and Validation of the Arabic Version of the Caring Nurse-Patient Interaction Scale-Patient Version (CNPI-23P). Cancer Nurs. 2019. DOI:10.1097/NCC.0000000000000755
Souza AC, Alexandre NMC, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade[J]. Epidemiol Serv Saude. 2017;26(3):649-659. doi:10.5123/S1679-49742017000300022
Post MW. What to Do With "Moderate" Reliability and Validity Coefficients? [J]. Arch Phys Med Rehabil. 2016;97(7):1051-1052. doi:10.1016/j.apmr.2016.04.001
Yushen Ren. Reliability and Validity of Chinese Version of PURPOSE-T and the preliminary application in hospitalized patients [D].Tianjin Medical University,2019. DOI:10.27366/d.cnki.gtyku.2019.000681
Bentley TGK, Cohen JT, Elkin EB, et al. Validity and Reliability of Value Assessment Frameworks for New Cancer Drugs[J]. Value Health. 2017;20(2):200-205. doi:10.1016/j.jval.2016.12.011
Chen AC, Lee MS, Chen WJ, Lee ST. Assessment in orthopedic training-an analysis of rating consistency by using an objective structured examination video[J]. J Surg Educ. 2013;70(2):189-192. doi:10.1016/j.jsurg.2012.11.002
Wu Yuchen, Ding Nannan, Jiang Biantong, et al. Diagnostic tools of intensive care unit acquired weakness: a systematic review[J]. Chin Crit Care Med.2018,30(12):1154-1160. DOI：3760/cma.j.issn.2095-4352.2018.12.011
Tavakol M. The reliability of assessments: The Bayesian Cronbach's alpha[J]. Med Teach. 2017;39(5):561. doi:10.1080/0142159X.2017.1296121
Trizano-Hermosilla I, Alvarado JM. Best Alternatives to Cronbach's Alpha Reliability in Realistic Conditions: Congeneric and Asymmetrical Measurements[J]. Front Psychol. 2016;7:769. Published 2016 May 26. doi:10.3389/fpsyg.2016.00769
de Vet HCW, Mokkink LB, Mosmuller DG, Terwee CB. Spearman-Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications[J]. J Clin Epidemiol. 2017;85:45-49. doi:10.1016/j.jclinepi.2017.01.013 []
Dupépé EB, Davis M, Elsayed GA, et al. Inter-rater reliability of the modified Medical Research Council scale in patients with chronic incomplete spinal cord injury[J]. J Neurosurg Spine. 2019;1-5. doi:10.3171/2018.9.SPINE18508
Maidan I, Freedman T, Tzemah R, Giladi N, Mirelman A, Hausdorff JM. Introducing a new definition of a near fall: intra-rater and inter-rater reliability[J]. Gait Posture. 2014;39(1):645-647. doi:10.1016/j.gaitpost.2013.07.123
Alex J. Bowers, Xiaoliang Zhou. Receiver Operating Characteristic (ROC) Area Under the Curve (AUC): A Diagnostic Measure for Evaluating the Accuracy of Predictors of Education Outcomes[J]. Journal of Education for Students Placed at Risk (JESPAR),2019,24(1). DOI:10. 1080/10824669.2018.1523734
Janssens A, Cecile J W, Martens Forike K. Reflection on modern methods: Revisiting the area under the ROC Curve.[J]. International journal of epidemiology, 2020.pii: dyz274. DOI: 10.1093/ije/dyz274.
Ke BS, Chiang AJ, Chang YI. Influence Analysis for the Area Under the Receiver Operating Characteristic Curve[J]. J Biopharm Stat. 2018;28(4):722-734. doi:10.1080/10543406.2017. 1377728
Kawahara K, Suzuki T, Yasaka T, et al. Evaluation of the site specificity of acute disuse muscle atrophy developed during a relatively short period in critically ill patients according to the activities of daily living level: A prospective observational study[J]. Australian critical care : official journal of the Confederation of Australian Critical Care Nurses. 2017, 30(1): 29-36. DOI:10.1016/j.aucc.2016.01.003
Cottereau G, Dres M, Avenel A, et al. Handgrip Strength Predicts Difficult Weaning But Not Extubation Failure in Mechanically Ventilated Subjects[J]. Respiratory care. 2015,60(8): 1097-1104. DOI:10.4187/respcare.03604.
Tipping CJ, Bailey MJ, Bellomo R, et al. The ICU mobility scale has construct and predictive validity and is responsive. A multicenter observational study [J]. Ann Am Thorac Soc, 2016, 13 (6): 887-893.DOI: 10.1513/AnnalsATS.201510-717OC.
Sommers J, Vredeveld T, Lindeboom R, et al. de Morton mobility index is feasible, reliable, and valid in patients with critical illness [J]. Phys Ther, 2016, 96 (10): 1658-1666. DOI: 10.2522/ptj.20150339.
Nordon-Craft A, Schenkman M, Edbrooke L, et al. The physical function intensive care test: implementation in survivors of critical illness [J]. Phys Ther, 2014, 94 (10): 1499-1507. DOI:10.2522/ptj.20130451.
Witteveen E, Sommers J, Wieske L, et al. Diagnostic accuracy of quantitative neuromuscular ultrasound for the diagnosis of intensive care unit-acquired weakness: a cross-sectional observational study [J]. Ann Intensive Care, 2017, 7 (1): 40. DOI: 10.1186/s13613-017- 0263-8.
Mitobe Y, Morishita S, Ohashi K, et al. Skeletal Muscle Index at Intensive Care Unit Admission Is a Predictor of Intensive Care Unit-Acquired Weakness in Patients With Sepsis[J]. Journal of clinical medicine research. 2019;11(12):834-841. DOI:10.14740/jocmr 4027.
Wieske L, Verhamme C, Witteveen E, et al. Feasibility and diagnostic accuracy of early electrophysiological recordings for ICU-acquired weakness: an observational cohort study [J]. Neurocrit Care, 2015, 22 (3): 385-394. DOI: 10.1007/s12028-014-0066-9.
Tzanis G, Vasileiadis I, Zervakis D, et al. Maximum inspiratory pressure, a surrogate parameter for the assessment of ICU-acquired weakness[J]. BMC Anesthesiol, 2011(11):14. DOI: 10.1186/1471-2253-11-14.
Peñuelas O, Muriel A, Frutosvivar F, et al. Prediction and outcome of Intensive Care Unit- Acquired Paresis. J Intensive Care Med, 2018, 33(1):16-28. DOI:1177/0885066616643529.
Witteveen E, Wieske L, Sommers J, et al. Early prediction of intensive care unit-acquired weakness: a multicenter external validation study[J]. Journal of Intensive Care Medicine, 2018:885066618771001. DOI:10.1177/0885066618771001
Yibin Guo, Wei Guo, Yuchen Qin, et al. Consistency test based on Kappa coefficient and its software implementation [J]. Chinese Journal of Health Statistics, 2016, v.33(01):169-170.
Vanbelle S. A New Interpretation of the Weighted Kappa Coefficients[J]. Psychometrika. 2016;81(2):399-410. doi:10.1007/s11336-014-9439-4

Download PDF

Version 1

posted

You are reading this latest preprint version

Reliability, Validity, and Cutoff Point of the Chinese Version of the Chelsea Critical Care Physical Assessment Tool in Critically Ill Patients

Status:

Version 1

Abstract

Figures

1. Introduction

2. Materials And Methods

2.1. Ethical approval of the study protocol

2.2. Translation, cross-cultural adaptation and pre-testing

2.3. Translation and back-translation

2.4. Cross-cultural adaptation

2.5. Pre-testing

2.6. Verification of CPAx-Chi

2.6.1. Participants and sample size

2.6.2. Study design

2.7. Statistical analyses

3. Results

3.1. Characteristics of participants

3.2. Validity

3.2.1. Content validity

3.2.2. Criterion validity

3.3. Reliability

3.4. Best cutoff point for the diagnosis of ICU-AW using CPAx-Chi

3.5. MRC-Score and CPAx-Chi were consistent for the diagnosis of ICU-AW

4. Discussion

4.1. Translation

4.2. Validity of CPAx-Chi

4.3. Reliability of CPAx-Chi

4.4. Best cutoff point, sensitivity and specificity of CPAx-Chi

4.5. Strengths of our study

4.6. Weaknesses of our study

5. Conclusions

Declarations

References

Status:

Version 1