Our predefined search strategy identified 3361 titles; none were identified through searching reference lists of published systematic reviews, and review of included study reference lists identified 1 more. The screening process and reasons for exclusion at each step are presented in Figure 1.
We identified 6 studies describing 6 prediction models that that met our inclusion criteria, which included 7075 patients.[15-20] Four studies were from North America, one from Australia, and one from Europe. Four included general medicine patients and two included specific subpopulations that could be cared for on a general medical ward (stroke patients and patients with advanced cirrhosis) (Table 1).
The mean age of patients in the derivation cohorts ranged from 55 to 83 years.
Risk of Bias
The risk of bias for included studies was moderate to high (Table 2).
Inadequate description of the source population’s inclusion and exclusion criteria was found in 5 of 6 included studies.
Prediction Models
Four studies used univariable analysis to screen for predictors to be used for modeling while the other two models used clinical reasoning to select predictor variables. To build the final model, five prediction models used various selection algorithms while one included all the variables that were hypothesized to be predictive (Table 3).
Four models used predictors that were available with in the first 24 hours of hospital admission while two models used variables collected throughout the hospital stay. All prediction models had a binary outcome as the dependent variable, however, each study defined outcome differently (Table 3). Four models defined the outcome as a place (home, care facility, or rehabilitation hospital) while two models defined the outcome as the need for support services after discharge. Model discrimination was generally good (range derivation C-statistics 0.75 – 0.85) and similar but slightly lower in the validation cohorts (range validation C-statistics 0.70 – 0.80) with one study not reporting discrimination statistics. Two models tested calibration with the Hosmer-Lemeshow goodness of fit test and found no evidence of poor fit. Other calibration metrics, such as calibration plots, were not reported.
Predictors
All variables associated with the need for post discharge supportive services in predictive models are presented in Table 3. Many variables were associated with the outcome in univariable analysis in a single study but not in multivariable analyses. Variables present in 2 or more studies were assessed for the strength of evidence using the GRADE tool (Table 4).
There is high quality evidence that age, impaired physical function, ADL disabilities, and frailty increase the probability of needing supportive services after discharge. There is moderate evidence that a diagnosis of stroke, IADL disabilities and receiving supportive services prior to hospital admission increase the probability of needing supportive services after discharge. Interestingly a greater number of comorbidities was a significant predictor in univariable analyses in 3 studies but not in any multivariable analyses. Indicators of support at home (marital status and living with someone) had moderate-weak evidence of predicting discharge without supportive services. Lastly several specific diagnoses (heart failure, metastatic cancer and heart valve disease) had weak to very week evidence for predicting the need for supportive services.
Causal Pathway
We constructed a causal pathway to explain the relationships between the predictor variables and the outcome (Figure 2). Activities of daily living andcare giver at home directly affected the need for supportive services after discharge whereas age and comorbidities act by their effects on physical function and cognitive function to affect ADL disabilities. Stroke directly causes ADL and IADL disabilities while the causative impact of other specific diagnoses is less certain.
Discussion
Our systematic review identified 6 validated models that predict the need for supportive services after discharge from a non-elective medical admission. All models had good discrimination, although there was at least a moderate risk of bias for all studies and no models were externally validated. Furthermore, other important model characteristics, such as calibration that indicates if the predicted risk is similar to the actual risk, were not reported for most models. Our GRADE analysis of identified predictors suggests that age, impaired physical function, ADL disabilities and frailty all have high quality evidence as factors that predict the need for supportive services after discharge. These factors should be included in future studies to derive, validate and evaluate models to predict discharge care needs to support more efficient and high-quality care.
Models
Our review included far fewer studies than other recent reviews because we excluded studies that performed no model validation. Validation is a crucial step in model development as it assesses whether the model accurately represents the real world.[21] This is especially important when there are many potential predictor variables and few outcomes in the derivation dataset. In these circumstances a relationship between a predictor and the outcome may be present by chance.[22] Despite limiting our inclusion criteria to studies with validated models, several methodologic concerns remained. Only 1 model used the recommended best practice of using clinical reasoning to guide variable selection and final model building.[23] The rest of the models used univariable analysis to select variables and then an automated selection algorithm for model building. This process can result in inaccurate or false relationships between predictors and the outcome by accidentally conditioning on a collider.[24] Any model that seeks to explain the causes of an outcome needs to start with a clinical reasoning or a formal causal hypothesis so that appropriate potential causal factors and confounders can be included, thereby avoiding false or inaccurate effect estimates. Making causal hypotheses explicit is critical to furthering the field and allowing critical appraisal by peers.
Predictors
Consistent with previous studies, we found that impairments in physical function and ADL disabilities are strong predictors of the need for supportive services after discharge.[25, 26] This is intuitive because support for ADLs is the primary service offered by retirement homes or long-term care facilities. Conversely, inability to perform ADLs necessitates supportive services. Numerous medical diagnoses, illness severity scores, and comorbidity scores were associated with need for supportive services after discharge in univariable analysis but most of them were excluded in variable selection processes. This likely occurred because comorbidities increase the need for services after discharge primarily by their impact on physical and cognitive function as illustrated in our causal pathway (Figure 2). Physical and cognitive function mediate the effect of comorbidities on the need for supportive services. This is an example of how important and potentially modifiable predictors may be removed from a multivariable model when a downstream mediator is included.
Marital status and living with someone were both negatively associated with the need for supportive services after discharge. Clearly, an informal caregiver may reduce the need for formal caregiving by assisting with ADLs and IADLs.[27] Informal caregivers may also impact the need for supportive services by providing emotional support and reducing loneliness.[28] Several studies in our review found a univariable association between measures of mental health and need for supportive services but the associations were not significant in final multivariate models.
None of the included studies tested income or financial resources as predictors. The omission is likely pragmatic but considering the importance of these variables for other health outcomes we suggest that the predictive contribution of financial resources or other proxies of socioeconomic status should be considered in future derivation of models predicting need for supportive services.[29] Financial resources may not be a strong predictor in countries with publicly funded healthcare.
Strengths and limitations
Our study followed systematic review methods outlined by the Cochrane collaboration with a comprehensive search strategy and screening, data extraction, and quality appraisal performed in duplicate. Excluding non-validated models drastically reduced the number of included studies but these exclusion criteria allowed us to evaluate higher quality models more robustly. A limitation in our results is the lack of any externally validated models and the risk of bias of included studies. This precluded us from identifying whether predictors were highly influenced by model overfitting in the original study’s dataset. External validation is necessary before using a model in clinical practice and its absence demonstrates the need for further work in this field. We were also unable to perform meta-analysis as each study specified a slightly different outcome variable.