Implementation, Spread and Impact of The Patient Oriented Discharge Summary (PODS) Across Ontario Hospitals: A Mixed Methods Evaluation

doi:10.21203/rs.3.rs-87036/v1

Download PDF

Research article

Implementation, Spread and Impact of The Patient Oriented Discharge Summary (PODS) Across Ontario Hospitals: A Mixed Methods Evaluation

https://doi.org/10.21203/rs.3.rs-87036/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 17 Apr, 2021

Read the published version in BMC Health Services Research →

You are reading this latest preprint version

BACKGROUND: Traditional discharge processes lack a patient-centred focus. This project studied the implementation and effectiveness of an individualized discharge tool across Ontario hospitals. The Patient Oriented Discharge Summary (PODS) is an individualized discharge tool with guidelines that was co-designed with patients and families to enable a patient-centred process.

METHODS: Twenty one acute-care and rehabilitation hospitals in Ontario, Canada engaged in a supported community of practice and worked over a period of 18 months to implement PODS. An effectiveness-implementation hybrid design using a triangulation approach was used with hospital-collected data, patient and provider surveys, and interviews of project teams. Key outcomes included: reach within and spread outside target populations, change in patient-centred processes, patient and provider experience, understanding of discharge instructions, and healthcare utilization. Statistical methods included linear mixed effects models and generalized estimating equations.

RESULTS: Of 65,221 discharges across hospitals, 41,884 patients (64%) received a PODS. There was variation in reach and implementation pattern between sites, though none of the between site covariates was significantly associated with implementation success. Both high participation in the community of practice and scoring highly on quality of PODS were associated with higher implementation success. PODS improved family involvement during discharge teaching (7% increase, p=0.026), use of teach-back (11% increase, p<0.001) and discussion of help needed (6% increase, p=0.041). Although unscheduled healthcare utilization decreased with PODS implementation, it was not statistically significant.

CONCLUSIONS: This project highlighted the system-wide adaptability and ease of implementing PODS across multiple patient groups and hospital settings. PODS demonstrated an improvement in patient-centred discharge processes linked to quality standards and health outcomes. A community of practice and high quality content may be needed for successful implementation.

Health Economics & Outcomes Research

Health Policy

Implementation

Discharge

Patient Centred

Transitions in Care

Local Adaptability

triangulation

Patient Experience

Quality Improvement

Hospital

Traditional discharge summaries are laden with clinical information meant for the primary care provider. Instructions for patients are often not patient-centered or not included.[1] Moreover, while approximately 80–90% of hospitals in Ontario have a discharge planning process, previous reports suggest it is not standardized throughout the province, where involvement of the patient in this process and education provided is not consistent.[2] Patient-centered discharge tools, unlike traditional discharge summaries, engage the patient and family during the hospital admission with an individualized discharge plan and are associated with improved patient outcomes.[3–6] While improving care transitions is a focus of health systems globally, no Canadian tools exist to our knowledge. Very few tools from other countries have been co-designed with patients and families to address barriers to understanding and use of their discharge instructions.[7, 8]

The Patient Oriented Discharge Summary (PODS) is an individualized discharge tool designed with patient and family engagement that contains five sections of information that are actionable and useful for patients and families; 1) medications, 2) changes to daily activities and diet, 3) follow-up appointments, 4) resources for patients and families, and 5) expected and worrisome symptoms to watch out for after leaving hospital (Fig. 1).[1] PODS utilises design features such as plain language, large fonts, pictograms, and white space for patients to take notes that enhance retention and understanding of information.[9] PODS includes accompanying process guidelines which highlight key patient-centered processes proven to increase adherence to instructions.[10, 11] PODS is designed to be modular. Every organization that implements it is tasked with adapting the tool and guidelines to meet the needs of their patients and internal processes. Hospitals differ in who completes and delivers the PODS, when it gets completed and delivered, and whether it is built into the electronic health record. For example, some hospitals have nurses responsible for completing and delivering the tool the day before discharge, others have patients completing their own tool over the course of their stay with the support of staff, and others have a multidisciplinary team completing sections of the tool that are then combined electronically.

A pilot study in Toronto demonstrated that PODS was both usable and feasible and suggested improvements in patient understanding of discharge instructions and patient experience.[12] In the present work, we aimed to understand 1) implementation of PODS in different settings and factors influencing implementation success and 2) the impact of PODS on patient-centred discharge processes, patient and provider experience, and 30-day emergency department (ED) visits and readmissions.

Setting and context

Ontario is Canada’s largest province with a population of more than 14 million people and 123 acute care and 58 rehabilitation hospitals, spread across 14 Local Health Integration Networks (LHINs), designated geographical areas. The hospitals are commonly grouped as being rehabilitation or acute care and as being academic, community, or small hospitals.[13, 14] This project was the result of an Accelerated Research to Improve Care (ARTIC) grant, government funding for province-wide quality improvement (QI) implementation of evidence based initiatives.[15] Following the consolidated framework for implementation research (CFIR),[16] factors known to facilitate successful implementation were built into both the processes of identifying participating hospitals and supporting the implementation. As part of the project application process, hospitals were selected based on the presence of an executive sponsor and a clear rationale for implementing PODS. Organizations also required previous quality improvement experience and sufficient capacity for implementation. Hospitals were chosen from all 14 LHINs and from within all hospital categories to be representative of the province as a whole.

Of 42 hospitals that applied, 21 were invited to participate in a community of practice (COP) and received a stipend to support their implementation and data collection. The hospitals involved all had a high level of organizational commitment and readiness, but ranged widely in size, geographic area, target patient population, discharge process (i.e. what members of the healthcare team were involved in providing patient education at discharge), whether PODS was implemented in isolation or as part of broader discharge process improvements, and whether the process was supported through the electronic medical record (EMR). There were eighteen acute care and three rehabilitation hospitals, eight which were considered academic hospitals, five large community hospitals and eight small community hospitals with under 100 beds (Table 1).

Table 1

Description of target populations in each hospital implementing PODS (n = 21 hospitals).
ID	Type	Size	Target population	Rationale for implementing PODS	Main Responsible Provider	When is it done
1	A	A	Mental health inpatients	Poor existing process	Social work or nurse	Week of discharge
2	A	A	all inpatients	quality and consistency	Physician	week of discharge
3	A	A	all medicine acute and sub-acute	quality and consistency	Physician	week of discharge
4	A	A	all inpatients	general patient centred care	Multidisciplinary	week of discharge
5	A	A	medicine, chronic disease, oncology, surgery	general patient centred care	Nurse	day before discharge
6	A	A	Medicine - focus on elderly	general patient centred care	Team	day of discharge
7	A	C	all surgery	general patient centred care	Nurse	over whole stay
8	A	C	mental health ED and inpatient followed by rehab	reduce readmissions	nurse	week of discharge
9	A	C	all inpatients and ED (QBPs at first)	poor existing process	nurse	day before discharge
10	A	C	Surgery	poor existing process	nurse	day before discharge
11	A	S	medicine, surgery, and rehab	poor existing process	nurse	day before discharge
12	A	S	medicine, surgery, and ED	poor existing process	nurse	week of discharge
13	A	S	all inpatient (target CHF, COPD, and Stroke at first)	reduce readmissions	nurse	week of discharge
14	A	S	all inpatients and ED	general patient centred care	nurse	day before discharge
15	A	S	all inpatients	quality and consistency	nurse	week of discharge
16	A	S	all geriatric inpatients	general patient centred care	nurse	week of discharge
17	A	S	medicine, surgery, and obstetrics	poor existing process	nurse	day before discharge
18	A	S	medicine, surgery, obstetrics	quality and consistency	nurse	week of discharge
19	R	A	all inpatients (rehab includes stroke)	poor existing process	social work or nurse	over whole stay
20	R	A	inpatient Stroke	quality and consistency	nurse	over whole stay
21	R	C	Rehab including stroke	General patient centred care	nurse	Week of discharge
Type: A = acute, R = rehab; Size: A = academic, C = community, S = small

Our team supported the implementation of PODS at the 21 hospitals in three stages: (1) Start Up: hospitals were guided through the process of adapting the PODS tool and process together with stakeholders; (2) Plan Do Study Act (PDSA): hospitals “went live” with either a pilot group of their full target group and then iteratively tested, refined, and evaluated the tool and process while implementing; and (3) Scale Up: hospitals spread the use of the tool and mentored others who were interested. Using a supported community of practice model, our team hosted regular meetings for education, shared knowledge and mentorship. A website was developed to house central resources and collective knowledge.[17] Project teams were encouraged to engage with patients and families throughout the project. A project advisory group was formed to guide the project with representation from the hospital, community, and patients including authors (SHG, TH, AC, HA, CB, KO). The group provided advice on project implementation, evaluation, and interpretation of results.

The project started in April 2017, with hospitals beginning implementation of PODS between April 2017 and March 2018. Majority of hospitals “went live” in October and November of 2017. Evaluation data was collected between April 2017 and December 2018 and spanned all three stages of start up, when hospitals had yet to discharge any patients with the PODS, and PDSA and scale up, when hospitals expanded the number of patients receiving the intervention.

Study design

Due to variation in adaptation and implementation across organizations, we chose to use an effectiveness-implementation hybrid design that uses both quantitative and qualitative methods for evaluating complex interventions.[18] Specifically, we employed a hybrid design that tested implementation while also collecting information on the intervention and related outcomes.

Data collection and analysis

Implementation was studied by examining the percentage of the target population reached, the pattern and quality of implementation, and spread of implementation outside of the target group. Individual, organizational, and system factors impacting implementation were also studied. Effectiveness was studied through patient-centred processes, patient understanding, patient and provider experience, and unscheduled healthcare resource utilization.

Quantitative

Implementation data was collected through quarterly submissions of implementation reach within the target population, and spread outside of the target population reported by hospitals. Additionally, during end of project interviews, project teams were asked to rate the quality of their implementation using measures of consistency, completeness, quality of the content of their PODS, and quality of their process of providing PODS to patients. Factors impacting implementation were collected through demographic data on the target populations reported quarterly by hospitals and through information about the organization and their discharge process collected during end-of-project interviews with each project team (see Additional File 1 for the interview guide). Factors were chosen based on whether they were thought of as important by the project advisory team, whether they were present in the literature on implementation success, or whether they were found as a theme that emerged through qualitative data collected throughout the project.

Effectiveness data was collected through provider and patient/family surveys developed for this project (see Additional File 1), as well as hospital-reported data. Providers involved in the discharge process as well as a random sample of patients and families in the target population completed hospital-administered surveys pre- and post-implementation. Provider survey measures included the use of teach-back methods and involvement of family during discharge education, in order to see if using PODS impacted important and evidence-based patient-centred processes. Additionally, providers reported whether the intervention added to workload and whether the intervention added value to the discharge experience, chosen as balancing measures that might impact success and sustainability of the implementation. Patient and family survey measures included whether the patient had a discussion in hospital about the help they would need at home, a rating of understanding of medications, a rating of understanding of what to do if they are worried about their condition, and whether the intervention added value to the discharge experience. These questions were chosen because they are relevant questions related to transitions in care found in the Canadian Institute for Health Information Patient Experience Survey for Inpatient Care, a validated survey used at many Canadian hospitals [28]. Quarterly submissions from hospitals included data on all-cause 30-day return ED visits and readmissions for the target population.

Qualitative

Qualitative data was collected throughout the project through surveys and presentations at community of practice meetings and open-ended questions in both provider and patient and family surveys. The themes that emerged through the data were used to create a semi-structured interview guide developed specifically for this project (see Additional File 1) that was then used to conduct end-of-project interviews over the phone with project teams from each of the 21 hospitals. Interviews were conducted by authors SHG (PhD) and CM, who were coordinating the CoP and are trained in qualitative research methods. The purpose of collecting the qualitative data was known to the project teams and was collected on each hospital’s implementation process, the patient and provider experience, and barriers and facilitators impacting the implementation guided by the CFIR framework.[16] Interviews were between 30 and 60 minutes. They were recorded and transcribed for analysis.

Statistical analysis

Implementation was evaluated by measuring the proportion of patients in the target population who received a PODS in each quarter. To this end, a generalized linear mixed effects model was used, with fixed effects for quarter relative to implementation and between site covariates, and random slopes and intercepts for each site to model within site temporal trends. PODS spread outside of the target population was assessed using a zero-inflated mixed effects negative binomial model. Exploratory analysis was conducted to examine any association between factors at the patient, hospital, and process-level, as well as quality ratings and whether a hospital had high implementation within the target population (defined as 75% of target reached in Q4 after implementation). Factors were chosen as supported by literature and the qualitative findings.

Numerical summaries of provider, patient, and family reported data on patient-centred processes, patient understanding, and discharge experience were assessed by comparing means before and after implementation using generalized estimating equations (GEE) to account for within site clustering of responses.[19, 20] Change in healthcare utilization was analysed by plotting change over time. Linear mixed effects regression was used to assess percentage of 30-day return ED visits and readmissions, with percent PODS, within site, as the predictor of interest.

The analyses and data visualizations were performed using R version 3.6.2 by John Matelski and supervised by George Tomlinson (Biostatistics Research Unit, UHN).

Qualitative data was analysed using an iterative constant comparative process involving descriptive and interpretive analyses, open-coding, and identifying themes in the data. Study leads and members of the research team (SHG, TH, CM), read transcripts, then met to discuss initial codes and develop a preliminary coding framework. Interpretation of themes was discussed among team members to achieve consensus. Theoretical saturation, constant comparative analysis, trustworthiness, and validity checks provided assurance of data quality and rigor.

Using a triangulation approach, qualitative and quantitative results were interpreted together, towards the understanding of factors that may influence successful implementation and to draw inferences from the data.

This study was approved by the University Health Network Research Ethics Board.

Quantitative

Implementation

The mean age of the target population across all hospitals was 66.4 ± 13.5 years, 48.7% ± 7.8% were male, and 7.8% ± 8.8% had a language barrier. Rehabilitation hospitals had the lowest (average of 80) target population discharges per quarter and academic hospitals had the largest number of discharges with an average of 813 per quarter. Between April 2017 and December 2018, there were 65,221 discharges within the target populations and 41,884 patients received a PODS (64.2%).

PODS implementation as a percentage of target discharges increased over time, reaching 78% of the target population by the end of the study. The odds of implementation within the target group also increased, from 2.90 in the second quarter to 24.29 in the fifth quarter. In addition to planned spread within the target groups, 12 sites had unplanned spread with a total of 25,190 patients outside the target groups receiving a PODS.

Implementation patterns varied widely from site to site (Fig. 2) with three common implementation patterns: (1) Seven hospitals reached nearly 100 percent of the target population by the second quarter, (2) seven hospitals slowly increased coverage within the target population, and (3) seven hospitals had a dip in implementation as their original target group was increased to a larger group.

Measurement of implementation quality was high across institutions. Average quality scores ranged from 8.1 for both acute and community hospital categories to 9.3 for small hospitals (Table 2).

Table 2

Project lead ratings out of ten on the quality of the implementation
Measure	All Hospitals	Acute	Rehab	Academic	Community	Small
Consistency	8.6	8.7	8.3	8.6	8.3	10.0
Completeness	8.0	7.6	8.6	8.1	7.9	8.6
Content Quality	8.5	8.2	8.8	8.8	8.3	10.0
Process Quality	8.0	7.9	8.0	8.1	7.9	8.7
Average	8.3	8.1	8.4	8.4	8.1	9.3

Very few patient, hospital or organizational factors showed correlation with a high level of implementation (Table 3). Trends showed correlation between higher implementation with higher ratings of quality and high participation in the community of practice.

Table 3

Correlation between patient-level, hospital-level, process, quality, and provider experience with high implementation (75% of target reached by Q4)
Variable	High/Low (n)	Mean	p-value
Age	12 high 9 low	66.3 65.0	0.915
Male	12 high 9 low	48.7 50.8	0.166
Language Barrier	9 high 8 low	9.2 8.1	1.000
Hospital Category	A − 5 high, 3 low C − 3 high, 2 low S − 4 high, 4 low	41.7, 33.3 25.0, 22.2 33.3, 44.4	1.000
Hospital Type	A – 9 high, 9 low R – 3 high, 0 low	75.0, 100.0 25.0, 0.0	0.229
COP participation level	High – 9 high, 6 low Low – 0 high, 3 low Medium – 3 high, 0 low	75.0, 66.7 0, 33.3 25.0, 0.0	0.080
EMR integration	Yes – 5 high, 4 low No – 7 high, 4 low	41.7, 44.4 58.3, 55.6	1.000
Additional QI	Yes – 7 high, 4 low No – 5 high, 4 low	58.3, 55.6 41.7, 44.4	1.000
Multidisciplinary Implementation	Yes – 10 high, 5 low No – 2 high, 4 low	83.3, 55.6 16.7, 44.4	0.331
Project lead rating of consistency	12 high 9 low	9.0 8.1	0.146
Project lead rating of completeness	12 high 9 low	8.3 7.6	0.297
Project lead rating of content quality	12 high 9 low	9.2 7.7	0.018
Project lead rating of process quality	12 high 9 low	8.4 7.4	0.248
Mean workload increase	9 high 7 low	0.7 0.7	0.832
Mean value – provider reported	10 high 7 low	0.8 0.9	0.304

Effectiveness

Hospitals surveyed 627 providers and 324 patients/families pre-implementation. 348 providers and 173 patient/families completed post-implementation surveys. A majority (64.1%) of providers felt that PODS added to their workload, however 86.2% found that it added value to the experience of providing discharge instructions. Workload and value added were not correlated with high implementation (Table 3). 98% of patients and families reported that PODS added value to their discharge experience. Caregiver presence, use of teach-back and discussion about help needed at home improved with PODS implementation (Table 4). Over time, as sites increased implementation and quality, readmissions and ED visits decreased (Fig. 3). However, there was no significant change in readmissions or ED visits as a function of implementation reach (Table 4).

Table 4

Effectiveness Outcomes
Variable	Pre/Post	Mean	P value
Provider and Patient/Family Reported Patient-centered Outcomes
Provider-reported caregiver presence (scale out of 10)	Pre Post	6.30 7.00	0.026
Provider-reported use of Teach Back (scale out of 10)	Pre Post	5.40 6.50	< 0.001
*Patient-reported discussion about Help needed at home (percentage yes)**	Pre Post	0.86 0.92	0.041
Patient-reported understanding about medications* (scale out of 3)	Pre Post	2.51 2.56	0.26
Patient-reported understanding of what to do if worried* (scale out of 3)	Pre Post	2.42 2.49	0.654
Hospital Reported Healthcare Utilization Outcomes
	% (95% CI)
Change in return all-cause 30-day ED visits for a 10% increase in % of target population receiving a PODS	-0.068 (-0.546–0.410)		0.776
Change in return all-cause 30-day ED visits for a 10% increase in % of target population receiving a PODS	-0.372 (-0.233–0.977)		0.224
*questions 19, 37, and 38 of the CIHI CPES-IC survey.[28]

Qualitative

Implementation

All hospitals felt successful in their implementation. The most common barriers sited were competing priorities within the hospital, limited resources, and changes in project staff. Hospitals felt supported in working to overcome these barriers through the community of practice and by working together with their executive sponsor. All hospitals identified a need for continual work to improve implementation quality. Even for hospitals that implemented with nearly 100 percent of their target population early on, they continued to work on improving quality, as can be seen from the following quote:

We have to watch and train on using it properly and the use of plain language and ongoing PDSA is needed. We do audits and are providing feedback to the providers via practice hints. This is one of the ways that we augment the PDSA cycle.

Hospitals varied in their implementation patterns for several reasons. Findings showed that hospitals identified that 100% reach early was either due to size, as it was easier for smaller hospitals to implement a change in practice across all patient groups simultaneously, or due to implementation within an EMR, as some EMRs required mandatory PODS completion. A majority of hospitals that had a slower increase or dip in their implementation pattern identified initially utilizing PDSA cycles with small groups then slowly expanding to their full target group. Many hospitals identified trialing PODS with small groups at first facilitated implementation because it increased staff engagement and integration of PODS into the frontline process.

Hospitals reported many factors that facilitated success. In general, these fell into three categories: (1) the intervention, (2) the implementation process, and (3) the organization.

Factors about the intervention that facilitated success were adaptability of the tool to local context of various patient groups and unique processes within the same hospital, co-design of PODS included patient and family input, and unplanned and natural spread. Local adaptability was described as being particularly important, as is highlighted in the following quote:

We adapted because our patients have a high amount of dementia. We wrote it in a way that a caregiver or loved one could read it. That way it’s not given directly to the patient and the caregiver is looped into the discussion.

Factors about the process of implementation that facilitated success were strong engagement with multi-disciplinary stakeholders, integration with frontline processes via a thorough current state assessment including removing alternative workflows and building the tool into the EMR if applicable, and participation in the community of practice, as is highlighted in the following quote:

We benefited greatly from having a platform for collaboration and learning from other sites during the implementation phase which was hosted by the lead project team. Being part of the group allowed us to secure strong leadership support and dedicate resources to patient and family engagement.

Organizational-level factors that facilitated success were having a commitment to the implementation, smaller size hospitals, and integrating implementation into other QI initiatives. Smaller hospitals identified that it was easier to coordinate the project and engage and educate staff. Senior management in smaller hospitals, who wear “multiple hats”, found it easier to be more involved. Larger hospitals found it helpful to create small pilot projects and then expand to other departments. Integrating PODS into other QI initiatives was identified as both a facilitator and a consequence of implementation, as described in the following quote:

PODS was part of a bigger discharge push. We leveraged our Nursing Workflow Optimization project (an informatics project) because PODS rollout and nursing workflow are well aligned. We also developed several new inputs that helped with implementing PODS [...] All of these new inputs were piggybacked off of PODS [...] Instead of being a product of optimization PODS helped drive optimization.

Effectiveness

Many patients and families commented on the usefulness of PODS, noting that they referred to the form at home and found having all the information in one place valuable. Some patients felt that it was not necessary for them, but that it would be especially useful if “you don’t have a lot of healthcare experience.” Those with previous experience in the same hospital commented on the improvement from previous forms. Several patients said it would have been more useful if provided earlier during their stay.

Many providers commented on the usefulness of PODS for patients and families. Many also identified that it was only useful when the whole care team collaborated on completing the document and several providers commented that it enhanced collaboration. Many providers specifically noted the value of the teach-back process. The majority of providers who identified increased workload associated with PODS also noted qualitatively that the increase was small but worthwhile given the value added, as supported by the following quote:

It increases workload a little, but it is worthwhile because it helps me to feel more confident that the patient know what to do and expect when they get home.

Over the course of the project, many diverse Ontario hospitals succeeded in implementing the PODS intervention with variation in reach and implementation pattern between sites. High participation in the community of practice and quality ratings were associated with higher implementation success. PODS met a need to improve standardization and quality of patient-centred discharge processes, which was supported by a growing patient-centred focus across hospitals and policy circles.[8]

Implementation

As expected, consistent with the CFIR framework,[16] having organizational commitment and a structured community of practice with a process of iteration and stakeholder involvement, were important to implementation success. This was also reflected through qualitative reports identifying how the community of practice and executive sponsor helped in overcoming common barriers. To this end, our team, together with the hospitals involved in this project, developed an online community of practice open to anyone interested in implementing PODS. Future hospitals interested in implementing PODS should ensure executive commitment and consider joining the online community of practice or developing their own communities with other hospitals or within their hospital.

Local adaptation of the PODS tool was found to be a facilitator in this project and is known to play an important role in the diffusion of innovations.[21, 22] Likewise, local adaptation is considered the most important intervention characteristic within the CFIR.[16] The majority of individualized discharge tools are not developed through a process of co-design with patients and families.[7] The co-design process used to develop PODS was an important facilitator to implementation because organizations valued the source of the intervention and felt confident in the strength and quality of the tool. This also occurs when hospitals feel confident in the quality of their PODS, consistent with the association between high quality ratings and implementation success. These factors are intervention characteristics identified in the CFIR.[16]

Small hospital size was an identified facilitator. This is inconsistent with implementation frameworks,[16, 23] which find larger organizations more likely to be successful predominantly due to stability of the organization and having sufficient resources to run the project. Interestingly, the CFIR notes several features of an organization’s internal setting as important for implementation success that are consistent with the smaller organizations in our cohort, namely having intertwined networks and good communication. Additionally, all hospitals, regardless of size, had to ensure sufficient resources for implementation before joining the project. Although our data did not show correlation between hospital size and implementation success, smaller hospitals had higher quality ratings highlighting the importance of combining reach and quality when evaluating implementation success.

There were many factors studied that did not have any association with implementation success including patient age, patient sex, whether the patient had a language barrier, whether the hospital was acute or rehabilitation, and whether the providers found that it added to their workload. These were factors that we would have thought might make the PODS more difficult to implement, however seem to be offset by the value added by the tool and by adapting the tool to make it part of the frontline process, meeting the needs of patients, families, and healthcare providers.

Effectiveness

The PODS intervention was found to be effective in improving patient-centred processes, specifically involving the family, using teach-back, and having conversations about help needed at home. These findings are especially important in light of a recent Ontario-wide study,[24] which found that family involvement in discharge planning and knowing who to call with questions about your condition, both key features of PODS, were two of the top five priorities identified to help improve discharge experience. The remaining top responses in the study focused on lack of available homecare supports. Although PODS does not address homecare capacity, it facilitates a discussion of the help available or needed by patients at home prior to discharge.

As a balancing measure and in line with evidence,[25] we know that provider workload and perceived value are important measures for adoptability of QI initiatives such as PODS. We found that providers indicated that PODS did increase their workload, yet added value to the discharge experience and neither factor on its own was correlated with increased implementation. This was confirmed with qualitative data, which showed that the added value was enough to offset the minimal addition to workload. This also fits with the CFIR,[16] which identifies the perceived advantage to implementing a tool as a factor supporting implementation.

Resource utilization decreased over the course of the project, however no connection was found between the percentage of the target population receiving a PODS and resource utilization. The lack of connection can be partially explained by the earlier identified need to look at both reach and quality to assess success. Over time, with both implementation reach and improved quality due to improving PODS quality and other QI initiatives, there was a decrease in resource utilization. Similarly, a review of interventions using the ideal transitions in care framework,[26] found that although the majority of interventions designed to improve transitions do not result in reduced utilization, having multiple interventions concurrently addressing different elements of transitions results in the best chances of reducing readmissions.

Strengths and limitations

This project has strengths as a large-scale implementation was evaluated that included many diverse hospitals representative of the Ontario-wide landscape. Large scale quality improvement implementations are rarely described in the literature.[27] By studying the implementation of PODS, we are able to understand how the supported community of practice was helpful and why organizations implemented the intervention in different ways It provided insight into how future quality improvement communities of practice might be structured to support the needs of all participants. It also provided insight into interpretation of effectiveness results.

There were limitations with how hospital data on resource utilization was reported. Data was collected on whole target populations and not collected specific to patients who received PODS. Logically, this data still provides a useful estimate of resource utilization because the percentage of the target population reached is known and consistently increased over time. For survey data collected from providers, patients, and families, there was clustering because of variation in the number of responses each hospital was able to obtain and clustering in the actual site level means of the response variables. This is a limitation in terms of predicting impact for particular sites, yet is a strength in that the estimated effects pool over a diversity of sites. For our analysis, we used this strength and aggregated the data and did not use it to infer organization-specific conclusions. Additionally, there was approximately 5% missing data in survey responses. At this level, we would expect to find significant effects using the GEE methods used. The addition of qualitative methods to triangulate and analyze results was also used as a method of minimizing these limitations. Lastly, this project was not designed to study causal inferences and designs with individual patient or cluster randomization are needed to study a causal link between PODS and health outcomes including adherence to discharge advice and healthcare utilization.

This project confirmed the adaptability and ease of implementation of the PODS tool and highlighted the usefulness of implementing PODS to improve patient-centred discharge processes, a need and growing focus among hospitals internationally. In particular, we found that quality of PODS content and participation in a community of practice were factors impacting implementation success that should be considered by hospitals implementing PODS and similar interventions in the future. Future research in the field of implementation of healthcare QI should consider studying hospital size, which was found to be a facilitator to implementation that differs from general implementation frameworks. Our findings can help inform policy related to hospital transitions as well as, more generally, future system-wide collaborative implementation projects centered on patient-centered tools where different implementation patterns and practices are prevalent.

PODS – Patient Oriented Discharge Summary

ED – Emergency Department

ARTIC – Accelerating Research to Improve Care

QI – Quality Improvement

CFIR – Consolidated Framework for Implementation Research

COP – Community of Practice

PDSA – Plan Do Study Act

GEE – Generalized Estimating Equations

UHN – University Health Network

Ethics Approval and Consent to Participate

This study was approved by the University Health Network Research Ethics Board as Project Number 17-6272. The UHN REB provided permissions to access and use the data described in this study. As only aggregate and non-identifiable data was being used for this quality improvement initiative that was ongoing, the UHN REB deemed informed consent as unnecessary for this project.

Consent for Publication

Not Applicable

Availability of Data and Materials

The datasets used during the current study are available from the corresponding author on reasonable request.

Competing Interests

The Authors declare that they have no competing interests.

Funding

The implementation and evaluation of PODS across Ontario was funded by Health Quality Ontario and the Council of Academic Hospitals of Ontario through the ARTIC program. The ARTIC program was critical in our ability to implement and spread the PODS initiative. The evaluation reported here presents the first phase of a CIHR funded Transitions in Care Evaluation grant held by Drs. Karen Okrainec and Shoshana Hahn-Goldberg. Dr. Karen Okrainec is supported by an Early Research Award which supports members of her research team and the research described in this manuscript.

Authors’ Contributions

SHG, TH, AC, MK, VR, GT, JM, HA, CB, and KO contributed to the conception and design. SHG, CF, AT, CF, SH, BN, and BO contributed to acquisition. SHG, MK, VR, GT, JM, and KO contributed to analysis. All authors contributed to interpretation of data. SHG drafted the work and all authors contributed to revision, approved the submitted version, and have agreed both to be personally accountable for their own contributions and to ensure any questions related to the work are appropriately investigated and resolved.

Acknowledgements

The authors acknowledge the input and advice of our project advisory group as well as the critical work and support of all the individual organizations along with their project teams. Their implementation evaluation is included in this report for the work they completed while implementing PODS to improve the transition experience for patients and families (Bruyere Continuing Care, The Centre for Addiction and Mental Health, Chatham-Kent Health Alliance, Campbellford Memorial Hospital, Georgian bay General Hospital, Geraldton District Hospital, Grand River Hospital and the Waterloo Wellington LHIN, Kingston Health Sciences Centre, London Health Sciences Centre, Markham Stouffville Hospital, Oakville Trafalgar Hospital, Perth and Smith Falls District Hospitals, Ross Memorial Hospital, Renfrew Victoria and St. Frances memorial, St. Joseph General Elliot Lake, St. Joseph’s healthcare Hamilton, Thunder Bay Regional Health Sciences Centre, Temiskaming Hospital, Toronto Rehabilitation Hospital, Toronto Western Hospital, and William Osler Health System).

Hahn-Goldberg S, Okrainec K, Huynh T, et al. Co-creating patient-oriented discharge instructions with patients, caregivers, and healthcare providers.J Hosp Med 2015;10(12):804-807. doi:10.1002/jhm.2444
McMartin K. Discharge Planning in Chronic Conditions: An Evidence-Based Analysis. Ont Health Technol Assess Ser 2013;13(4):1–72.
Centers for Medicare & Medicaid Services (CMS). Medicare program: Hospital inpatient value-based purchasing program. Federal Register 2011;76(88):26490-26547.
Lapointe-Shaw L, Mamdani M, Luo J, et al. Effectiveness of a financial incentive to physicians for timely follow-up after hospital discharge: A population-based time series analysis. CMAJ 2017;189(39):E1224-E1229.
Norwegian Ministry of Health and Care Services. The coordination reform–proper treatment–at the right place and right time. Summary in English. Report No. 47 (2008–2009) to the Storting. Accessed on February 15, 2020 at https://www.regjeringen.no/en/dokumenter/report.no.-47-to-the-storting-2008-2009/id567201/
World Health Organization. (‎2016)‎. Transitions of care. World Health Organization. Accessed on February 15, 2020 at https://apps.who.int/iris/handle/10665/252272.
Okrainec K, Lau D, Abrams HB, et al. Impact of patient-centered discharge tools: A systematic review. J Hosp Med. 2017;12(2):110-117. doi: 10.12788/jhm.2692.
Health Quality Ontario. Transitions Between Hospital and Home: Care for people of all ages. Quality Standards Accessed on February 13, 2020 at https://www.hqontario.ca/Portals/0/documents/evidence/quality-standards/qs-transitions-between-hospital-and-home-quality-standard-en.pdf
Mueller P, Oppeneimer D. The pen is mightier than the keyboard: Advantages of longhand over laptop notetaking. Psychol Sci 2014;25(6): 1159-1168.
Ha Dinh T, Bonner A, Clark R, et al. The effectiveness of the teach-back method on adherence and self-management in health education for people with chronic disease: A systematic review.JBI Database of Systematic Reviews and Implementation Reports 2016; 14(1), 210-247. doi:10.11124/jbisrir-2016-2296
Rodakowski J, Rocco P, Ortiz M, et al. Caregiver integration during discharge planning for older adults to reduce resource use: A meta-analysis.Journal of the American Geriatrics Society 2017;65(8), 1748-1755. doi:10.1111/jgs.14873
Hahn-Goldberg S, Okrainec K, Damba C, et al. Implementing patient-oriented discharge summaries (PODS): A multisite pilot across early adopter hospitals.Healthcare Quarterly 2016;19(1), 42-48.
Canadian Institute for Health Information, the Government of Ontario, the Ontario Hospital Association and the Hospital Report Research Collaborative. 2007 Hospital Report: Acute Care. Accessed on February 13, 2020 at https://secure.cihi.ca/free_products/OHA_Acute07_EN_final_secure.pdf.
Canadian Institute for Health Information, the Government of Ontario, the Ontario Hospital Association and the Hospital Report Research Collaborative. 2007 Hospital Report: Rehabilitation. Accessed on February 13, 2020 at https://secure.cihi.ca/free_products/OHA_Rehab_07_EN_final_secure.pdf.
Health Quality Ontario and the Council of Academic Hospitals of Ontario. The ARTIC Program: Reflecting on five years of spread and scale 2019. Accessed on December 1, 2019 from: https://www.hqontario.ca/Portals/0/documents/qi/artic/the-artic-program-reflecting-on-five-years-of-spread-and-scale-en.pdf
Damschroder LJ, Aron DC, Keith RE et al.Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sci 2009;4(50). doi:10.1186/1748-5908-4-50
PODS-toolkit website. 2019. Accessed on December 1, 2019 at: http://pods-toolkit.uhnopenlab.ca/
Curran G, Bauer M, Mittman B, et al. Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact. Med Care 2012; 50(3): 217–226. doi:10.1097/MLR.0b013e3182408812.
Bell ML, Horton NJ, Dhillon HM, et al. Using generalized estimating equations and extensions in randomized trials with missing longitudinal patient reported outcome data. Psycho‐oncology 2018;27(9), 2125-2131.
Akter T, Sarker EB, Rahman S. A tutorial on GEE with applications to diabetes and hypertension data from a complex survey. Journal of Biomedical Analytics 2018;1(1), 37-50.
Rogers EM. Diffusion of Innovations. 5th ed. New York: Free Press 2003.
Berwick DM. Disseminating Innovations in Health Care. JAMA 2014;289(15):1969-1975.
Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. The Milbank Quarterly 2004;82(4), 581-629.
Kiran T, Wells D, Okrainec K, et al. Patient and caregiver priorities in the transition from hospital to home: results from province-wide group concept mapping. BMJ Quality & Safety
Hayes CW, Goldmann D. Highly Adoptable improvement: a practical model and toolkit to address adoptability and sustainability of quality improvement initiatives. The Joint Commission Journal on Quality and Patient Safety 2018;44(3):155-63.
Burke RE, Guo R, Prochazka AV, et al. Identifying keys to success in reducing readmissions using the ideal transitions in care framework. BMC Health Services Research. 2014;14(1):423.
Rohweder C, Wangen M, Black M, et al. Understanding quality improvement collaboratives through an implementation science lens. Preventive Medicine 2019;129:105859.
Canadian Institute for Health Information. Canadian Patient Experiences Survey — Inpatient Care. Accessed February 7, 2019 at https://www.cihi.ca/sites/default/files/document/cpes-prem-preliminary-measures-table-april2019-en-web.pdf

Download PDF

Journal Publication

published 17 Apr, 2021

Read the published version in BMC Health Services Research →

Editorial decision: Major revision
07 Dec, 2020
Review #3 received at journal
23 Nov, 2020
Review #2 received at journal
19 Nov, 2020
Reviewer #4 agreed at journal
03 Nov, 2020
Reviewer #3 agreed at journal
01 Nov, 2020
Reviewer #2 agreed at journal
30 Oct, 2020
Review #1 received at journal
29 Oct, 2020
Reviewer #1 agreed at journal
28 Oct, 2020
Reviewers invited by journal
26 Oct, 2020
Editor assigned by journal
30 Sep, 2020
Submission checks completed at journal
29 Sep, 2020
Editor invited by journal
29 Sep, 2020

You are reading this latest preprint version

Implementation, Spread and Impact of The Patient Oriented Discharge Summary (PODS) Across Ontario Hospitals: A Mixed Methods Evaluation

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Setting and context

Study design

Data collection and analysis

Quantitative

Qualitative

Statistical analysis

Results

Quantitative

Implementation

Effectiveness

Qualitative

Implementation

Effectiveness

Discussion

Implementation

Effectiveness

Strengths and limitations

Conclusions

List Of Abbreviations

Declerations

References

Supplementary Files

Status:

Journal Publication

Version 1