Development and Psychometric Properties of Surveys to Assess Patient and Family Caregiver Experience With Care Transitions

doi:10.21203/rs.3.rs-154517/v1

Download PDF

Research Article

Development and Psychometric Properties of Surveys to Assess Patient and Family Caregiver Experience With Care Transitions

https://doi.org/10.21203/rs.3.rs-154517/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 09 Aug, 2021

Read the published version in BMC Health Services Research →

You are reading this latest preprint version

Background

The purpose of this study was to develop and administer surveys that assess patient and caregiver experience with care transitions and examine the psychometric properties of the surveys. The surveys were designed to include the transitional care services or components of care, provided in the hospital and at home, that matter most to patients and their family caregivers, as well as their assessments of the overall quality of the transitional care they received.

Methods

Patients were recruited prior to discharge from 43 U.S. hospitals. The analysis dataset included responses from 9,282 patients, 1,245 Time 1 caregivers (who helped the patient in the hospital), and 1,749 Time 2 caregivers (who helped the patient at home). The psychometric properties of the survey items and composite measures were examined for each of the three surveys, including (1) item response variability and missing data, (2) exploratory factor analysis, (3) internal consistency and site-level reliability, and (4) correlations among the outcome composite measures and with other survey items. Items that performed poorly across multiple analyses were dropped from the final instruments.

Results

Overall, the final patient and caregiver surveys had acceptable psychometric properties, with a few exceptions. Exploratory factor analyses supported the composite measures, which had acceptable internal consistency reliability—Overall Quality of Transitional Care (patient and caregiver surveys), Patient Overall Health (patient survey) and Caregiver Effort/Stress (caregiver surveys). All surveys had acceptable site-level reliability except when the sample sizes needed to achieve 0.70 site-level reliability were higher than the actual sample sizes in the dataset. In all surveys, the Overall Quality of Transitional Care composite measure was significantly correlated with other composite measures and most of the survey items.

Conclusions

The final patient, T1 caregiver, and T2 caregiver surveys are psychometrically sounds and can be used by health systems, hospitals, and researchers to assess patient and caregiver experience with care transitions. Results from these surveys can be used as the basis for making improvements to transitional care delivery that are centered on what matters most to patients and their family caregivers.

Health Economics & Outcomes Research

Health Policy

Care transitions

transitional care

patient experience

caregiver experience

family caregiver

survey

psychometric

healthcare quality

hospital

Patient transitions in care from the hospital to post-acute settings or home continue to be fraught with potential gaps in care and services that can put patients at risk and overburden their family caregivers [1, 2]. It is therefore important to identify the critical transitional care services or groups of services that matter most to patients and family caregivers and that yield the best outcomes. A review of the literature on patient experience with healthcare [3] found that studies about patient experience focus on individualized care and tailoring of services to meet patients’ needs and engage them as partners in their care, which is integral to the principles and practice of patient- and family- centered care. Other research has found that better patient care experiences are associated with better clinical outcomes [4, 5], better patient safety culture within hospitals [6], and lower 30-day hospital readmission rates for acute myocardial infarction, heart failure, and pneumonia [7].

While patients’ experiences are increasingly recognized as a critical component of the overall quality of care, much less attention has been given to the experiences of family caregivers, who often bear much responsibility in caring for and managing the patient’s care during transitions across and between health settings. When it comes to care transitions, family caregivers need more information and knowledge about how to care for the patient, need to be more involved in identifying patient needs [8], want to feel cared for and about by medical providers, and want to feel prepared and capable of implementing patient care plans [9]. Family caregivers need information during every step of the process, both before and after hospital discharge, which underscores the importance of communication with healthcare professionals [10].

Understanding patient and family caregiver experience with healthcare is critical in moving toward care that is more patient-and-family-centered. While there are numerous measurement instruments designed to assess patient experience with healthcare, only a subset of these include a focus on care coordination across the continuum of care or family caregiver experiences with care transitions from hospital to home. Therefore, the purpose of this study was to develop and administer survey instruments that assess patient and family caregiver experiences with care transitions from hospital to home and examine the psychometric properties of the surveys. The surveys were designed to include the transitional care services, provided in the hospital and at home, that matter most to patients and their family caregivers, as well as their assessments of the overall quality of the transitional care they received.

This study was undertaken as one of the specific aims of a much larger project funded by the Patient-Centered Outcomes Research Institute (PCORI) called Project ACHIEVE (Achieving Patient-Centered Care and Optimized Health In Care Transitions by Evaluating the Value of Evidence). The overall aims of Project ACHIEVE were to identify the transitional care services and outcomes that matter most to patients and family caregivers, and to identify which combinations of transitional care strategies, or groups of services, yield desired outcomes among a large and diverse cohort of United States (US) hospitals [11, 12].

All study procedures were approved by the Institutional Review Boards at the University of Kentucky, Kaiser Permanente Southern California, and Westat. The study protocol was carried out in accordance with relevant guidelines and regulations.

Survey item development, cognitive testing, and pilot testing

To develop the content for the surveys, we first conducted a comprehensive literature review to identify the types of transitional care services that are important from both the patient and family caregiver perspectives. We reviewed articles that assessed transitional care services and outcomes [13-17]. We identified existing surveys and validated measures of patient experience, including Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys [https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html; https://homehealthcahps.org/] and Patient-Reported Outcomes Measurement Information System [PROMIS^®] measures [18-20]. We also reviewed research on family caregiver burden and stress [21-23].

The Project ACHIEVE research team also conducted qualitative focus groups and individual interviews with a total of 138 patients and 110 family caregivers across the U.S. [9]. This qualitative data collection elicited descriptions of patient and caregiver experiences around care transitions and identified the outcomes that were most important from their perspectives. Survey content was also informed by site visits to hospitals included in the study to better understand the facilitators and barriers of effective care transitions [24]. In addition, a Stakeholder Advisory Group (SAG), which included patient and caregiver representatives, and Scientific Advisory Council (SAC) provided important input on survey content, item wording, and survey length throughout survey development and analysis.

Based on these diverse sources of input, the ACHIEVE research team identified main content areas for inclusion in the patient and caregiver surveys and drafted survey items to assess those content areas. Some survey items were adapted from existing surveys and other items were developed to assess content areas where existing items were not available or sufficient for our measurement goals.

The patient survey was designed for patients recently discharged from the hospital to assess their experiences with care both in the hospital and once they got home. The caregiver survey was developed as a parallel instrument with the patient survey to assess similar questions about care in the hospital and at home, but from the perspective of the caregiver. The caregiver survey was designed for the family member or friend who was identified by the patient as the person who provided the most help while the patient was in the hospital and/or once they got home. The caregiver survey had two versions intended for caregivers at two different points in time: a Time 1 (T1) caregiver who provided most of the support during the patient’s hospitalization, and a Time 2 (T2) caregiver who was most involved with the patient’s care at home. A patient could have both a T1 and T2 caregiver, or at only one of these points in time. In addition, the T1 and T2 caregiver could be the same family member or friend, or a different one.

The research team conducted cognitive interviews with patients and caregivers to pretest the draft survey items. The goal of the cognitive interviews was to assess item comprehension, relevance, and ease of responding. Sixty-eight cognitive interviews (34 patients and 34 caregivers) were conducted to iteratively test variations of the patient and caregiver survey items. Both patients and caregivers were recruited to vary in age, gender, and race/ethnicity. Results of cognitive testing were used to refine survey items for pilot testing.

The research team conducted a five-month pilot study in late 2016/early 2017 in five hospitals, resulting in 131 patient and 100 caregiver responses. The aims of the pilot were to test the draft patient and caregiver surveys and test the recruitment and data collection protocols. Based on the findings from the pilot test and input from the larger ACHIEVE research team, the SAG and the SAC, some items were revised, the 86-item patient survey was reduced to 57 items, and the 72-item caregiver survey was reduced to 53 items. The shortened surveys were then used for the study’s main data collection. The patient and caregiver surveys were also translated into Spanish.

Measures

Table 1 describes the measures included in the patient and caregiver surveys, showing the number of items in each section. The majority of items were similar in the patient and caregiver surveys, with wording customized as needed. However, some items were unique. For example, the patient survey included items about patient-reported outcomes, whereas the caregiver survey asked about caregiver effort/stress. The surveys also included background questions about respondent characteristics.

Table 1. Main data collection patient and caregiver survey sections and numbers of items

Patient Survey

(57 items)

Time 1/ Time 2 Caregivers

(53 items)

Description of Survey Item Content

12 items

9 items

Introduction (beginning)/Background (end)

· Questions confirming respondent eligibility (patients/caregivers). Patient/caregiver sociodemographic and other characteristics.

13 items

(14 in Spanish version)

15 items

(16 in Spanish version)

In the Hospital

· Transitional care services/components received in the hospital, such as whether they were told or shown what to do at home, understood what to do, practiced, received information, felt ready for discharge (1= Yes, definitely, 2= Yes, somewhat, 3= No), and had a doctor’s appointment scheduled before leaving the hospital (1=Yes, 2=No).

· Healthcare professional communication items asking if healthcare professionals explained things in a way they could understand, cared for them as a person, and if they trusted the judgment of the healthcare professionals (1= Yes, definitely, 2= Yes, somewhat, 3= No).

23 items

22 items

Since the Patient Has Been Home

· Transitional care services/components received since the patient has been home, such as medical supplies or equipment, physical or occupational therapy, or home visits from a healthcare professional (1=Yes, 2=No).

· If the patient took medications, if they had contact information for healthcare professionals, and if they had help managing their/ the patient’s care (1=Yes, 2=No).

· Healthcare professional communication, if they trusted the judgment of the healthcare professionals, and received conflicting information from healthcare professionals (1= Yes, definitely, 2= Yes, somewhat, 3= No).

4 items

Overall Quality of Transitional Care

· Ratings of the quality of care the patient received in the hospital, at home, and from healthcare professionals (1 = Poor to 5 = Excellent), including whether healthcare professionals were there for patients/caregivers as much as they needed (0 = No, 1 = Yes, somewhat, 2 = Yes, definitely).

5 items

----

Patient Overall Health

· Patient-reported items from the adult version of PROMIS® (Cella et al., 2012; Hays et al., 2009; Yu et al., 2002), including physical health, mental or emotional health, sleep (1 = Poor to 5 = Excellent), bodily pain (1 = Not at all to 5 = All the time), and ability to carry out everyday activities in the past week (1 = Not at all to 5 = Completely).

----

3 items

Caregiver Effort/Stress

· Caregiver-reported amount of effort (1 = No effort to 4 = A little effort) and stress (1 = Not at all stressful to 4 = Very stressful) involved in caring for the patient since the patient has been home, and whether the effort of taking care of the patient since the hospital has changed (1= A lot easier, 3 = About the same, 5 = A lot harder).

Survey items assessing the Overall Quality of Transitional Care for the patient, Patient Overall Health and Caregiver Effort/Stress were included as composite measures. Composite measures are a combination of two or more survey items that are related to one another both conceptually and statistically, thus providing an overall summary measure of the underlying construct. These composite measures were included as outcome measures for patients and caregivers.

Data Collection

Hospital recruitment

Forty-three hospitals^{^[1]} across the U.S. were recruited into the ACHIEVE Study using a purposive sampling strategy to ensure representation of the following characteristics: 1) urbanicity; 2) safety-net; 3) critical access; 4) integrated delivery system (including Kaiser hospitals); 5) participation in alternative payment models (e.g., Accountable Care Organizations); and /or 6) participation in a formal evidence-based TC programs (e.g., Project RED) or community-based transitional care program (e.g., CMS Community-based Care Transitions Program (CCTP)).

Patient and caregiver recruitment

Medicare beneficiaries or dual-eligible patients that were discharged from the medical or surgical units at the participating hospitals were eligible to participate. For all non-Kaiser hospitals (n=30), inclusion criteria required patients to have traditional Medicare Fee-for-Service (FFS). For Kaiser hospitals (n=13), inclusion criteria included patients with Medicare Advantage or FFS. Exclusion criteria included: 1) in-hospital death, 2) transferred to another acute-care hospital, 3) discharged against medical advice, 4) admission for primary diagnosis of a psychiatric condition, rehabilitation, or medical treatment of cancer; 5) current prisoner; or 6) under suicide watch.

Hospital staff, overseen by a hospital-designated ACHIEVE Coordinator, recruited patients and Time 1 (T1—in the hospital) family member or friend caregivers of the patient. Hospital staff approached patients before discharge to obtain HIPAA authorization, consent to be contacted to complete a mail or phone survey, and contact information for a T1 caregiver who helped them during their hospitalization, if applicable. On a weekly basis throughout the data collection field period, hospital staff provided the research team with contact information for consenting discharged patients and their T1 caregivers. Contact information for Time 2 (T2—since the patient has been home) caregivers was requested from patients that completed the patient survey. Over 44 weeks of patient and caregiver recruitment from June 2017 to April 2018, 43 hospitals recruited 17,638 patients; and 41 hospitals recruited 5,031 T1 caregivers (two hospitals did not recruit T1 caregivers).

Patient survey administration

Patients were contacted beginning 51 days after discharge per the Centers for Medicare and Medicaid Services (CMS) guidelines to avoid conflicts with Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) data collection. Patient survey administration included a two-wave mail survey with phone follow-up for nonrespondents. Patient data collection was conducted over 49 weeks from August 2017 through July 2018. Patients received an initial mail survey packet which included a cover letter explaining the project, the survey, and a $5 prepaid cash incentive. The exception to the prepaid cash incentive was for patients from an integrated health system, who received a $5 promised incentive upon completion of the survey, as preferred by the system’s IRB protocol.

Seven days after the initial mailing, all patients were mailed a reminder postcard. If no response was received 24 days following the initial survey mailing, a second survey was mailed to non-respondents. Ten days after the second survey mailing, up to five follow-up phone calls were made in an attempt to obtain phone responses from non-respondents. A Spanish language survey and materials were mailed only to patients who requested this option during hospital recruitment; however, patients were able to complete the phone interview in English or Spanish based on their preference. On average, the patient survey was completed and/or returned 75 days after discharge.

Time 1 (T1) and Time 2 (T2) Caregiver survey administration

Both the T1 and T2 caregiver surveys were administered only by phone, with up to five phone call attempts. Caregivers were promised a $5 incentive upon completion of the survey, and interviews were conducted in both English and Spanish. Interviewers contacted the T1 caregiver 14 to 28 days after patient discharge; on average, the T1 caregiver survey was completed 18 days after patient discharge. Data were collected from T1 caregivers from July 2017 through May 2018 (about 42 weeks).

T2 caregivers were contacted at least 51 days after patient discharge, after patients completed their survey and provided the T2 caregiver name and phone number. On average, the T2 caregiver survey was completed 85 days after patient discharge. Interviewers collected data from T2 caregivers from August 2017 through July 2018 (about 47 weeks).

Analyses

Several psychometric analyses were conducted with the goal of identifying conceptually meaningful and reliable outcome composite measures in the patient and caregiver surveys. We also examined the psychometric properties of survey items that were not grouped into composite measures. Psychometric analyses included (1) item response variability and missing data patterns, (2) exploratory factor analysis, (3) internal consistency and site-level reliability, and (4) correlations among the proposed outcome composite measures and other survey items.

Item variability and missing data

As a first step, we examined item frequencies to evaluate the variability of responses. Items with little response variability may not be helpful in differentiating higher-scoring from lower-scoring individuals and hospitals. To assess item variability, we examined: 1) top box scores (the top, most positive responses) for items with fewer than four response options (e.g. percent of respondents that answered “Yes” for items with Yes/No response options, or percent “Yes, definitely” for items with Yes, definitely/Yes, somewhat/No response options), and 2) percent positive scores (percent of respondents that answered using the top two most positive responses) for items with four or more response options (e.g., percent "Very well/Moderately well or Excellent/Very good). For the item assessing the frequency of patient-reported bodily pain, a response indicating lower frequency was considered to be positive (i.e., percent Not at all/Once in the past week). To indicate low item variability, we flagged items that were extremely positive with top box scores or percent positive scores greater than 95%. We also flagged items that were extremely negative with percent positive responses or top box responses lower than 50%.

Next, we identified items with high percentages of missing data. High missingness might indicate that items are not relevant to a large portion of respondents. Sources of missing data in the patient, T1 caregiver, and T2 caregiver surveys included tailored inapplicable responses (e.g., “I already knew what to do”), valid skips (based on filter questions), and other types of missing (not answered, don’t know, or refused). Items were flagged as having high missingness if all missing responses combined (tailored inapplicable, valid skips, and all other missing) exceeded 65%.

Exploratory factor analysis (EFA)

We conducted an EFA to evaluate the proposed latent measurement structure in the patient, T1 caregiver, and T2 caregiver surveys, separately for each survey. We used iterated principal axis factors as the method of extraction, with varimax (orthogonal) rotation to maximize the dispersion of factor loadings within factors (i.e., the number of factor loadings close to one and close to zero). Factor loadings, or correlations between items and factors, range from -1.00 to 1.00. In general, factor loadings with absolute values above 0.40 (which explain around 16% of the variance in the item) are considered acceptable [26].

Composite measure internal consistency reliability and site-level reliability

Next, we examined Cronbach’s alpha (α) to determine the internal consistency reliability of the items within each composite measure to assess whether respondents answered the items in a similar way. Cronbach’s alpha ranges from 0 to 1, with higher alphas indicating better reliability. The minimum criterion for acceptable reliability is an alpha of 0.70 [27].

To examine the variability of the composite measures and items within hospitals compared to between hospitals, we computed site-level reliability. Site-level reliability, which is directly related to the standard error of measurement, captures the extent to which responses from patients and caregivers within or associated with the same hospital are more similar to each other than they are to responses from other hospitals. In other words, site-level reliability helps to assess how well a measure differentiates hospitals. It does so by comparing between-site variability to within-site variability, while adjusting for the average number of respondents within each hospital. Similar to internal consistency reliability, values of 0.70 or higher are considered acceptable for site-level reliability (Nunnally & Bernstein, 1994).

Composite measure correlations

Finally, we examined individual-level Spearman’s rank order correlations among the outcome composite measures and between the outcome composite measures and the survey items. Since the survey items were designed to assess different aspects of transitional care in hospitals, the composite measures and items in the survey should be related to one another. Therefore, correlations among the composite measures and between the composite measures and survey items should show a correspondence or convergence that would result in moderate or moderately high correlations. However, correlations that are very high may indicate a significant amount of overlap, implying that the composite measures or items may be measuring the same or very similar concepts. On the other hand, correlations that are very low, close to zero, may indicate that the composite measures or items are not related to one another, potentially measuring unrelated concepts.

Criteria for evaluation item performance

Items that performed poorly across multiple analyses and/or in two or more surveys were dropped from the final instruments. When considering which items to drop from the surveys, we placed most emphasis on item analysis, as items with low variability and a high percentage of missing data would not be very useful to hospitals looking to measure and improve care transitions. Exceptions to dropping included items that were considered conceptually important to measuring care transitions and items for which a large percentage of missing data was expected (e.g., Q11_A. Hospital: Written information in Spanish). Demographic/background items were excluded from psychometric analysis.

Footnote:

[1] The 43 hospitals include two locations of the same hospital, which were counted as separate entities for the purposes of psychometric analysis, but considered a single site in the larger analytic study; this difference is negligible analytically.

Overall response rates for the patient, T1 caregiver, and T2 caregiver surveys across the 43 participating hospitals were 57%, 28%, and 35%, respectively (Table 2).

Table 2. Response Rate Statistics for Patient and Caregiver Surveys

Summary Statistic	Patients	T1 Caregivers	T2 Caregivers
Number of completed surveys	9,450	1,262	1,788
Number of surveys administered	16,573	4,455	5,106
Overall response rate	57%	28%	35%

Creating the analysis dataset involved several steps. First, we combined and cleaned patient mail and phone survey responses. During the cleaning process, the following records were removed from the patient survey: (1) Non-contact phone records, (2) Phone and mail proxy respondent records (i.e., those filled out by someone else that the patient), (3) Phone and mail records where respondents did not confirm the patient’s hospital stay, (4) Phone and mail records with no responses to substantive (non-demographic) items, and (5) Phone and mail duplicate records (the most complete or earliest receipted record was retained). The T1 caregiver phone records underwent similar cleaning procedures, including removal of (1) non-contact phone records, (2) records where respondents did not confirm they were the T1 caregiver for the named patients, and (3) records with no substantive data or ineligible respondents (e.g., paid caregivers). T2 caregiver phone record data cleaning steps were identical to that of the T1 survey, with the additional removal of records where the patient self-identified as the T2 caregiver. Next, the patient, T1 caregiver, and T2 caregiver records were combined to link caregivers to patients.

To determine record “completeness,” we applied HCAHPS survey criterion requiring at least a 50% completion of applicable-to-all (ATA) questions^{^[2]} [25]. Sensitivity analyses using 90% ATA completion showed no significant difference in responses. The final analysis dataset consisted of 12,276 patient and caregiver responses representing 43 hospitals. Table 3 provides the final number of respondents by respondent type.

Table 3. Number of respondents by respondent type in analysis dataset

Respondent type	Overall number of respondents
Patients (43 hospitals)	9,282
T1 caregivers (41 hospitals)	1,245
T2 caregivers (43 hospitals)	1,749
Total Responses (43 hospitals)	12,276

Table 4 presents the characteristics of the 43 participating hospitals and provides the comparison of study hospitals to the 2015 American Hospital Association (AHA) registered hospitals on selected characteristics. The study hospitals were more likely than AHA hospitals to be from the Northeast and West, to be large (≥300 beds), and have nongovernment/non-for-profit ownership. In addition, the study hospitals were more likely to be large, urban and teaching compared to 2019 CMS Impact hospitals (Table 5).

Table 4. Distribution of study hospitals by AHA hospital characteristics[3]

Hospital Characteristic	Study Hospitals (N = 43)		AHA Hospitals (N = 6,251)
Geographic Region[4]	N	%	N	%
Midwest	8	19%	1,701	27%
Northeast	10	23%	803	13%
South	8	19%	2,576	41%
West	17	40%	1,171	19%
Licensed Beds
<100	6	14%	3,452	55%
100-299	14	33%	1,909	31%
≥300	23	53%	890	14%
Ownership
Government, non-federal	8	19%	1,476	24%
Nongovernment, non-for-profit	34	79%	3,099	50%
Investor-owned, for-profit	1	2%	1,676	27%

Table 5. Distribution of study hospitals by CMS Impact hospital characteristics[5]

Hospital Characteristic	Study Hospitals (N = 43)		CMS Hospitals (N = 3,331)
Teaching Status[6]	N	%	N	%
Major Teaching	16	37%	382	11%
Minor Teaching	20	47%	1,316	40%
Nonteaching	7	16%	1,633	49%
Urban/Rural Classification
Large urban	24	56%	1,354	41%
Other urban	8	19%	954	29%
Rural	11	26%	1,023	31%

Table 6 presents patient respondent characteristics. The majority of patient respondents were female (53%), White (78%), and Non-Hispanic (86%). Twenty-seven percent of patient respondents had at least a 4-year college degree. Most patients (80%) had a family member or friend who helped to take care of them at home. The most common category of informal caregiver was husband/wife (52%), followed by son/daughter (including in-laws) (27%).

Table 6. Patient respondent characteristics (N = 9,282)

Patient characteristics	N	%
Gender
Male	4,297	47%
Female	4,833	53%
Total	9,130	100%
Missing	152
Education
Some high school or less	1,299	15%
High school graduate or GED	2,383	27%
Some college or 2-year degree	2,791	31%
4-year college graduate	993	11%
More than 4-year college degree	1,447	16%
Total	8,913	100%
Missing	369
Hispanic, Latino, or Spanish origin
Yes	1,264	14%
No	7,465	86%
Total	8,729	100%
Missing	553
Race
White	6,908	78%
Black or African American	840	9%
Asian	266	3%
Native Hawaiian or Other Pacific Islander	39	<1%
American Indian or Alaska Native	75	1%
Other	533	6%
More than one race	247	3%
Total	8,908	100%
Missing	374
Patient had a family member or friend who helped take care of them at home
Yes	7,106	80%
No	1,814	20%
Total	8,920	100%
Missing	362
Family member or friend’s relationship to the patient (of the 7,106 who answered Yes, above)
Husband/Wife	3,269	53%
Partner/Significant Other (includes boyfriend/girlfriend)	233	4%
Son/Daughter (includes in-laws)	1,642	27%
Brother/Sister (includes in-laws)	267	4%
Father/Mother (includes in-laws)	100	2%
Grandson/Granddaughter	118	2%
Other Relative	113	2%
A Friend or Someone Else	449	7%
Total	6,191	100%
Missing	915

Note: Totals differ due to missing data

T1 and T2 caregiver respondent characteristics are presented in Table 7. The majority of both T1 and T2 caregiver respondents were female (72% and 70%, respectively). Approximately one-third both T1 and T2 caregivers had at least a 4-year college degree. Most of the caregivers were not working or were retired (64% of T1 and 70% of T2), and identified as the husband or wife of the patient (58% of T1 and 61% of T2). Approximately one-quarter of both T1 and T2 caregivers identified as sons or daughters of the patient (including in-laws). The majority of caregivers had been caring for the patient for 12 months or more (58% of T1 and 56% of T2), lived with the patient (78% of T1 and 84% of T2), and identified as the patient’s sole caregiver (51% of T1 and 60% of T2).

Table 7. T1 (N = 1,245) and T2 (N = 1,749) caregiver respondent characteristics

Caregiver Characteristics	T1		T2
Caregiver Characteristics	N	%	N	%
Gender
Male	348	28%	527	30%
Female	874	72%	1,212	70%
Total	1,222	100%	1,739	100%
Missing	23		10
Education
Some high school or less	112	9%	177	10%
High school graduate or GED	264	22%	427	25%
Some college or 2-year degree	410	34%	577	33%
4-year college graduate	214	18%	266	15%
More than 4-year college degree	210	17%	280	16%
Total	1,210	100%	1,727	100%
Missing	35		22
Current employment status
Full-time for pay	296	25%	314	18%
Full-time unpaid	14	1%	24	1%
Part-time for pay	115	10%	170	10%
Part-time unpaid	9	1%	8	<1%
Not working or Retired	773	64%	1,210	70%
Total	1,207	100%	1,726	100%
Missing	38		23
Relationship to patient
Husband/Wife	717	58%	1,064	61%
Partner/Significant Other (includes boyfriend/girlfriend)	39	3%	62	4%
Son/Daughter (includes in-laws)	336	27%	408	23%
Brother/Sister (includes in-laws)	50	4%	57	3%
Father/Mother (includes in-laws)	35	3%	39	2%
Grandson/Granddaughter	19	2%	18	1%
Other Relative	13	1%	20	1%
A Friend or Someone Else	36	3%	81	5%
Total	1,245	100%	1,749	100%
Missing	0		0
Length of time the caregiver has taken part in or overseen patient’s care
Less than 3 months	371	30%	281	16%
At least 3 months but less than 12 months	149	12%	488	28%
12 months or more	705	58%	964	56%
Total	1,225	100%	1,733	100%
Missing	20		16
Caregiver lives with patient
Yes	945	78%	1,465	84%
No	273	22%	270	16%
Total	1,218	100%	1,735	100%
Missing	27		14
Other people help caregiver care for patient
Yes	594	49%	684	40%
No	620	51%	1,046	60%
Total	1,214	100%	1,730	100%
Missing	31		19

Note: Totals differ due to missing data

Item variability

As the first step in the psychometric analysis, we examined item variability at the respondent level. Table 8 presents percent positive and top box responses for survey items in the patient, T1 caregiver, and T2 caregiver surveys. For patients, the percent positive/top box responses ranged from 4% to 96%. Two items had percent positive scores greater than 95% (Q20. Home: How well been able to use supplies/equipment? 96% for patients; and Q22. Home: How well been able to take care of wound/surgical site? 96% for T1 and T2 caregivers). Percent positive/top box responses ranged from 1% to 96% for T1 caregivers and 4% to 96% for T2 caregivers. Twenty items had percent positive/top box scores less than 50%; 17 items in the patient survey, eight items in the T1 caregiver survey, and seven items in the T2 caregiver survey. Items with excessively high or low percent positive/top box scores were flagged as having low variability. Five items were flagged as having low variability (> 95% or < 50%) across all three surveys.

We also examined the percentages of missing responses for all survey items to identify items with excessive missingness (> 65%). These percentages in Table 8 combine missingness due to tailored inapplicable responses (e.g., “I already knew what to do”), valid skips, and other sources (not answered, don’t know, or refused). Seven survey items had greater than 65% missing values in the patient, T1 caregiver, and T2 caregiver surveys, indicating that the majority of respondents across all three surveys did not answer these questions. Three of the seven items were also identified as having low item variability (Q19_B through Q19_D) across all three surveys and were therefore dropped from the final surveys. Because Q19_A was not meant to be a standalone item and had a high percentage of missing values in all three surveys, it was also dropped from the final surveys. Finally, Q22 was dropped from the final caregiver surveys because of its low variability and high missingness. Despite excessive missingness, Q11_A (Hospital: Written information in Spanish) was not considered problematic because very few respondents took the survey in Spanish, so the item was retained.

Table 8. Individual-level item variability – Percent positive/top box scores and percent of missing data (Patients [PT], T1 caregivers [T1], and T2 caregivers [T2])

Survey Item #			% Top box/ % Positive	% Missing
Q2	Hospital: Were you told/shown what to do?	PT	72%	10%
		T1	62%	27%
		T2	69%	43%
Q3	Hospital: Understood what to do at home?	PT	71%	3%
		T1	64%	9%
		T2	68%	31%
Q4	Hospital: Get to practice things you would need to do at	PT	50%	21%
	home?	T1	46%	53%
		T2	51%	*65%*
Q5	Hospital: Explain things in a way you could understand?	PT	81%	3%
		T1	83%	9%
		T2	84%	31%
Q6	Hospital: Cared about you as a person?	PT	84%	3%
		T1	73%	9%
		T2	74%	32%
Q7	Hospital: Trusted HC professionals’ judgments?	PT	80%	3%
		T1	77%	9%
		T2	79%	31%
Q8	Hospital: Got information about symptoms to watch out	PT	69%	3%
	for?	T1	67%	9%
		T2	74%	32%
Q9	Hospital: HC professional talked to you about prescription	PT	76%	6%
	and OTC medicines? (filter question)	T1	77%	12%
		T2	76%	33%
Q10	Hospital: Were side effects of medicine clear?	PT	61%	15%
		T1	62%	25%
		T2	63%	42%
Q11	Hospital: Helpfulness of written information^a	PT	90%	11%
		T1	93%	30%
		T2	92%	42%
Q11_A	Hospital: Written information in Spanish?	PT	81%	95%
		T1	77%	98%
		T2	83%	96%
Q12	Hospital: Was doctor appointment scheduled?	PT	85%	2%
		T1	85%	12%
		T2	89%	33%
Q13(R)	Hospital: Was it too soon to leave hospital? (negatively	PT	82%	2%
	worded, reverse coded) (filter question)	T1	73%	10%
		T2	77%	31%
Q14	Hospital: Reason because needed more care at home?	PT	71%	83%
		T1	67%	77%
		T2	65%	84%
Q15	Home: Had HC prof contact info? (filter question)	PT	88%	2%
		T1	88%	16%
		T2	86%	2%
Q16	Home: Got help with problems or questions [when you	PT	80%	42%
	contacted HC professionals]?	T1	82%	57%
		T2	82%	45%
Q17	Home: Had to take any prescription or OTC medicine?	PT	88%	2%
	(filter question)	T1	*95%*	16%
		T2	93%	1%
Q18(R)	Home: Has there been a time when did not take medicine	PT	84%	17%
	As directed? (negatively worded, reverse coded) (filter	T1	92%	21%
	question)	T2	90%	10%
Dropped	Home: Did not take medicine… Because forgot to take	PT	64%	87%
survey #	medicine? (DROPPED FROM FINAL SURVEYS)	T1	39%	94%
Q19_A		T2	59%	91%
Dropped	Home: Did not take medicine… Because could not afford?	PT	8%	88%
survey #	(DROPPED FROM FINAL SURVEYS)	T1	1%	94%
Q19_B		T2	4%	91%
Dropped	Home: Did not take medicine… Because of medicine side	PT	29%	88%
survey #	effects? (DROPPED FROM FINAL SURVEYS)	T1	28%	94%
Q19_C		T2	28%	91%
Dropped	Home: Did not take medicine… Because didn't know	PT	6%	88%
survey #	how/when to take medicine? (DROPPED FROM FINAL	T1	8%	94%
Q19_D	SURVEYS)	T2	13%	91%
Q19	Home: Needed to use supplies or equipment? (filter	PT	67%	2%
	question)	T1	78%	16%
		T2	76%	<1%
Q20	Home: How well been able to use supplies/equipment?^a	PT	96%	37%
		T1	93%	36%
		T2	94%	25%
Q21	Home: Had to take care of wound or surgical site? (filter	PT	32%	3%
	question)	T1	27%	16%
		T2	26%	1%
Q22	Home: How well been able to take care of wound/surgical	PT	94%	69%
	site?^a(DROPPED FROM FINAL CAREGIVER	T1	96%	78%
	SURVEYS, BUT KEPT IN FINAL PATIENT SURVEY)	T2	96%	74%
Q23	Home: Received transportation assistance? (filter question)	PT	10%	2%
		T1	---	---
		T2	---	---
Q24	Home: Wanted transportation assistance?	PT	11%	21%
		T1	---	---
		T2	---	---
Q25	Home: Received meals? (filter question)	PT	4%	1%
		T1	---	---
		T2	---	---
Q26	Home: Wanted meals?	PT	9%	13%
		T1	---	---
		T2	---	---
Q27	Home: Received physical/occupational therapy? (filter	PT	44%	2%
	question)	T1	---	---
		T2	---	---
Q28	Home: Needed physical/occupational therapy?	PT	10%	49%
		T1	---	---
		T2	---	---
Q29	Home: Had home visit? (filter question)	PT	49%	2%
		T1	52%	17%
		T2	52%	1%
Q30	Home: Wanted home visit?	PT	6%	52%
		T1	14%	61%
		T2	8%	53%
Q31	Home: Talked with HC professional? (filter question)	PT	85%	2%
		T1	67%	16%
		T2	67%	1%
Q32	Home: HC prof helped manage changes or unexpected	PT	59%	43%
	problems?	T1	77%	69%
		T2	72%	59%
Q33	Home: Explained things in a way you could understand?	PT	82%	20%
		T1	89%	45%
		T2	88%	34%
Q34	Home: Cared about you as a person?	PT	85%	20%
		T1	75%	46%
		T2	71%	35%
Q35	Home: Trusted HC prof's judgments?	PT	81%	20%
		T1	83%	45%
		T2	81%	34%
Q36(R)	Home: HC professional told you something that went	PT	87%	20%
	against what another HC professional said? (negatively	T1	84%	45%
	worded) (reversed)	T2	83%	34%
Q37	Hospital: Rate hospital in preparing you for taking care of	PT	79%	4%
	self/patient at home^a	T1	76%	13%
		T2	79%	32%
Q38	Home: Rate ability to take care of self/patient^a	PT	72%	3%
		T1	85%	17%
		T2	85%	1%
Q39	Home: Rate care from HC profs since home^a	PT	82%	16%
		T1	86%	32%
		T2	85%	10%
Q40	Overall, have HC profs been there as much as you	PT	72%	5%
	needed?	T1	70%	5%
		T2	73%	3%
Q41	Rate physical health^a	PT	43%	3%
		T1	---	---
		T2	---	---
Q42	Rate mental/emotional health^a	PT	61%	3%
		T1	---	---
		T2	---	---
Q43	Rate sleep^a	PT	38%	3%
		T1	---	---
		T2	---	---
Q44	Bodily pain^a	PT	40%	4%
		T1	---	---
		T2	---	---
Q45	Carry out everyday physical activities^a	PT	49%	3%
		T1	---	---
		T2	---	---
Q46	Home: Has a family member or friend helped care for	PT	80%	4%
	you?	T1	---	---
		T2	---	---
Q49	How confident are you in filling out medical forms by	PT	75%	2%
	yourself?^a	T1	---	---
		T2	---	---
Q50	Do you usually ask someone to help you read materials	PT	25%	2%
	you receive from the hospital?	T1	---	---
		T2	---	---
CQ5	Hospital: Did CG talk with any HC in hospital about the	PT	---	---
	patient? (filter question)	T1	92%	<1%
		T2	86%	19%
CQ15	Hospital: Did CG receive written information? (filter	PT	---	---
	question)	T1	87%	14%
		T2	93%	34%
CQ29	Home: Patient received help for mental health problem?	PT	---	---
	(filter question)	T1	12%	17%
		T2	14%	2%
CQ30	Home: CG wanted patient to receive help for mental	PT	---	---
	health problem?	T1	91%	29%
		T2	92%	18%
CQ33	Home visit scheduled when caregiver could be present?^a	PT	---	---
		T1	82%	57%
		T2	80%	50%
CQ44	Home: How much effort for CG to care for patient?^a	PT	---	---
		T1	36%	17%
		T2	40%	1%
CQ45	Home: How stressful for CG to care for patient?^a	PT	---	---
		T1	68%	17%
		T2	66%	1%
CQ46	How caring for patient has changed from hospital until	PT	---	---
	now?^a	T1	51%	17%
		T2	58%	2%

Notes: “Q”= the final patient survey item number when the item is on the patient survey only or both the patient and caregiver surveys; “CQ” = the final caregiver survey item number when the item is on the caregiver survey only. The percent missing includes tailored inapplicable responses (e.g., “I already knew what to do”), valid skips (based on the filter questions), and other missing (not answered, didn’t know, or refused). HC = healthcare; OTC = over the counter; CG = caregiver.

^aPercent positive response, the two most positive responses, is shown for this item; all other items display top box scores.

Exploratory factor analysis (EFA) for proposed composite measures

Separate EFAs were conducted for the patient, T1, and T2 caregiver surveys. The initial EFAs included all items comprising the proposed composite measures in each survey: Overall Quality of Transitional Care (patient and caregiver surveys), Patient Overall Health (patient survey), and Caregiver Effort/Stress (caregiver surveys). The initial EFA revealed issues with Q38 (Home: Rate ability to take care of self/patient) in the Overall Quality of Transitional Care composite measure for patients and T1 caregivers. Specifically, this item had a factor loading above 0.40 on two factors for patients and did not load above 0.40 on either factor for T1 caregivers. We therefore removed Q38 from the composite measure models and repeated the analysis. To maintain consistency, we made this change for the patient and caregiver surveys. However, Q38 was retained for all subsequent analyses as a single-item measure. Results of the final EFA for patients are presented in Table 9. The EFA retained two factors. All factor loadings for items on their respective composite measures were above 0.40 (range 0.51 to 0.85).

Table 9. Final exploratory factor analysis factor loadings for patients

Composite measures and items		Factor 1	Factor 2
Overall Quality of Transitional Care
Q37	Hospital: Rate hospital in preparing you for taking care of self/patient at home	0.24	0.63
Q39	Home: Rate care from HC professionals since home	0.20	0.85
Q40	Overall, have HC professionals been there as much as you needed?	0.17	0.67
Patient Overall Health
Q41	Rate physical health	0.76	0.26
Q42	Rate mental/emotional health	0.69	0.29
Q43	Rate sleep	0.59	0.17
Q44	Bodily pain	0.51	0.06
Q45	Carry out everyday physical activities	0.58	0.20

Note: HC = healthcare.

The EFAs for both the T1 and T2 caregiver surveys also retained two factors (Table 10). All factor loadings for items on their respective composite measures were above 0.40 for both T1 and T2 caregivers (range 0.60 to 0.92 and 0.65 to 0.78, respectively).

Table 10. Final exploratory factor analysis factor loadings for T1 and T2 caregivers

Composite measures and items			Factor 1	Factor 2
Overall Quality of Transitional Care
Q37	Hospital: Rate hospital in preparing you for taking care of self/patient at home	T1	0.62	0.17
Q37		T2	0.69	0.14
Q39	Home: Rate care from HC profs since home	T1	0.69	0.07
Q39	Home: Rate care from HC profs since home	T2	0.76	0.07
Q40	Overall, have HC profs been there as much as you needed?	T1	0.66	0.10
Q40	Overall, have HC profs been there as much as you needed?	T2	0.65	0.09
Caregiver Effort/Stress
CQ44	Home: How much effort for CG to care for patient?	T1	0.08	0.60
CQ44	Home: How much effort for CG to care for patient?	T2	0.08	0.72
CQ45	Home: How stressful for CG to care for patient?	T1	0.18	0.92
CQ45	Home: How stressful for CG to care for patient?	T2	0.14	0.78

The pattern and magnitudes of the factor loadings in all three surveys indicated a clear differentiation between the factors, reflecting the proposed measurement structure.

Composite measure internal consistency reliability

Table 11 presents Cronbach’s alpha values measuring internal consistency reliability for each outcome composite measure, as well as alpha values if an item were to be deleted. The two composite measures in the patient survey—Overall Quality of Transitional Care and Patient Overall Health—had internal consistency reliability above the criterion of at least 0.70 (α = 0.79 for both composite measures). Similarly, the two composite measures in the T1 and T2 caregiver surveys—Overall Quality of Transitional Care and Caregiver Effort/Stress—had internal consistency reliabilities of at least 0.70 (T1 α = 0.70, 0.72 and T2 α = 0.75, 0.72, respectively). For all three surveys, deleting any items would not improve reliability.

Table 11. Composite measure internal consistency reliability (Patients [PT], T1 caregivers [T1], T2 caregivers [T2])

Composite measures and items		Internal consistency reliability (Alpha if item deleted next to each item)
Composite measures and items		PT	T1	T2
Overall Quality of Transitional Care		0.79	0.70	0.75
Q37	Hospital: Rate hospital in preparing you for taking care of self/patient at home	0.75	0.63	0.67
Q39	Home: Rate care from HC profs since home	0.64	0.59	0.63
Q40	Overall, have HC profs been there as much as you needed?	0.74	0.61	0.69
Patient Overall Health		0.79	---	---
Q41	Rate physical health	0.71	---	---
Q42	Rate mental/emotional health	0.73	---	---
Q43	Rate sleep	0.76	---	---
Q44	Bodily pain	0.79	---	---
Q45	Carry out everyday physical activities	0.76	---	---
Caregiver Effort/Stress		---	0.73	0.72
CQ44	Home: How much effort for CG to care for patient?	---	---	---
CQ45	Home: How stressful for CG to care for patient?	---	---	---

Composite measure site-level reliability

To evaluate the within-hospital and between-hospital variability of item scores, we computed site-level reliability estimates for the outcome composite measures, their constituent items, and all other survey variables. The site-level reliability for patient composite measures was 0.82 for Overall Quality of Transitional Care and 0.78 for Patient Overall Health. The site-level reliability for Overall Quality of Transitional Care was 0.72 for T1 caregivers and 0.62 for T2 caregivers (below the criterion of 0.70). The Caregiver Effort/Stress composite measure had site-level reliability below the criterion for both T1 (0.63) and T2 (0.64) caregivers. Site-level reliability for the final survey items is shown in Supplemental Table 1. We note that when the composite measures and/or items did not meet the criterion for acceptable site-level reliability, the sample size needed to achieve hospital-level reliability of at least 0.70 was higher than the actual sample sizes in the dataset.

Composite measure correlations

Next, we examined individual-level Spearman’s rank-order correlations among the outcome composite measures in the patient, T1 caregiver, and T2 caregiver surveys. In both the patient and caregiver surveys, the Overall Quality of Transitional Care composite measure was significantly correlated with the other composite measures and most of the survey items. In the patient survey, Patient-Reported Outcomes was significantly related to the Overall Quality of Transitional Care (r_s= 0.40, p <.05). In the caregiver surveys, Caregiver Effort/Stress was significantly related to Overall Quality of Transitional Care for T1 caregivers (r_s= 0.19, p <.05) and for T2 caregivers (r_s= 0.18, p <.05). We also examined individual-level correlations between the composite measures and other survey items (in Supplemental Table 2). Out of 44 possible associations between the other survey items and the patient composite measures, the majority were statistically significant (p < .05) (with Overall Quality of Transitional Care - 41 correlations; with Patient-Reported Outcomes - 40 correlations). Out of 41 possible associations between the other survey items and the caregiver composite measures, the majority of these were also statistically significant (p < .05) (with Overall Quality of Transitional Care - 32 correlations for T1 caregivers and 31 for T2 caregivers; with Caregiver Effort/Stress – 30 correlations for T1 caregivers and 33 for T2 caregivers).

Final survey items

After reviewing the performance of individual survey items other than those grouped into composite measures across the different psychometric analyses, we identified and removed items with multiple analytic issues from the final patient and caregiver surveys (four items from the patient survey, five from the T1 caregiver survey, and five from the T2 caregiver survey, shown in Supplemental Table 3). The final patient, T1 caregiver, and T2 caregiver surveys (shown in Supplemental Appendixes A, B, and C) have 56, 51, and 51 items respectively.

Footnotes:

[2] “Applicable to all” indicates if a question was applicable to all respondents, excluding questions skipped based on previous responses.

[3] Based on the 2015 AHA Annual Survey Data Set

[4] States and territories are categorized into the following regions:

Midwest: IL, IN, MI, OH, WI, IA, KS, MN, MO, NE, ND, SD

Northeast: CT, ME, MA, NH, RI, VT, NJ, NY, PA

South: DE, FL, GA, MD, NC, SC, VA, DC, WV, AL, KY, MS, TN, AR, LA, OK, TX

West: AZ, CO, ID, MT, NV, NM, UT, WY, CA, OR, WA

[5] Based on fiscal year (FY) 2019 Final Rule Impact File

[6] Teaching status derived from two variables: 2015 AHA Annual Survey MAPP status and CMS 2019. Impact file residents to bed ratio.

Lack of appropriate, well-organized transitions can lead to unplanned hospital readmissions and poor patient outcomes. While hospitals continue to focus efforts to improve care transitions and reduce readmissions, it is important to understand the care transitions experience from the perspective of patients and their family caregivers when deciding where to invest finite resources. Our study developed and tested surveys designed to assess the delivery of transitional care services or components of care, provided in the hospital and at home, from the patient and family caregiver perspectives, as well as their assessments of the overall quality of the transitional care they each received. Importantly, the transitional care services included in the survey were based on what mattered most to patients and family caregivers in transitional care, as assessed in another study in Project ACHIEVE [9].

Across the patient and caregiver surveys, most items had reasonable response variability and missingness, even though overall responses tended to be positive. Percent positive or top box scores were similar between T1 and T2 caregiver. Only four items were dropped from the final patient and caregiver surveys and one item was dropped from the final caregiver surveys because of low variability and/or a high percentage of missing values.

When examining the initial factor structure of the three proposed composite measures—Overall Quality of Transitional Care (patient and caregiver surveys), Patient Overall Health (patient survey), and Caregiver Effort/Stress (caregiver surveys)—one problematic item was identified (Q38—Home: Rate ability to take care of self/patient). After dropping the item, the final factor analysis yielded good item factor loadings (above 0.40) that supported the final composite measures. The final composite measures demonstrated good internal consistency reliability (above 0.70).

Site level reliability of the composite measures was also good for the patient survey, but did not reach acceptable levels for T2 caregivers on Overall Quality of Transitional Care, and for T1 and T2 caregivers on Caregiver Effort/Stress. We noted that when the composite measures and/or items did not meet the criterion for acceptable site-level reliability, the sample size needed to achieve hospital-level reliability of at least 0.70 was higher than the actual sample sizes in the dataset. Because T1 and T2 caregivers were recruited through patients and response rates were much lower for caregivers than for patients (28% for T1 and 35% for T2 caregivers), the average number of caregiver respondents associated with a specific hospital was low (for the items in the final survey the average number of caregivers ranged from 8 to 30 for T1s and 7 to 41 for T2s). Therefore, it is likely that the small number of respondents affected the reliability of the hospital-level caregiver composite measure scores.

In the patient survey, Patient-Reported Outcomes was significantly related to the Overall Quality of Transitional Care, and Caregiver Effort/Stress was significantly related to Overall Quality of Transitional Care for T1 and T2 caregivers. In addition, the majority of the survey items within each survey were also related to the composite measures. These significant correlations support the construct validity of the composite measures in relation to the survey items. Including built-in “outcome” measures within the surveys enables analyses to examine how the receipt of certain transitional care services or components relates to patient and caregivers’ overall transitional care, patient-reported health outcomes, and caregiver effort/stress, which are important patient and family-centered outcomes of care.

Because the survey items ask about many different types of services or components of care, including communication, providing education and information, medical supplies or equipment, transportation assistance, meals, physical or occupational therapy, most of the items are not grouped into composite measures. Instead, the type or number of services or components of care can be examined to identify which groups or clusters result in more positive patient and caregiver experiences and outcomes. Another study by the Project ACHIEVE team [12] conducted analyses of survey data from hospitals and selected data from the patient survey and identified five groups of transitional care components or strategies that were most likely to co-occur and be delivered by hospitals. The strategies patients reported receiving were more important in predicting 30-day hospital readmissions than strategies that hospitals reported delivering.

Our study’s strengths include broad input from a Stakeholder Advisory Group (SAG), which included patient and caregiver representatives, a Scientific Advisory Council (SAC), and the ACHIEVE research team, at key steps in the survey development and testing process which enabled us to incorporate comprehensive content about transitional care services received both in the hospital and at home. In addition, our iterative development process involved extensive cognitive testing and pilot testing prior to main data collection. Our strong study design included caregivers that were identified by and linked to patients rather than using cross-sectional, unrelated samples. Finally, we conducted our study with a large sample of hospitals, patients, and caregivers over an extended period of data collection of 49 weeks.

There are also several limitations of the study. Importantly, the 51-day wait time constraint before we could begin patient data collection, that was imposed to avoid overlap with Hospital CAHPS patient experience data collection, is very likely to have affected patient recall of their hospital experiences. While we tried to overcome that limitation by surveying T1 caregivers shortly after the patient’s discharge, the caregivers’ responses are not a proxy for patients since they are from a different perspective. Although we had high response rates for patients, caregiver response rates were lower and because patients did not always identify caregivers, the actual number of caregiver responses was low relative to patient responses. In addition, although we had a large sample of hospitals, it is possible that those hospitals that agreed to participate in the study may already have been more engaged in transitional care than hospitals not included in the study and therefore led to more positive survey results. Finally, the majority of patient respondents were white (78%) and high-school educated or higher (85%) so the results obtained in our study may not be as generalizable to more diverse patient populations.

Our study included a comprehensive survey development and testing process that included extensive involvement and input from patients, caregivers, and researchers throughout the entire process. In addition, our study’s large-scale data collection spanned almost an entire year with thousands of patients and caregivers. The surveys were designed to include a broad range of transitional care services, provided in the hospital and at home, that matter most to patients and their family caregivers, as well as their assessments of the overall quality of the services received during care transitions, self-reported patient health, and caregiver effort/stress. While there are numerous existing measures that assess patient experience with healthcare delivery across various settings of care, there are a limited number of measures that specifically focus on transitions of care, and even fewer that obtain a family caregiver perspective. Our study fills these important gaps, especially in measurement of family caregiver experiences with care transitions [28].

The surveys can be used independently to assess patient and caregiver experiences with care transitions, but the parallel focus of surveys also allows for more nuanced comparisons. Patients can identify their family caregivers so patient and caregiver experiences on equivalent items can be compared, as well as comparisons between caregivers that provided assistance during the patient’s hospital stay and those that assisted the patient at home. The corresponding nature of the items and composite measures in the patient and caregiver surveys adds new possibilities for examining associations among these critical perspectives on care transitions.

Psychometric analyses provided overall support for the transitional care composite measures and items in the final patient and caregiver surveys. The final patient, T1 caregiver, and T2 caregiver surveys are psychometrically sound and can be used by health systems, hospitals and healthcare researchers to assess patient and caregiver experience with care transitions. Results from these surveys can be used as the basis for making improvements to transitional care delivery that are centered on what matters most to patients and their family caregivers.

ACHIEVE (Achieving Patient-Centered Care and Optimized Health In Care Transitions by Evaluating the Value of Evidence); AHA (American Hospital Association); ATA (applicable-to-all); CAHPS (Consumer Assessment of Healthcare Providers and Systems); CCTP (Community-based Care Transitions Program); CG (caregiver); CMS (U.S. Centers for Medicare and Medicaid Services); EFA (Exploratory factor analysis); FFS (Fee-for-Service); HC (Healthcare); HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems); IRB (Institutional Review Board); OTC (Over the counter); PCORI (Patient-Centered Outcomes Research Institute); PROMIS (Patient-Reported Outcomes Measurement Information System); SAC (Scientific Advisory Council); SAG (Stakeholder Advisory Group); T1 (Time 1); T2 (Time 2); US (United States)

Ethics approval and consent to participate

Institutional review boards at the University of Kentucky (14-0789-F3R, 15-0298-F3R, 47657), Kaiser Permanente Southern California (10594), and Westat (FWA 00005551) reviewed and approved the study protocol. HIPPA authorization was obtained from all patients when recruited from hospitals. Informed consent was obtained from all patient and caregiver telephone survey participants. Mail-based patient surveys were prefaced with a cover sheet containing the elements of informed consent.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

Research reported in this manuscript was funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (TC-1403-14049). The statements in this manuscript are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.

Contributions

JS directed the patient and caregiver survey development study and led the manuscript; KZ conducted analyses and wrote results; DC and TF helped develop the patient and caregiver surveys; JR led pilot and main data collection; MW served as the PI of the larger Project ACHIEVE study; JL, TD, HN, MM, SM, KH, CL, JC, JB, and MW contributed to survey development and data collection design and contributed to the study design; all authors read and approved the manuscript.

Corresponding author

Correspondence to [email protected].

Acknowledgements

We would like to acknowledge PCORI for funding the project, the entire Project ACHIEVE research team, Stakeholder Advisory Board, Scientific Advisory Council, the 43 participating hospitals for their efforts in recruiting patients, and the patient and family caregivers who participated in the study.

Naylor, M. and Keating, S. 2008. Transitional Care: Moving patients from one care setting to another. Am J Nurs. 2008 September; 108(9 Suppl): 58–63. doi:10.1097/01.NAJ.0000336420.34946.3a. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2768550/
Mansukhani RP, Bridgeman MB, Candelario D, Eckert LJ. Exploring Transitional Care: Evidence-Based Strategies for Improving Provider Communication and Reducing Readmissions. P T. 2015 Oct;40(10):690–4. PMID: 26535025; PMCID: PMC4606859. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4606859/
Wolf, J., Niederhauser, V., Marshburn, D., and LaVela, S. Defining patient experience. Patient Experience Journal. 2014. 1(1): 7–14. DOI: 10.35680/2372-0247.1004 Available at: https://pxjournal.org/journal/vol1/iss1/3
Glickman, S. W., Boulding, W., Manary, M., Staelin, R., Roe, M. T., Wolosin, R. J., Schulman, K. A. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circulation. Cardiovascular Quality and Outcomes. 2010; 3:188–195.
Meterko, M., Wright, S., Lin, H., Lowy, E., & Cleary, P. D. Mortality among patients with acute myocardial infarction: The influences of patient-centered care and evidence based medicine. Health Services Research. 2010; 45(5 Pt 1): 1188–1204.
Sorra, J., Khanna, K., Dyer, N., Mardon, R., & Famolaro, T. Exploring relationships between patient safety culture and patients’ assessments of hospital care. Journal of Patient Safety. 2012; 8(3): 131–139.
Boulding, W., Glickman, S. W., Manary, M. P., Schulman, K. A., & Staelin, R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. American Journal of Managed Care. 2011; 17(1):41–48.
Almborg, A.H., Ulander, K., Thulin, A., and Berg, S. Discharge planning of stroke patients: The relatives’ perceptions of participation. Journal of Clinical Nursing. 2009; (18): 857–865. doi: 10.1111/j.1365-2702.2008.02600.x
Mitchell et al. Care transitions from patient and caregiver perspectives. Annals of Fam Med. 2018. 16(3):225–231.
Plank, A., Mazzoni, V., and Cavada, L. Becoming a caregiver: New family carers’ experience during the transition from hospital to home. Journal of Clinical Nursing. 2012; (21): 2072–82. doi: 10.1111/j.1365-2702.2011.04025.x
Li, J., Brock, J., Jack, B., Mittman, B., Naylor, M., Sorra, J., Mays, G., Williams, M. Project ACHIEVE—using implementation research to guide the evaluation of transitional care effectiveness. BMC Health Services Research. 2016; 16:70. DOI 10.1186/s12913-016-1312-y.
Li, J., Du, G., Clouser, J., Stromberg, A., Mays, G., Sorra, J., Brock, J., Davis, T., Mitchell, S., Nguyen, H., Williams, M. Improving evidence-based grouping of transitional care strategies in hospital implementation using statistical tools and expert review. BMC HSR, 2021; 21(35), https://doi.org/10.1186/s12913-020-06020-9.
Naylor, M.D., Shaid, E.C., Carpenter, D.C., Gass, B., Levine, C., Li, J., Malley, A., McCauley, K., Nguyen, H.Q., Watson, H., Brock, J., Mittman, B., Jack, B., Mitchell, S., Callicoatte, B., Schall, J., and Williams, M.V. Components of Comprehensive and Effective Transitional Care. Journal of the American Geriatrics Society. 2017; 65(6):1119–1125. doi: 10.1111/jgs.14782 PMID: 28369722 PMCID: PMC5497308
Hansen LO, et al. Project BOOST: Effectiveness of a multihospital effort to reduce rehospitalization. J Hosp Med. 2013;8(8):421–7.
Jack BW, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med. 2009;150(3):178–87.
Coleman EA, et al. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med. 2006;166(17):1822–8.
Naylor MD, et al. Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. JAMA. 1999;281(7):613–20.
Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology. 2010;(63): 1179–1194. doi:10.1016/j.jclinepi.2010.04.011.
Hays, R. D., Bjorner, J. B., Revicki, D. A., Spritzer, K. L., & Cella, D. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Quality of Life Research. 2009; 18(7):873–880, whereas the sleep outcome was based on the PROMIS Sleep Disturbance and Sleep-Related Impairment Item Banks Adult (PROMIS http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis/list-of-adult-measures)
Yu, L., Buysse, D. J., Germain, A., Moul, D. E., Stover, A., Dodds, N. E., ... & Pilkonis, P. A. Development of short forms from the PROMIS™ sleep disturbance and sleep-related impairment item banks. Behavioral sleep medicine. 2012;10(1):6-24.
Elmstahl, S., Malmberg, B. Annerstedt, L. Caregiver's burden of patients 3 years after stroke assessed by a novel caregiver burden scale. Arch Phys Med Rehabil. 1996;77(2):177–82.
Pearlin, L.I., Mullan, J.T., Semple, S.J., and Skaff, M.M. Caregiving and the stress process: An overview of concepts and their measures. The Gerontologist. 1990;(30):583–594.
Zarit, S. H., Reever, K.E., and Bach-Peterson, J. Relatives of the impaired elderly: Correlates of feelings of burden. The Gerontologist. 1980;(20):649–655.
Scott AM, Li J, Oyewole-Eletu S, Nguyen HQ, Gass B, Hirschman KB, et al. Understanding Facilitators and Barriers to Care Transitions: Insights from Project ACHIEVE Site Visits. Jt Comm J Qual Patient Saf. 2017;43(9):433–47.
Centers for Medicare & Medicaid Services. CAHPS® Hospital Survey (HCAHPS) Quality Assurance Guidelines V12.0. 2017. http://www.hcahpsonline.org/globalassets/hcahps/quality-assurance/qag_v12.0_2017.pdf.
Stevens, J. P. Applied multivariate statistics for the social sciences (4th ed.). 2002. Mahwah, NJ: Lawrence Erlbaum.
Nunnally, J. C., & Bernstein, I. H. Psychometric theory (3rd ed.). 1994. New York: McGraw-Hill.
Gillick MR. The Critical Role of Caregivers in Achieving Patient-Centered Care. JAMA. 2013; 310(6):575–576. doi:10.1001/jama.2013.7310

No competing interests reported.

Download PDF

Journal Publication

published 09 Aug, 2021

Read the published version in BMC Health Services Research →

Editorial decision: Major revision
04 May, 2021
Reviewers agreed at journal
22 Feb, 2021
Reviews received at journal
22 Feb, 2021
Reviewers agreed at journal
30 Jan, 2021
Reviewers invited by journal
28 Jan, 2021
Editor assigned by journal
28 Jan, 2021
Editor invited by journal
28 Jan, 2021
Submission checks completed at journal
28 Jan, 2021
First submitted to journal
24 Jan, 2021

You are reading this latest preprint version

Development and Psychometric Properties of Surveys to Assess Patient and Family Caregiver Experience With Care Transitions

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Results

Conclusions

Background

Methods

Results

Discussion

Strengths And Limitations

Conclusions

List Of Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1