Design and Setting
The Implementation Science Center for Cancer Control Equity (ISCCCE) is a collaboration between the Massachusetts League of Community Health Centers (Mass League), the Harvard T.H. Chan School of Public Health (HSPH), Massachusetts General Hospital (MGH), and Dana Farber Cancer Institute (DFCI) funded by the National Cancer Institute. Mass League is the state primary care association, which “serves as an information source on community-based health care to policymakers, opinion leaders, and the media, and provides a wide range of technical assistance to health centers and communities” [14]. Mass League collaborates with the ISCCCE Implementation Lab (I-Lab) to build the research capacity of community health centers and supports implementation of evidence-based interventions among community health centers participating in implementation research pilots [15].
This study uses a cross-sectional survey to measure the inner setting of community health centers that were engaged with ISCCCE when it was initially launched. This includes staff perspectives from: 1) sites that received funding for and participated in the first round of implementation pilot studies and a learning community; and 2) sites that received funding for and participated in a learning community and capacity building only. The learning community is designed to increase engagement in and organizational capacity for participating in implementation research [15]. Informed by CFIR, items on the survey include aspects of the inner setting as well as characteristics of the individual staff responsible for implementation of evidence-based interventions [5].
Participants and Recruitment
ISCCCE research staff collaborated with leaders from Mass League to invite 123 staff members from 12 community health centers to participate in the survey. To gather a range of perspectives on the inner setting of each community health center, 1–3 people were sampled within each of five job type categories – leadership, clinical, quality improvement, community direct service, community outreach and engagement. We included staff who were actively engaged in pilot research and/or Learning Community activities. At community health centers where staff with specific roles were not engaged in ISCCCE activities, the primary contact was asked to provide a roster of staff names in each role. For roles with more than three staff members, study personnel randomly selected participants in each role type using a random number generator.
Once participants were sampled, a Mass League leader sent a prenotification email to all potential participants to notify them about an upcoming survey invitation coming from the ISCCCE research team. Several days after the email notification, health center staff were formally invited to participate in the survey using an individualized link to mitigate duplicate responses. This tiered outreach approach in collaboration with Mass League was designed to build familiarity and trust with the survey among health center staff. Up to three survey reminders were emailed over several weeks. Staff received a $25 gift card for completing the survey.
Measures
The research team fielded an online survey via REDCap between November 2020 and March 2021. To minimize the respondent burden on community health center staff, the research team and Mass League partners used a collaborative process to review and prioritize items for inclusion. The survey employed measures with good internal consistency and discriminant validity from the implementation science literature on learning climate, available resources, implementation stress, and leadership engagement [8]. All inner setting survey items were measured using a 5-point Likert scale with a “1” rating meaning “strongly disagree” and “5” rating meaning “strongly agree” and had previously been adapted to the community health center setting by Fernandez and colleagues [8]. Four items on learning climate (Cronbach’s alpha = 0.85) defined as “a climate in which leaders express their own fallibility and need for team members’ assistance and input, team members feel that they are essential, valued, and knowledgeable partners in the change process, individuals feel psychologically safe to try new methods, and there is sufficient time and space for reflective thinking” were adopted from the Practice Adaptive Research [16]. Four items assessing the commitment, involvement, and accountability of leaders were adopted from this same measure (Cronbach’s alpha = 0.92) [16]. Three items on available resources (e.g., money, training, staffing) were from the Organizational Readiness to Change Assessment [17]. Four items on perceived stress, strain, and role overload are from the TCU Organizational Readiness for Change measure (Cronbach’s alpha = 0.85) [18]. Additionally, the survey included items on participants’ roles (i.e., select all that apply from 10 options, see Table 1), years of tenure within the center, and demographics (e.g., gender, race/ethnicity, age).
Table 1
Descriptive statistics of community health center staff (N = 63) across 12 MA CHCs
| N | % |
Gender | | |
Women | 49 | 90.7 |
Men | 5 | 9.3 |
Race/ethnicity | | |
White | 32 | 50.8 |
Black/African American | 10 | 15.9 |
Asian | 9 | 14.3 |
Hispanic/Latino | 5 | 7.9 |
Other | 2 | 3.2 |
Role | | |
Clinical services | 30 | 47.6 |
Quality Improvement | 27 | 42.9 |
Management | 25 | 39.7 |
Leadership | 22 | 34.9 |
Administrative | 14 | 22.2 |
Community outreach | 11 | 17.5 |
Technology/Data services | 8 | 12.7 |
Community direct services | 7 | 11.1 |
Referrals | 1 | 1.6 |
Consultation | 1 | 1.6 |
| Mean (SD) | Range |
Age | 31.0 (9.7) | 27–63 |
Job Tenure | | |
Total years in position | 4.8 (5.8) | 0–27 |
Total years employed at center | 6.3 (6.1) | 1–27 |
We also worked with partners at the Mass League to identify unique characteristics of the community health center inner setting that may influence equity – the main focus of our center. Structural characteristics of the work infrastructure and language access services were identified as top priorities. Dichotomous (yes/no) items on human resource benefits offered by the center (e.g., formal mentoring, tuition assistance, personal counseling or employee assistance programs), and translation services from the Cultural Competency Assessment Tool for Hospitals [13] were included in order to capture aspects of the community health center infrastructure that could support equity and potentially reduce staff turnover, which can create significant challenges for implementation. The survey took an average of 15 minutes to complete.
Statistical analysis
Sample demographics are characterized with relative frequencies. Respondent reports of inner setting characteristics are described through means, standard deviations (SD), and intraclass correlation coefficient (ICC) for each item and aggregate scale scores. Together, these provide a description for each item and scale score of (a) average or expected score (mean), (b) the total variability (SD), and (c) the within-CHC variability. The ICC provides a description of to what extent the scale scores are similar or different within CHCs. This is particularly useful for understanding the utility of these measurement tools as individual measures of perceptions of the inner setting among CHC staff and/or as organizational measures collected from CHC staff. Likewise, the ICC can be interpreted as the extent to which CHC staff have consistent perceptions of the CHC’s inner setting characteristics, which may be substantively informative in and of itself. Further, we also report frequencies for each response option. Aggregate scores for inner setting characteristics were created by averaging survey responses pertaining to each characteristic as recommended [8].
Stratified analyses describe perception of inner setting characteristics by role. In order to examine the relationship between roles and inner setting characteristics, a non-overlapping role variable was created. Anyone who identified leadership as one of their roles was categorized as a leader. Subsequent role categories created were those who did any clinical work, those who were involved in community direct service or community outreach, and those involved in quality improvement. Any remaining staff were categorized as “other”.
Equity-focused inner setting characteristics are described with percentages. To minimize respondent burden, only those who said they were involved in management or quality improvement were asked about HR benefits and translation services available at the health center. Participants within a health center did not consistently report health center resources such as tuition reimbursement, languages available, or the existence of a written translation policies. In analysis, if anyone at the center reported these characteristics, that center was counted as having the policy or practice.