The five communities in the study were chosen from a larger set of communities engaging in an early childhood systems transformation effort in 2018, The Early Childhood Learning and Innovation Network for Communities (EC-LINC; https://cssp.org/our-work/project/early-childhood-learning-and-innovation-network-for-communities/) led by the Center for the Study of Social Policy (CSSP). Participating communities in EC-LINC bring together early childhood brokering or hub organization with specific community partners to strengthen systems and services to better meet the needs of families with young children. In 2018, there were 10 EC-LINC communities. Those in the study were chosen because of their interest in strengthening how they partnered with health care systems to promote family protective factors. During this time, pediatric health care clinics within each community also self-selected and received resources to implement DULCE or I-SCRN as part of program developers’ larger efforts to implement the models. Clinics that opted in to DULCE or I-SCRN in the communities were then approached to participate in the study. DULCE clinics were recruited by the research team and CSSP, and interested I-SCRN clinics responded to a recruitment flyer distributed by the research team and the AAP. The final sample included nine clinics—seven implementing DULCE and two implementing I-SCRN. All study procedures were approved by the University of Chicago Social Services Administration-Chapin Hall Institutional Review Board (IRB).
We describe each data collection method next, beginning with qualitative data collection methods across various stakeholder groups over time and then transitioning to quantitative methods including electronic health record extraction and longitudinal survey methods. For the quantitative data collection with families, community-based Field Interviewers completed the interviews (see Family Longitudinal Interviews section). For qualitative interviews and focus groups, data collection was conducted by a research team led by the PhD-level qualitative study lead. Interviews and focus groups were conducted in English, Spanish and in one community Haitian-Creole, and lasted 45 to 90 minutes and were audiotaped and transcribed. Table 1 describes the levels of primary data collected, data sources, methods, and number of participants per time point. Data were collected in each of three years (2018-2020) of the study. In 2018, interviews and focus groups were conducted with national leadership of the three innovations, EC-LINC leadership in the five communities, and leaders of the clinic innovation teams. We commenced family longitudinal interviews in March 2018 completing the final longitudinal interviews with families in February 2020. In 2019, we focused data collection at the community and clinic levels.
Table 1. Data collected at the national, community system, pediatric healthcare clinic, and family levels
Levels and Data Sources
|
Sample Description
|
Methodf
|
No. of participants by time pointg
|
T1
|
T2
|
T3
|
2018 2019 2020g
|
National Innovations
|
National leadership of DULCE, I-SCRN and HMG innovations
|
I
|
16
|
|
|
Community Systems
|
Early Childhood Lead Organizations
|
EC-LINC Leadership in each community
|
I
|
11
|
|
|
Community Service Providers
|
Mental health, child care/early education, home visiting, and public health providersa
|
FG
|
|
39
|
|
Families
|
Parents or caregivers with early childhood service experiences (e.g. home visiting, early intervention)a
|
FG
|
|
58
|
|
Centralized Referral Study
|
A sample of three community Help Me Grow (HMG affiliatesb
|
|
|
|
|
Help Me Grow Staff
|
Leadership staff within each HMG affiliate
|
FG
|
|
13
|
|
Community Service Providers
|
Mental health, child care/early education, home visiting, or public health services providers that referred families to HMGb
|
FG
|
|
24
|
|
Healthcare Providers
|
Pediatric providers that referred families to HMG
|
I
|
|
11
|
|
Families
|
Parents/caregivers who had utilized HMGb within the past six months
|
FG
|
|
17
|
|
Pediatric Healthcare Clinics
|
Innovation teams
|
DULCE and I-SCRN team membersc
|
I
|
54
|
|
|
Clinicians
|
Health care staff providing clinical services, regardless of being involved in the DULCE or I-SCRN innovationsd
|
Sh
|
|
63
|
|
Families
|
Parents/caregivers who received pediatric health care services at each clinic within the past year
|
FG
|
|
110
|
|
Families (n=908)
|
Parents of newborns age 2 to 24 weeks who received pediatric health care services at each clinice
|
SI
|
888i
(98%)
|
764
(84%)
|
707
(78%)
|
aCommunity service providers were nominated for focus groups by the early childhood lead organization; families with experience accessing early childhood services were nominated for interviews by either the early childhood lead organization or community service providers.
bHMG affiliates (n=3) were selected based on geographic diversity (2 urban and 1 rural community) and years of HMG implementation in response to a request for applications. Community service providers, healthcare providers, and families with recent referral experiences with HMG were nominated by HMG staff.
cIncludes the physician or nurse practitioner innovation champion, DULCE family specialist (as applicable), DULCE legal partner (as applicable), DULCE early childhood lead (as applicable), social worker(s), clinic administrator, and in some clinics behavioral health leadership.
dIncludes physicians, pediatricians, nurse staff, medical assistants, and physician assistants.
e908 families were recruited from the 9 participating health clinics by study field interviewers February 1, 2018-May 31, 2019.
fI=Interview; FG=Focus group; S=Survey; SI=Survey Interview. Family focus groups were conducted in both English and Spanish (15 total English FGs and 13 total Spanish FGs).
gTime points refer to data collection as planned and may not precisely match calendar years of data completion.
hThe clinician survey included two measures, the Maslach Burnout Inventory Human Services Survey for Medical Personnel (Maslach & Jackson, 2016) and the American Academy of Pediatrics Periodic Survey of Fellows, as adapted by Help Me Grow (Cornell, 2016).
iFor family survey interviews, T1, T2 and T3 are infant age 2 weeks-6 months (T1), 7-11 months (T2), and 12-15 months, with at least 30 days between interviews.
National Innovations
Interviews were completed with 16 of the DULCE, I-SCRN and HMG national teams members (N=16). The interviews centered on three topics: (1) lessons learned about support clinic innovation; (2) the conceptual approach to addressing toxic stress and family engagement; and (3) emerging challenges and opportunities for the field. The purpose of this investigation was to understand the innovation model and implementation plans at the outset of the study in 2018.
Community Systems
Multiple levels of community-level data were collected to understand each community’s early childhood service system. First, we conducted telephone interviews with 1-2 EC-LINC leadership staff per community (n=11). EC-LINC leaders were asked about the vision for early childhood-healthcare partnerships and lessons learned from DULCE and I-SCRN implementation and from supporting Help Me Grow as a centralized referral network.
We then conducted focus groups with community service providers (n=36) recruited by the early childhood organizations such as mental and behavioral health providers, child care and early education providers, home visiting providers, regional centers, and public health departments. The focus groups inquired about approaches to mitigating toxic stress, healthcare-early childhood partnerships, policies and practices related to screening and referral for social determinations of health, and family engagement strategies.
To understand family perspective, we asked the early childhood organizations and providers to nominate and invite parents or caregivers who could speak to experiences accessing early childhood services to a focus group. We conducted 8 focus groups (4 in English and 4 in Spanish) with a total of 58 parents/caregivers. Parents/caregivers had diverse experiences with the early childhood and healthcare systems (participation in home visiting, healthy pregnancy programs, or as community leaders). The 90-minute focus groups inquired about experiences with community providers and pediatric primary care in their community.
Finally, we partnered with one participating state to understand the role of Help Me Grow in facilitating the referral network and bridging health care and community-based services for families. The purpose was to learn from HMG staff, community providers, health care providers, and families about their experiences with the roles of HMG as a centralized early childhood referral system. Three HMG affiliate locations responded to a request for applications and were selected for geographic representation and diversity in the number of years of HMG implementation. Within the three communities (two urban, one rural/suburban), we conducted 45-minute telephone interviews with HMG staff and health care providers (n=11). We conducted 3 English and 2 Spanish focus groups with families (n=17), and 3 focus groups with community providers (n=24). Focus groups were 60 minutes and asked about perceptions of facilitators and barriers to participating in the centralized referral system and critical opportunities to strengthen referral pathways in each community. Healthcare provider interviews also centered on these topics.
Pediatric Health Care Clinics
We collected data from three groups at each clinic: (1) innovation team members (DULCE or I-SCRN), (2) clinicians, and (3) families. Innovation team members (n=54) were interviewed in 2018 to gain perspective about early experiences with implementation. The interviews asked about core innovation components, opportunities, and barriers to integrating SDOH screening into the practice’s standard of care. Participants included the physician or nurse practitioner “champion,” trained family specialists, social workers, clinic administrators, and other team members supporting implementation. In 2019, we surveyed a broader group of clinic staff. We invited staff at each clinic (physicians, pediatricians, nurse staff, medical assistants and physician assistants) to complete an online survey, resulting in 63 completed surveys. The survey was designed to measure clinician attitudes and practices related to early childhood healthcare services, screening, and perceptions of occupational burnout (see Table 1 for the measures used).
Finally, we asked clinic innovation team members and early childhood organizations to nominate parents/caregivers to be invited to a 90-minute focus group. Reflecting the languages preferred by families served by the clinics, we conducted 7 focus groups in English and 8 focus groups in Spanish with 110 total participants and at least one focus group conducted at each clinic. Focus groups asked about experiences with screening and referral for social needs in the clinic setting, and interactions with community providers following referrals.
Family Longitudinal Interviews
Aiming to enroll 110 parents/caregivers of infants per clinic, we recruited 908 parents/caregivers to participate in three in-person survey interviews before their infant’s 2-year birthday (92% of recruitment goal). The first interview was conducted when infants were 2 weeks to 6 months of age, followed by midpoint and final interviews at infant age 7-11 months and 12-15 months, respectively. After the initial interview, each follow-up interview was scheduled to allow a period of no fewer than 30 days between interview waves.
Recruitment was facilitated through close collaboration with clinics, and conducted by 12 community-based field interviewers. Field interviewers were bicultural and fluent in English and Spanish and in one community: Haitian-Creole. Recruitment strategies were discussed, reviewed, and refined collaboratively with clinics throughout the study time period to accommodate clinic flow and patient protection. Early challenges in recruiting families led to highly effective collaborative decisions. Recruitment techniques included waiting room recruitment (field interviewers approached families in the waiting room), warm hand-offs from clinic staff to the field interviewer, clinic staff introducing the study to parents by phone followed by field interviewer phone contact, and texting parents. Field interviewers used study-issued surface notebooks for all communication and data collection with families.
Data sharing agreements enabled collaboration between Chapin Hall and clinic staff and between clinics and field interviewers. The study included extensive field interviewer supervision by a bilingual and bicultural researcher using real-time information and data about contact attempts and interview completion rates with parents to support field interviewers in reaching their recruitment and retention goals. Supervision also addressed the challenges experienced by Field Interviewers when engaging with caregivers and their infants to maintain boundaries. Field interviewers also were provided monthly summaries of their progress-towards-goals based on the targeted recruitment number, also discussed during monthly individual supervision. Study retention was 84% at midpoint, and 78% at the final interview (Table 1).
Family Longitudinal Interview Survey Measures
Table 2 presents the measures set administered with parents/caregivers. Interviews included a group of established and newly developed measures of five areas of family experiences: (1) the home and neighborhood environment, (2) parent adverse childhood experiences, (3) resilience and personal mastery, (4) parenting, and (5) access to social support and resources. Measures were selected to provide a snapshot of comprehensive household and environmental risk and protective factors in infancy among families receiving pediatric care in participating clinics. The study will use this snapshot to examine how clinic engagement and support for impacted families entering pediatric care with diverse constellations of strengths and needs. Among the measures assessed at enrollment, we collected parents’ report of Adverse Childhood Experiences (ACEs), using the expanded scale developed by the Center for Youth Wellness (27). Though primarily interested in current household and environmental circumstances, caregiver ACEs are associated with factors directly impacting the child’s environment, including caregiver mental health, substance use, and relational well-being and were therefore important to capture as part of the overall assessment of the household.
Table 2. Family Longitudinal Survey Measures
Measure
|
|
Construct
|
Description
|
Family Measures
|
Home and Neighborhood Environments
|
Safe Environment for Every Kid–Parent Questionnaire (SEEK) (37)
|
|
Environmental Safety
|
14-item screening tool used by child healthcare professionals for identifying psychosocial risk factors. Validated among pediatric primary care populations. (38,39,40)
|
Housing Quality Scale (41)
|
|
Perception of housing quality
|
5-item self-report scale of housing quality including physical condition of current housing. Reliability of this measure was established by developers; reliability confirmed and validity established through pilot testing in this study.
|
Housing Instability Index (42)
|
|
Housing stability
|
10-item self-report scale assessing instability over the previous 6 months, including items related to eviction, difficulty paying for housing, and transiency. Reliability of this measure was established by developers; reliability confirmed and validity established through pilot testing in this study.
|
Neighborhood Disorder Scale (41)
|
|
Neighborhood safety climate
|
5-item self-report assessing perceptions of neighborhood disorder (e.g. drug use, graffiti, loitering, and buildings in disrepair). Reliability of this measure was established by developers; reliability confirmed and validity established through pilot testing in this study.
|
Neighborhood Danger Scale (41)
|
|
Perception of neighborhood safety
|
5-item self-report scale measuring perception of safety at home at night, on neighborhood stress during the day, and on neighborhood stress at night. Reliability of this measure was established by developers; reliability confirmed and validity established through pilot testing in this study.
|
Experiences of Life Stressors
|
Center for Youth Wellness–Adverse Childhood Experiences Youth Version (adapted) (43)
|
|
Caregiver experiences of childhood adversity
|
19-item self-report tool measuring childhood experience with stressful live events such as physical, emotional, and sexual abuse, physical and emotional neglect, household dysfunction, parental incarceration or death, and. Longitudinal studies are currently working to establish validity and reliability for this tool. Results reported in the current study are twelve of the nineteen total items that were asked of caregivers during the first interview.
|
Functional Impact of Life Events for Parents (FITS-P) (44)
|
|
Functional impact of exposure to toxic stress
|
4-item binary self-report scale assessing caregiver functional impact resulting from exposure to toxic stress, developed and validated for this study.
|
Experiences of Resilience and Personal Mastery
|
Connor-Davidson Resilience Scale (CD-RISC) (45)
|
|
Resilience
|
25-item Likert self-report scale of resilience widely used across service systems and populations with established psychometric properties including test-retest and internal consistency reliability, and convergent, and divergent validity with a wide breadth of established measures.
|
Pearlin Mastery Scale (46)
|
|
Parental mastery
|
7-item Likert self-report scale assessing individual mastery defined as the extent to which individuals perceive their life as under their own control. This scale assesses beliefs about mastery over current circumstances, future possibilities, and ability to solve problems independently. This scale has established internal consistency and test-retest reliability, and correlations with other scales and variables (46,47,48).
|
Parenting Experiences
|
Infant Development Questionnaire (IDQ) (49)
|
|
Infant development knowledge
|
15-item binary-response self-report measure of maternal knowledge of infant development. Psychometric properties of this measure have not been established.
|
Parental Stress Scale (PSS) (50)
|
|
Parenting stress
|
18-item Likert self-report measure of stress in parenting. Psychometric properties for this scale are robust, suggesting that the scale has sufficient internal consistency reliability, convergent validity with other parenting stress measures, and divergent and predictive validity.
|
Parenting Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO) (7)
|
|
Development-ally supportive parenting behaviors
|
29-item brief observational measure of four positive parenting practices: affection, responsiveness, encouragement, and teaching. This tool has been shown to be reliable and valid for use with parents (51)
|
Child Health and Development
|
Brief Infant-Toddler Social and Emotional Assessment (BITSEA) (52)
|
|
Child social-emotional development
|
42-item self-report screening tool for identifying child social-emotional and behavioral problems and delays among children ages 12 through 36 months of age. The BITSEA is a well-established tool with robust psychometric properties, including excellent test-retest reliability, concurrent and predictive and discriminant validity and acceptable specificity (53).
|
National Survey of Children’s Health (Adapted) (54)
|
|
Child health status
|
The NSCH is designed to produce national and state-level data on the physical and emotional health of children 0 - 17 years old in the United States. Number of items vary due to response but estimated average time for collection of data per survey is 2.5 hours. Psychometric information is generally not available for the 14-month wave, but is for the 24- and 36- month waves.
|
Access to Social Support and Resources
|
Health Families Parenting Inventory Mobilizing Resources Subscale (55)
|
|
Caregiver ability to access and use community resources
|
5-item Likert subscale of the 63-item self-report measuring assessing parenting domains. Assesses caregiver ability to access and use community and societal resources. The full scale has well-established psychometric properties, including construct validity, internal consistency reliability, and promising results related to sensitivity to change (56).
|
DULCE Social Connectedness (57)
|
|
Social connections in the community
|
2-item self-report scale assessing caregiver access to emergency care and support when needed. Psychometric properties of this scale have not been tested. (57).
|
Early Head Start Community-Based Partner Measure (58)
|
|
Use/access to community-based services
|
A subset of items adapted from the 36-month exit interview conducted by Mathematica Policy Research, Inc. and the U.S. DHHS Administration on Children, Youth, and Families as part of the 2001 Early Head Start Evaluation. While not psychometrically validated, this measure provides an established means of inquiring about connection to a variety of community-based family support programs.
|
Help Me Grow Protective Factors Survey (adapted) (59)
|
|
Protective factors
|
10-item self-report measure of caregiver knowledge of child needs, perceived access to support, and functional and relational coping. We pilot tested this measure for this study and established construct validity and internal consistency reliability (α = .93).
|
Patient Engagement with Clinic
|
Clinic Inclusivity and Regard for Parents Scale (CIRP)
|
|
Inclusion in primary care decisions
|
11-item self-report Likert scale assessing parenting perceptions of engagement and inclusion in healthcare decision-making for their children. The CIRP was developed and pilot tested for this study. We established construct validity as well as internal consistency reliability (α = .93).
|
Parent Patient Activation Measure (PPAM) (60)
|
|
Caregiver activation in pediatric primary care
|
13-item self-report Likert scale that measures healthcare activation – or ability to manage a child’s health and healthcare. The PPAM was developed based on the Patient Activation Measure (61), which has well-established psychometric properties of internal consistency and test-retest reliability, criterion and construct validity.
|
Clinic Staff Measures
|
Maslach Burnout Inventory, Health Provider Version (62)
|
|
Professional burnout
|
50-item scale derived from the Human Services Survey for Medical Personnel assessing degree of emotional exhaustion, depersonalization, and personal accomplishment in work. This measure is validated by the extensive research that has been conducted for more than 35 years with this tool.
|
AAP Periodic Survey of Fellows (adapted) (63)
|
|
Screening and referral practices to address SDOH
|
This national survey has collected more than 75,000 pediatrician responses on hundreds of topics to inform policy, education, and advocacy. This measure is validated by the extensive research that has been conducted for more than 20 years with this tool.
|
Electronic Health Records
The study used electronic health record (EHR) and Medicaid administrative data to examine trends in quality of care, health care utilization and financial impact of the innovations. Outcomes assessed include: health care quality (adherence to immunization and well-child visit schedules; developmental, maternal depression, and lead screening; and continuity of care); ambulatory care-sensitive hospitalizations; avoidable Emergency Department (ED) visits; and urgent care use among children served in the participating clinics. These data were used to analyze Medicaid spending on service use (recommended preventive versus urgent care/hospitalization) to inform the cost value of the innovations. We extracted the EHR and Medicaid data for all children age birth to two served in the participating study clinics from January 1, 2014 (corresponding with implementation of the Affordable Care Act) to February 29, 2020 (corresponding to study end). For each outcome using these data, analyses compare children’s health care and ED use before and after implementation of the innovations, as well as outcomes for children who did and did not receive the innovations.