Feasibility of Delivering Parent-Implemented NDBI Interventions in Low Resource Regions:&nbsp;A Pilot Randomized Controlled Study

doi:10.21203/rs.3.rs-105121/v1

Download PDF

Research

Feasibility of Delivering Parent-Implemented NDBI Interventions in Low Resource Regions: A Pilot Randomized Controlled Study

https://doi.org/10.21203/rs.3.rs-105121/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 05 Jan, 2022

Read the published version in Journal of Neurodevelopmental Disorders →

You are reading this latest preprint version

Background: This implementation feasibility study was conducted to determine whether an evidence-based parent -implemented distance-learning intervention model for young children at high likelihood of having ASD could be implemented at fidelity by Part C community providers and by parents in low resource communities.

Methods: The study used a community academic partnership model to adapt an evidence-based intervention tested in the current pilot trial involving randomization by agency in four states and enrollment of 35 coaches and 34 parent-family dyads. After baseline data were gathered, providers in the experimental group received up to 12 hours of training while control providers received six webinars on early development. Six months of intervention with children-families then followed, concluding with data collection. Regression analyses were used to model outcomes of the coach behaviors, the parent fidelity ratings, and child outcomes.

Results: A block design model-building approach was used to test the null model followed by the inclusion of group as a predictor, and finally the inclusion of the planned covariates. Model fit was examined using changes in R²and F-statistic. As hypothesized, results demonstrated significant gains in (1) experimental provider fidelity of implementation compared to the control group; and (2) experimental parent fidelity of implementation compared to the control group. There were no significant differences between groups in child developmental scores.

Conclusions: Even though the experimental parent group averaged less than 30 minutes weekly with providers in the 6 months, both providers and parents demonstrated statistically significant gains with moderate effect sizes compared to control groups. Since child changes in parent mediated models are dependent upon the parents’ ability to deliver the intervention, and since parent delivery is dependent upon providers who are coaching the parents, these results demonstrated that two of these three links of the chain were positively affected by the experimental implementation model. However, lack of significant differences in child group gains suggests that further work is needed on this model. Factors to consider include amount of contact with the provider, amount of practice children experience, amount of contact both providers and parents spend on training materials, and motivational strategies for parents, among others.

Trial registration: Registry of Efficacy and Effectiveness Studies: #4360, registered 1xx, October, 2020 – Retrospectively registered, http://

Neurobiology of Disease

early intervention

ASD

parent implemented interventions

parent coaching

implementation research

ESDM

Specific interventions for young children with or at high likelihood for autism spectrum disorders (ASD) demonstrate powerful effects in reducing intellectual impairment, improving social communication and language development, and improving social skills, when initiated in early childhood (Fuller et al, 2020, Fuller & Kaiser, 2019, Hampton & Kaiser, 2016, Sandbank et al., 2020). However, many of these interventions are not implemented well within community settings due to technical aspects of the models, delivery intensity, and precision of intervention methods. In fact, very few empirically supported models for toddlers with ASD have demonstrated efficacy when assessed via community delivery with a recent paper finding that children with ASD who receive intervention in community settings have less favorable outcomes than children who receive intervention in clinical/University settings. (Nahmias et al., 2019). This might be because the community systems that serve young children often involve low income and culturally diverse areas and interventions have not been adapted to fit the needs of families in these areas. These characteristics, combined with low funding rates, low service intensity, and staffing difficulties, make it difficult to implement evidence-based practices (EBP) at fidelity. Yet, Part C, the public early intervention system for children under age three in the United States is, by its public and noncategorical nature, the most likely source of early intervention (EI) for the nation’s young children with signs of ASD.

The Part C philosophy focuses on having providers support parents to provide intervention for their child during everyday activities. Children with ASD benefit the most from interventions that include parents (Hampton & Kaiser, 2016), and parent participation in EI is predictive of long-term outcomes (Kim, Bal, & Lord, 2018). However, too seldom do Part C providers use evidence-based parent coaching methods. Part C providers tend to provide direct intervention services to children (Campbell & Sawyer, 2007), which allows for little carryover into daily life and does not realize the intent of Part C services for family learning (Aranbarri, et al., 2017). Part C providers consider young children at high likelihood of ASD to be the most difficult and discouraging children to serve, due to their intense intervention needs, difficulty engaging with the provider, and poor progress. Given the significant cost of educating children with ASD (Amendah, et al., 2011), and the importance of providing high-quality intervention at an early age, their limited access to evidence-based practices (EBP) in Part C EI is a major concern.

Recently, there have been some attempts to move evidence-based, parent-implemented autism interventions into early intervention systems with some early success (Stahmer, Rieth et al., 2019; Rooks-Ellis et al, 2020). Researchers have partnered with community providers to train them to use parent coaching strategies to teach parents a Naturalistic Developmental Behavioral Intervention (NDBI; Schreibman et al., 2015) that fit the context of the community (Brookman-Frazee et al., 2012). These interventions show promise for improving social communication outcomes in children with high likelihood of having ASD, however, samples are small and more data are needed regarding how to increase access to under resourced communities.

Efforts are still needed to scale out the use of evidence-based, parent implemented NDBI in low resource, Part C agencies. In order to address this need we worked with research community collaboratives in six states to adapt the Early Start Denver Model’s (ESDM) parent coaching strategies for use in low-intensity settings with Part C providers having limited experience with autism. ESDM is one of the very few comprehensive EI models that have been validated and replicated in multiple published, randomized trials (e.g., Dawson et al., 2010; Rogers et al., 2019). A recent meta-analysis of 12 controlled ESDM studies, found significant effects of ESDM on cognition and language compared to usual care groups, even though most of the studies involved low-intensity (1 hour per week) or group services delivered by parents or professionals (Fuller et al., 2020). Multiple studies have examined the effects on parents and children of parent implemented ESDM (P-ESDM) and demonstrated parent fidelity to the techniques and accelerated child learning in language, imitation and play (Vismara et al, 2009, 2010, 2011, 2012, 2013).

ESDM was collaboratively adapted for under resourced Part C communities by a multidisciplinary group of providers, funding agency representatives, parents, and researchers who provided feedback after review of ESDM manuals and other written materials. Adaptations involved greatly shortening and streamlining training materials and providing them asynchronously via distance learning; simplifying and shortening procedures for developing short term intervention objectives and progress monitoring tools, and creating video modules that described and illustrated the key strategies for parents to use with children through cartoons and parent -child videos. Additional adaptations addressed (1) community values (rural Colorado, rural Alabama, rural California, Montana, Arizona, and urban Philadelphia); (2) the limited time Part C providers have for learning, planning, and data collection, (3) the need to reach families with attractive and practical brief audio-visual learning materials that could be accessed through their phones, (4) the need to use a flipped classroom educational approach grounded in the principles of adult learning for flexible provider training, (5) methods for integrating ESDM approaches within the existing Part C Individual Family Support Plan (IFSP) and delivery approaches, and (6) the very limited service intensity delivered in these communities (as low as one one-hour visit per month). The resulting model was named the Community ESDM, or C-ESDM.

We used an iterative process to develop both the C-ESDM provider training approach and the parent learning materials from three sources: (1) experimental data using component analysis to examine key components of ESDM, (2) survey data from a multi-state, multi-level survey aimed at better understanding Part C services (Aranbarri et al., in preparation), and (3) ongoing discussions with our implementation teams. The provider training program included methods of measuring child, parent and provider progress, provider training materials, on-line parent lessons and materials. The on-line materials, “Help is in Your Hands” (HIIYH; www.helpisinyourhands.org), include four modules with 4 lessons per module focused on narrated video examples of families using the strategies during daily routines at home. Modules cover the following topics: (1) Increasing Children’s Attention to People (Positioning; Following the leader; Finding and making attention magnets; Child comfort zone); (2) Increasing Children’s Communication (Talking bodies; Responding to child body language; Gestures and sounds; Following and leading); (3) Creating Joint Activity Routines (Building Joint Activities in Four Easy Steps; Variations on the theme; Joint activity routines without toys); (4) ABCs of learning (A = antecedents; B = behavior; C = Consequences). All provider training activities and parent coaching materials were made available on-line and also covered during twice monthly one-hour provider interactive webinars.

The current study used a small randomized trial to test the feasibility and promise of the adapted model for use with Part C providers and families who had a young child with social communication challenges considered at high likelihood of a future autism diagnosis. This study aims to test the effects of this low intensity training approach for Part C providers on three groups: (1) providers’ use of parent coaching strategies, (2) parents’ use of interactive strategies, and (3) toddlers’ developmental skills.

The current pilot study examined the effectiveness of C-ESDM delivered in Part C systems across 4 states: Alabama, California, Colorado, and Pennsylvania. Recruitment began at the agency level, with providers nested within agencies, and families recruited from participating providers’ caseloads. Agencies were recruited via outreach from University partners in each state. Eligible agencies served children 0–3 through their state’s Part C Program. Video calls were scheduled with all potential providers at an agency, wherein study details were described. Interested providers were subsequently contacted by study staff and formally screened via phone or video call. Each agency was randomized to either the Community Early Start Denver Model (C-ESDM) or an active comparison group (All About Young Children; AAYC) immediately upon enrollment, using a matched pair, cluster-randomization procedure. After enrolling, providers recruited eligible families from their existing caseload. A study flyer and recruitment video were available to provide interested families more study details. Interested families were contacted by the study coordinator via phone and eligible families consented and enrolled electronically. The current study focused on evaluating the impact of C-ESDM on 3 levels: providers’ use of parent coaching strategies, parents’ use of interactive strategies, and toddlers’ developmental level.

Participants

Thirteen agencies, 35 providers (all female), and 34 families enrolled in the project. Seven agencies (14 providers) were randomized to the C-ESDM and six agencies (17 providers) were randomized to the comparison group. One agency (with two providers) and an additional provider at a different agency withdrew after assignment to the comparison group before providing any demographic or intake data. One provider (C-ESDM) provided intake and demographic data but withdrew before attending any webinars. One provider (comparison group) withdrew after completing demographic information but did not provide an intake session. In all, this left a final sample of 32 providers reporting demographic information and 31 providers with at least one provider fidelity data point. (See Table 1 for demographic information about providers.

Table 1

Provider Demographic Characteristics, by State and Group, shown as percentages of the group.
Measure		State				Treatment Group
		AL (n = 4)	CA (n = 8)	CO (n = 11)	PA (n = 9)	AAYC (n = 14)	CESDM (n = 18)
Provider Race/Ethnicity (%)
	White	50	75	100.0	100	92.9	83.3
	Black/African-American	50	-	-	-	-	11.1
	Prefer not to answer	-	25	-	-	7.1	5.6
	Hispanic or Latina	-	25	-	-	14.3	-
Highest Education (%)
	Associate	-	-	9.1	11.1	14.3	-
	Bachelor	-	25	9.1	33.3	21.4	16.7
	Master	100	62.5	81.8	55.6	64.3	77.8
	Doctorate	-	12.5	-	-		5.6
Typical Intensity of Services Provided
	1–2 hours per month	100	50	-	-	-	44.4
	3–5 hours per month	-	50	90.9	100	85.7	55.6
	More than 5 hours per month	-	-	9.1	-	14.2	-

Agency eligibility criteria included: 1) agency receives some Part C funding; 2) agency serves low income families (defined as below the state mean income; 3) agency provides low-intensity services (fewer than 15 hours per week); 4) agency has at least two providers without previous ESDM training willing to participate in the study. Provider eligibility criteria included: 1) employed as an early interventionist at a participating agency; 2) no previous training in ESDM; 43 serving or will serve one or more children with social-communication delays with high likelihood of ASD. Providers’ formal titles varied, but most were credentialed professionals working as early educators (early childhood special educators, special instructors or developmental interventionists) or allied health specialists (Speech-Language Pathologists, Physical and Occupational Therapists). Inclusion criteria for families and children included: 1) child chronological age between 12 and 30 months at study intake; 2) child’s provider is concerned about possible ASD and child meets risk criteria on either the Modified Checklist for Autism in Toddlers, Revised (M-CHAT-R; Robins, Fein, Barton, & Green, 2001) or Infant-Toddler Checklist (ITC; Wetherby & Prizant, 2003); 3) child is ambulatory with unimpaired hand use; 4) child does not have significant motor, medical, vision, or hearing problems or genetic conditions associated with ASD; 5) child receives fewer than 10 hours per week of early intervention (including the EI agency and other intervention sources such as applied behavior analysis); 6) English is used at least 60% of the time in the home and parent is able to consent and complete questionnaires in English; 7) participating caregiver is child’s legal guardian; 8) participating caregiver is willing to attend scheduled intervention sessions with participating provider; 9) parent has not previously received ASD-specific parent coaching; 10) family income is below the state mean. In response to recruitment challenges, eligibility criteria for children and families were changed midway through the study to allow for 1) increased family income ( in the last year we removed all income restrictions and 2) removal of requirement involving (1) provider concerns about ASD risk and (2) requirement that children meet ASD risk criteria on a screening tool. Even so, the enrolled children did in fact show ASD risk on screeners. Four children were screened using the ITC; all met concerns criteria. Twenty-five of the remaining children were screened using the M-CHAT-R and scored with “high” or “moderate” ASD concerns. There were no statistically significant differences between the intervention and comparison groups related to sociodemographics (proportion non-white, maternal education greater than high school, or income of $50,000 or more). Demographic characteristics of parents and children in each state are presented in Table 2.

Table 2

Child and Family Participant Demographic Characteristics, by State and Group
Measure		State				Treatment Group
		AL (n = 6)	CA (n = 8)	CO (n = 13)	PA (n = 7)	AAYC (n = 14)	CESDM (n = 20)
Child Age at Enrollment (M, SD)		25.02, 4.58	24.98, 4.42	26.84, 3.77	25.51, 2.85	26.8, 2.35	25.11, 4.52
Child Sex (% male)		83.3	37.5	69.2	100	71.4	70.0
Child Race/Ethnicity (%)
	White	16.7	100	76.9	28.6	64.3	60.0
	Black/African-American	83.3	-	15.4	42.9	28.6	30.0
	Asian	-	-	-	14.3	-	5.0
	Multiple	-	-	7.7	14.3	7.1	5.0
	Hispanic or Latino	-	25	23.1	42.9	28.6	20.0
Maternal Education (%)
	High School/GED/Vocational	16.7	62.5	23.1	57.1	42.9	35.0
	Some College	16.7	25	46.2	28.6	28.6	35.0
	College Degree	16.7	-	30.8	14.3	28.6	10.0
	Graduate Degree	50	12.5	-	-	-	20.0
Family Income (M, SD)		43,966; 26,472 Range: 8,800–70,000	58,875; 31,534 Range: 0–96,000	61,769; 42,528 Range: 20,000-170,000	26,857; 18,685 Range: 8,000–58,000	52,357; 45,771 Range: 0–170,000	49,640; 26,504 Range: 8,800–96,000

Procedures

Training Procedures

Intervention Group. The C-ESDM intervention delivery consisted of five components: 1. providers’ real-time webinars with trained ESDM parent coaches and in self-study, 2. providers’ group procedural learning through video review with the coach, 3. parents’ real-time learning during parent coaching with their providers, 4. parents’ independent learning through HIIYH videos and materials including parent manual, and 5. child learning through interactions with their parents within everyday activities.

Core learning materials:

Parent manual “An Early Start for your Child with Autism” (Rogers et al., 2012);

Website “Help Is in Your Hands” with its narratives, videos, and exercises (www.helpisinyourhands.org);

Provider video materials, training sessions and tools on the Help Is in Your Hand website;

Parent Refrigerator Lists, which cover the main topic of each week’s intervention session;

P-ESDM Parent Fidelity of Implementation Scale;

P-ESDM Coaching Fidelity of Implementation Scale;

Coach’s list of child objectives from the child’s IFSP broken down into 4–5 learning steps;

Child session data sheet capturing progress through objectives and learning steps.

C-ESDM providers received access to learning materials, webinars, and video coaching via telehealth in the C-ESDM strategies and in how to coach parents to implement these strategies. The training program included the materials listed above: methods of measuring child, parent and provider progress, provider training materials, and on-line parent lessons and materials. Prior to training, providers completed brief on-line knowledge assessments related to understanding of adult learning principles, early signs of autism, and parent coaching strategies. If they did not receive a score of 80% or better they reviewed brief videos introducing these concepts prior to beginning training. They also reviewed a brief introduction to the parent materials.

Providers attended four weekly telehealth group meetings that included a concept presentation, video examples, and discussions with 3–4 other providers from their agency. Meeting leaders were certified ESDM trainers who had developed the C-ESDM procedures and materials. Session topics included: (1) an introduction to HIIYH and Parent Coaching; (2) Parent Coaching structure and strategies; (3) building specific treatment objectives from IFSP goals and simple tracking methods for child progress; and (4) supporting parent learning. Each meeting included both didactic information as well as interactive activities related to the topic. Between session activities included practice using materials provided (e.g., coding intervention fidelity; practicing with data collection). Once providers completed the initial webinar, they began to use HIIYH with an enrolled family and met monthly for group supervision with their meeting leader. Enrolled parents received access to the HIIYH parent materials and the ESDM parent manual and providers could use any of the video and written materials and strategies during their Part C early intervention sessions with enrolled families.

Comparison group. The comparison group received directions to access publicly available online modules (All About Young Children: AAYC, CA Dept Ed 2013; allaboutyoungchildren.org) of high quality covering early developmental milestones from birth to 60 months in 5 domains: 1) social-emotional development, 2) language development and literacy; 3) number sense; 4) physical development; and 5) approaches to learning. The website included videos with examples of strategies to promote child development that could be viewed by providers and parents. Providers in the comparison group met monthly (for 6 months) via telehealth with a leader (developmental psychology PhD and early childhood specialist) who reviewed the materials covered and provided structured discussion on each topic but did not offer concrete suggestions for either parent coaching strategies or child interaction strategies. Providers could use the materials in their Part C intervention in any way they wished.

Assessment Procedures

Provider Assessments. Providers completed online questionnaires and session videos at study enrollment and exit (6 months later, or whenever their final family completed intervention). The initial provider video was taken of a session with a consented Part C family who was receiving ongoing intervention with that provider, in order to sample provider’s typical parent-coaching strategies used. During a Part C intervention session. After training and initiation of intervention with the project children enrolled in experimental or comparison groups, providers recorded each intervention session on a project-supplied iPad and uploaded the session videos to a secure, HIPAA-compliant website. The final video uploaded by each provider was selected as their “exit” video. Note that families/children in provider initial videos were not necessarily the same families/children that providers worked with and filmed for the exit videos. Raters naïve to timepoint coded provider fidelity from each intake and exit video. Analysis of provider change in fidelity focused on the initial and the last available session of the provider with the target child. The mean number of weeks between the provider initial and exit intervention sessions was 17.23 weeks (SD 7.03), which did not differ between the groups (p > .49). To track number of hours delivered, providers were asked to complete weekly online questionnaires indicating whether a session was scheduled with each family and whether it took place as scheduled.

Parent and Child Assessments

We reached out to our University partners (participating in the research community partnership that developed C-ESDM) in each state for help with child assessments and recruited seven assessors (all female), including graduate students (n = 1) and early intervention professionals (n = 6) working in their local communities. These assessors were hired as contractors (not participants) for the project and were naïve to provider group assignment. The study team sent each assessor a kit with a recording device, forms and necessary toys and stimuli to complete the assessments. The child assessments included two primary components: 1) A parent-child interaction and; 2) the ESDM Infant-Toddler Curriculum Checklist (IT-CC; Rogers et al., 2020 described below). For the parent-child interaction, the assessor asked the parent to play with their child in their typical way, first without any objects and then with toys either from home or from a selection of toys the assessor brought to support the curriculum assessment. The parent-child interaction lasted up to 20 minutes. The assessor then completed the IT-CC, described in detail below. Each assessment was digitally recorded for later scoring of parent and child behaviors. In addition to these live interactions, some parent measures were completed online by the parents. For the very few parents who did not complete the on-line measures, the assessors provided the surveys as paper and pencil measures.

Assessor training procedures included one initial telehealth training with a project member on the assessment procedures. Providers then submitted practice tapes for feedback on administration and scoring until they reached fidelity benchmarks specified for the IT-CC. Following this training, assessors began seeing families. Family contact information was provided to assessors via secure, HIPAA-compliant messaging, and assessors contacted families directly to schedule at a time that was mutually convenient. Assessments were scheduled in families’ homes and lasted approximately 1.5 hours. Assessors scored the IT-CC live at the time of assessment administration and submitted copies of their scores and videos of the assessment sessions via a secure website so that their scores could be checked for accuracy by a trained member of the research team. If an item was missing or incomplete, the assessor was contacted directly by the Project Coordinator to clarify.

The entire process of recruiting and training providers and assessors, identifying and enrolling eligible families and children, conducting the intervention, and gathering final data took approximately one year. Agencies were enrolled in rolling fashion and all the activities related to conducting the intervention study other than coding and data analysis were completed in a two-year period.

Measures

Infant-Toddler Checklist (ITC; Wetherby & Prizant, 2003). The ITC is a 25-item checklist that assesses infants’ language, communication, and play skills, and probes for parent concern. Empirically derived cut-offs for concerns range are available for infants 6 through 24 months. The ITC was used as an eligibility screener.

Modified Checklist for Autism, Revised (M-CHAT-R, Robins, Fein, Barton, & Green, 2001). Is a 20-item checklist designed to screen for ASD. It provides empirically derived cut-offs for concern and referral recommendations. The M-Chat was used as an eligibility screener.

ESDM Fidelity Checklist (Rogers & Dawson, 2010). The ESDM Fidelity Checklist consists of 12 items that are each given a score between 1 and 5, with 5 representing more frequent and higher quality use of each ESDM strategy and a total possible range of scores from 12 to 60. Trained coders naïve to group and timepoint scored parents on the Fidelity Checklist from parent-child interactions filmed at the assessments. Coders scored two routines from each assessment: one routine without toy play and one with toy play. Routines had to last a minimum of one minute to be coded. Scores were averaged across items for two routines for an average total fidelity rating. Twenty-nine percent of videos were independently coded by both coders for reliability. Intraclass correlation coefficients indicated high reliability: ICC = 0.85 (CI:0.62–0.95).

Coaching Practices Rating Scale (CPRS, Rush & Shelden, 2011). A modified version of the Coaching Practices Rating Scale (Rush & Shelden, 2011) was used to evaluate provider fidelity of implementation. Each of the 13 items were rated on a binary scale of present or absent, and these scores were summed for a total of 13 possible points. These behaviors were rated by two coders naive to timepoint and group assignment. Twenty percent of videos were independently coded by both coders for reliability. Intraclass correlation coefficients indicated high reliability: ICC = 0.92 (CI: 0.17–0.98).

ESDM Infant-Toddler Curriculum Checklist (IT-CC; Rogers et al., 2020). The IT-CC is a criterion-based measure of early development that spans the developmental range from 8 to 30 months and is adapted from the Early Start Denver Model Curriculum Checklist (ESDM; Rogers & Dawson, 2010). The IT-CC consists of 136 items organized in 9 developmental domains: Gestures Understood, Words Understood, Gestures Produced, Words Produced, Joint Attention, Dyadic Engagement, Imitation, Cognition, and Play Skills. Items are assessed during semi-structured play- and routines-based interactions carried out over approximately 90 minutes using a standard set of play materials. Each IT-CC item is rated as ‘acquired’, ‘partial/prompted’, or ‘unable or unwilling’, based on child behavior during play-based interactions throughout the entire assessment, as well as parent report. On the IT-CC, a score of ‘acquired’ on a given item represents a mastery level of that skill and is credited. No other score receives credit. The Cognitive domain was not utilized during the current study after pilot testing indicated the additional required materials were too burdensome for assessors to travel with into families’ homes. Thus, final scores for this study consist of one point per ‘acquired’ item, for a total score out of 124 possible points, expressed as a raw score (IT-CC Total Score). A team of gold-standard coders at the primary university site, naive to timepoint and group assignment, scored the IT-CC from videos. Their scores were used for all analyses, rather than the home assessors’ scores, because of the potential for assessors to become unblinded to family/provider group assignment. Intraclass correlation coefficients of assessors and gold standard coding team indicated high reliability: ICC = 0.93 (CI: 0. 89 to 0.95).

Analysis

A series of regression analyses were used to model outcomes of the coach behaviors, the parent fidelity ratings, and child outcomes. Preliminary analyses revealed that there were no site differences on intake variables (p $\ge$0.06). Therefore, a non-nested approach was taken. A block design model-building approach was used to test the null model (accounting only for pretest), followed by the inclusion of group as a predictor to address the primary research question, and finally the inclusion of the planned covariates. Variables that were significant predictors of outcome were retained in the model. Model fit was examined using changes in R² and the F-statistic. The first model tested the impact of the C-ESDM training on the outcome of coaching behaviors. The providers’ initial level of Coaching Practices fidelity and group assignment were included, in that order, to understand the effect of group assignment on provider fidelity. The second model tested the impact of the C-ESDM intervention on parent fidelity. The parent’s initial level of fidelity and group assignment were included, in that order, to understand the effect of group assignment on parent fidelity. The third model tested the effect of the C-ESDM parent coaching intervention on child outcomes. The child’s pretest score, group assignment, and changes in parent fidelity were included in the model in that order, to understand the effect of group assignment and the possible contribution of changes in the changes in parent fidelity on child outcomes. All interaction terms between pretest variables and group assignment were examined. All statistical analyses were completed using SPSS Statistics V. 26.

Providers in both conditions attended an average of 78.36% of possible webinars/coaching contacts, which did not differ by group (t(29) = 0.86, p = 0.93). This translated to a mean of 9.71 hours of webinar training/supervision sessions attended (SD = 2.11) by the C-ESDM group and 5.71 hours (SD = .47) attended by the comparison group providers. Provider-reported weekly session attendance data indicated no group differences in the number or proportion of family sessions completed (sessions completed: Mean_C−ESDM = 11.09, SD_C−ESDM = 5.99, Mean_comparison = 14.25, SD_C−ESDM = 5.68, t(15.72) = 1.71, p = 0.26; percent sessions attended: Mean_C−ESDM = 54.08%, SD_C−ESDM = 20.28, Mean_comparison = 64.33,% SD_comparison = 12.66, t(16.73) = 1.35, p = 0.19).

Coaching Outcomes

Regression analyses of coaching behaviors showed that initial coaching behavior rating was not significantly related to exit coaching behaviors (β = 0.24, se = 0.27, p = 0.38). To test the hypothesis that inclusion in the C-ESDM intervention would result in the use of more coaching behaviors, group was entered into the null model. Participation in the C-ESDM group predicted a significant increase in coaching behaviors compared to the control group (β = 4.30, se = 1.40, p = 0.007), with a significant improvement in model fit (R² change = 0.34, F = 9.39, p = 0.007). Observed means and standard errors for three primary outcome variables are shown in Table 3.

Table 3

Mean (SE) of outcome variables at initial and exit assessments.
	C-ESDM		Comparison (AAYC)
	Pre (n = 20)	Post (n = 13)	Pre (n = 12)	Post (n = 10)
Coaching Score	4.83 (0.62)	7.67 (0.74)	3.54 (0.93)	3.25 (1.20)
Parent Fidelity	3.24 (0.12)	3.66 (0.15)	3.21 (0.12)	3.15 (0.14)
Child IT-CC Total Score	41.40 (5.45)	59.85 (9.63)	44.75 (7.60)	62.50 (10.27)
Note. ANOVAs showed no significant differences between groups at intake (p≥0.24).

Parent Fidelity Outcomes

The C-ESDM group attended on average 11.09 sessions (sd.5.99), which was 54.08% of scheduled sessions (SD 20.28%), and the comparison group attended on average 14.25 (SD = 5.68) sessions, 64.33% (SD = 12.66%) of scheduled sessions. The groups did not differ significantly on number of sessions (t = 1.71, df = 15.72. p = 0.26) or percentage of scheduled sessions (t = 1.35, df = 16.73, p = 0.19). The results of the regression analyses showed that pretest parent fidelity was significantly related to posttest fidelity (β = 0.48, se = 0.22, p = 0.04). To test the hypothesis that inclusion in the C-ESDM intervention would result in higher parent fidelity ratings, group was entered into the null model. Participation in the C-ESDM group predicted a significant increase in parent fidelity compared to the control group (β = 0.520, se = 0.20, p = 0.02), with a significant improvement in model fit (R² change = 0.19, F = 6.40, p = 0.02).

Child Outcomes

The results of the regression analyses indicated that initial IT-CC total score was significantly related to exit score (β = 1.16, se = 0.14, p < 0.01). To test the hypothesis that inclusion in the C-ESDM intervention would result in higher child scores, group was entered into the null model. Participation in the C-ESDM group did not result in a significantly greater change in child scores compared to the comparison group (β = 1.17, se = 7.32, p = 0.87). Initial scores were significantly related to exit scores (p < 0.01). Changes in parent fidelity were not related to child outcomes (β = 5.49, se = 6.48, p = 0.40). Interactions between initial variable data and group assignment were examined for all analyses but these interactions were not significant.

Brief summary. This implementation feasibility study research-community partnership approach (Brookman-Frazee et al., 2012) was designed and executed in order to determine whether an evidence-based parent-implemented distance-learning intervention model for young children with or at high likelihood of having ASD: (1) could be learned and implemented at fidelity by community providers after very brief group training, (2) whether community providers could coach parents in ways that effectively transmitted evidence based skills in an average of one contact per week or less, as measured by fidelity of implementation measures, and (3) whether children of parents receiving the parent coaching model would demonstrate positive benefits in comparison to children whose parents received information on child development only. The study used a three-phase model, beginning with input from community partners in six sites. Phase two involved a component analysis of the parent model to determine which of the strategies to emphasize, some pilot work to test the training and coaching intervention methods. Phase three involved this pilot - controlled trial involving randomization by agency in four states and enrollment of 35 coaches working in the Part C system and 34 parent-family dyads. Families of qualifying children (based on social communicative delays and ASD risk) were enrolled by their EI providers and initial baseline data on provider coaching, parent-child interactions, and child development were gathered. Providers in the experimental group then received up to 12 hours of training via webinars, group sessions, and asynchronous learning materials, followed by initiation of intervention with enrolled families. Comparison group providers received six webinars on various aspects of early development, followed by initiation of intervention with children-families.

After approximately six months of intervention at whatever schedule the agency typically delivered (ranged from 2 hours per week to 1 hour per month), video measures of provider interactions with the dyad and videos of parent interactions with child were collected again as was developmental information on children (collected by a naïve evaluator). Results demonstrated significant gains in fidelity to the coaching model of providers in the experimental group compared to those in the control group. Results also demonstrated significant gains in fidelity to the intervention strategies of parents in the experimental groups compared to those in the comparison group, supporting the primary and secondary hypotheses of the study. Gains in provider coaching fidelity were not related to baseline coaching scores; however, gains in parent intervention fidelity were related to baseline fidelity scores. There were no significant differences between groups in child developmental scores.

Implications

This study focused on adaptation of a well-tested intervention to fit the needs of public agencies, providers, and families in four low resource areas across the country, chosen because these settings have very limited services and the families often face many difficulties in accessing high quality intervention for their young children at risk for ASD. The sites involved both urban and rural settings in locations where intensive services for young children with ASD were not available, nor was expertise in ASD present in the agencies that participated. Involvement of community representatives in various sites allowed for needed guidance about the needs, strengths, values and priorities of providers and families in each region. Use of distance learning and self-instructional learning activities was necessary because of the very limited time allowed by agencies for provider training and because of the geographic distances involved. Providers in the C-ESDM group met in small groups with a project coach one hour every two weeks for the first three months of the project, tapering off to once monthly by month six. Community providers delivered all interventions with parents and children; the project coaches never interacted with the family nor did they provide direct coaching to the providers during sessions. To our knowledge, other parent-mediated implementation studies have not relied on local providers to implement the experimental intervention in low resource settings, nor have they relied on distance learning and such limited contact to teach the intervention to the coaches. Even though the research project coaches averaged less than 30 minutes weekly in contact with the provider group over a 6 month period, and no time at all with the parents, both providers and parents in the experimental group demonstrated statistically significant gains with moderate effect sizes compared to the comparison group. Since child changes in parent -mediated models are dependent upon the parents’ ability to deliver the intervention, and since parent delivery is dependent upon providers who are coaching the parents, these results demonstrated that both of these links of the chain were positively affected by the implementation model being tested here.

However, lack of child change as measured by experimenter-administered measures suggests that further work is needed on this model. Our group sizes were not large enough to analyze factors influencing child change. Factors to consider include amount of contact with the provider, amount of practice children experience, amount of parent time spent on learning, and motivational strategies for parents. One of the settings we worked in provided only one hour per month of contact, and if illness, schedules, or holidays required cancellation, no make-up sessions could be provided. Given our own findings regarding weekly parent-coaching visits (Rogers et al 2019), it is difficult to imagine that a parent could learn to embed helpful strategies into natural routines and maintain new learning for a young child with autism symptoms with only one hour per month of contact. Thus, follow-up research is needed to determine what factors are necessary for changes in parent interaction strategies to permeate child behavioral repertoires.

A recent replication of these methods, not yet published, in British Columbia, found similar positive results in provider and parent fidelity, as well as significant positive changes in parent questionnaire measures, though not standardized measures, of child progress on multiple measures of development and symptoms in the experimental group. Positive change measured on standardized measures from a parent-mediated intervention is a very high bar. Very few low intensity parent-mediated models have published positive child effects as measured by standardized developmental measures. However, since change on standard scores is widely considered the most rigorous evidence of child improvement, and since many studies of intensive autism intervention have shown that such change is possible, we find it important to continue to strive for this outcome as well.

Until replication of the C-ESDM model demonstrates positive child-level findings, additional research is needed to further develop and test this approach. However, the underlying principles – use of distance technology to transmit strategies from existing efficacious models to community providers and to parents - were both feasible and successful in this study, have been well documented in the literature (Ashburner, Vickerstaff, Beetge, & Copley, 2016; Shire, Baker Worthman, Shih, & Kasari, 2020; Sutherland, Trembath, & Roberts, 2018; Vismara et al., 2018; Vismara, Young, Stahmer, Griffith, & Rogers, 2009; Wainer & Ingersoll, 2015) and thus can be considered an evidence-based practice.

The data gathered from this pilot RCT, designed to determine the feasibility of enhancing use of evidence-based early intervention practices for the youngest children with or at risk for ASD living in low resource areas of the United States, suggests that provider evidence-based coaching practices can be significantly enhanced, with resulting significant enhancements in parent use of evidence-based interaction practices following implementation of low - cost distance learning activities and oversight for providers. Significant enhancement in parent interaction strategies occurred even though the experimental parent group averaged less than 30 minutes weekly with providers over a 6- month period. Since child changes in parent mediated models are dependent upon the parents’ ability to deliver the intervention, and since parent delivery is dependent upon providers who are coaching the parents, these results demonstrated that two of these three links of the chain were positively affected by the experimental implementation model. However, lack of significant differences in child group gains points out that further work is needed to increase effectiveness of the model.

NDBI: naturalistic developmental-behavioral intervention; ASD: autism spectrum disorder; EI: early intervention; EBP: evidence-based practice; ESDM: Early Start Denver Model; P-ESDM: parent-implemented Early Start Denver Model; IFSP: Individualized Family Service Plan; C-ESDM: Community Early Start Denver Model; HIIYH: Help Is In Your Hands; AAYC: All About Young Children; M-CHAT-R: Modified Checklist for Autism in Toddlers, Revised; ITC: Infant-Toddler Checklist; IT-CC: Infant-Toddler Curriculum Checklist; HIPAA: Health Insurance Portability Accountability Act; CPRS: Coaching Practices Rating Scale.

Ethics approval and consent to participate: This study was approved by University of California Davis Institutional Review Board Investigator: Rogers, Sally, PhD IRB ID: 780328-19. All providers and family members gave consent to participate and to be videorecorded.

Consent for publication: not applicable

Availability of data and materials: The datasets and unpublished materials used and/or analyzed on this study are available from the corresponding author on reasonable request.

Competing interests:

SJR receives royalties from Guilford Publishing Company for project manuals used in this study.

AS, MT, GY, EF, MP, AB,& EG: no competing interests

Funding: This research project was funded by a grant from the Institute for Education Science (R324A150211 PI: Rogers) which covered all costs associated with the design and execution of this study. Dr. Fuller’s time was supported by NIMH grant T32 MH07312. The project also received support from the MIND Institute IDDRC, funded by the National Institute of Child Health and Human Development (P50 HD103526 PI: Abbeduto).

Authors contributions: all authors read and approved the final manuscript.

SJR: contributed to the design of the study, provided training to the C-ESDM providers, designed the HIIYH website materials, helped choose measures and plan the analyses, and was the lead author on this paper.

AS: contributed to the design of the study, provided training to the C-ESDM providers, worked with Dr. Rogers on development of the HIIYH materials, helped choose measures and develop the paper introduction, provided input throughout the paper.

MT: delivered training to comparison group, trained assessors and supervised in-home assessments and naïve coders, and contributed to data analysis, interpretation, and writing of this manuscript.

GY: planned, supervised or conducted all analyses and contributed to methods and results sections of the manuscript.

EF: contributed to the data analysis and write-up of the results

MP, AB, & EG: contributed community perspectives that guided that design and execution of the study

Acknowledgements The authors wish to acknowledge the tremendous contributions of MaryKate Miller to the management of this study, our community partners in Alabama, California, Colorado, Montana, New Mexico, and Pennsylvania for their input into the design and development of HIIYH, and the providers, children and families who participated in the study.

Authors’ information – NI

Amendah D, Grosse SD, Peacock G, Mandell DS. The economic costs of autism: A review. In: Amaral D, Geschwind D, Dawson G, editors. Autism spectrum disorders. Oxford: Oxford University Press; 2011. pp. 1347–60.

Ashburner J, Vickerstaff S, Beetge J, Copley J. Remote versus face-to-face delivery of early intervention programs for children with autism spectrum disorders: Perceptions of rural families and service providers. Research in Autism Spectrum Disorders. 2016;23:1–14. https://doi.org/10.1016/j.rasd.2015.11.011.

Brookman-Frazee L, Stahmer AC, Lewis K, Feder JD, Reed SR. Building a research-community collaborative to improve community care for infants and toddlers at-risk for autism spectrum disorders. Journal of Community Psychology. 2012;40:715–34. doi:10.1002/jcop.21501.

Campbell PH, Sawyer LB. Supporting learning opportunities in natural settings through participation-based services. J of EI. 2007;29(4):287–305. doi:10.1177/105381510702900402.

CA Department of Education, State Advisory Council on Early Learning and Care. (2013). All about Young Children: Information for families on children’s early development. http://allaboutyoungchildren.org.

Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, Donaldson A, Varley J. Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics. 2009;125:e17–23.

Fuller EA, Kaiser A. 2019. The effects of early intervention on social communication outcomes for children with autism spectrum disorder: a meta-analysis. Journal of Autism and Dev. Disorders, 2020, 50(5), 1683–1700. Doi:10.1007/s10803-019-03927-z.

Fuller EA, Oliver K, Vejnoska S, Rogers SJ. The effects of the Early Start Denver Model for children with autism spectrum disorder: A meta-analysis. Brain Sciences. 2020;10:368. doi:10.3390/brainsci10060368.

Hampton LH, Kaiser AP. Intervention effects on spoken-language outcomes for children with autism: a systematic review and meta-analysis. J Intellectual Disability Research. 2016;60(5):444–63. Doi:10.1111/jir.12283.

Kim SH, Bal VH, Lord C. Longitudinal follow-up of academic achievement in children with autism from age 2 to 18. J of Child Psychology Psychiatry. 2018;59(3):258–67. doi.org/10.1111/jcpp.12808.

Nahmias AS, Pellecchia M, Stahmer AC, Mandell DS. Effectiveness of community-based early intervention for children with autism spectrum disorder: a meta-analysis. J of Child Psychology Psychiatry. 2019;60(11):1200–9. doi.org/: 10.1111/jcpp.13073.

Robins DL, Fein D, Barton ML, Green JA. The Modified Checklist for Autism in Toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord. 2001;31:131–44. https://doi.org/10.1023/A:1010738829569.

Rogers SJ, Dawson G. Early Start Denver Model for young children with autism: Promoting language, learning, and engagement. New York: The Guilford Press; 2010.

Rogers SJ, Dawson G, Vismara L, Zierhut C, Winter J, McCormick C, Talbott M. P-ESDM Infant-Toddler Curriculum Checklist. NY: Guilford Press; 2020.

Rogers S, Estes A, Lord C, Munson J, Rocha M, Winter J, Greenson J, Colombi C, Dawson G, Vismara LA, et al. A multisite randomized controlled two-phase trial of the Early Start Denver Model compared to treatment as usual. J Am Acad Child Adolesc Psychiatry. 2019;58:853–65. doi:10.1016/j.jaac.2019.01.004.

Rooks-Ellis DL, Howorth SK, Boulette S, Kunze M, Sulinski E. Effects of a parent training using telehealth: Equity and access to early intervention for rural families. Journal of Childhood Education Society. 2020;1:141–66. https://doi.org/10.37291/2717638X.20201242.

Rush DD, Shelden ML. The Coaching Practices Rating Scale. In: Rush DD, Shelden MS, editors. The Early Childhood Coaching Handbook. Baltimore: Paul H Brooks Publishing; 2011.

Sandbank M, Bottema-Beutel K, Crowley S, Cassidy M, Dunham K, Feldman JI, et al. Project AIM: Autism intervention meta-analysis for studies of young children. Psych Bulletin. 2020;146(1):1–29. doi:10.1037/bul0000215.

Schreibman L, Dawson G, Stahmer AC, Landa R, Rogers SJ, McGee GG, Halladay Aet al. Naturalistic developmental behavioral interventions: Empirically validated treatments for Autism Spectrum Disorder. J Autism Dev Disord. 2015;45(8):2411–28. https://doi.org/10.1007/s10803-015-2407-8.

Shire SY, Baker Worthman L, Shih W, Kasari C. Comparison of face-to-face and remote support for interventionists learning to deliver JASPER intervention with children who have autism. Journal of Behavioral Education. 2020. https://doi.org/10.1007/s10864-020-09376-4.

Stahmer AC, Reith SR, Dickson KS, Feder J, Burgeson M, Swearcy K, Brookman-Frazee L. Project ImPACT for toddlers: pilot outcomes of a community adaptation for autism risk. Autism. 2019;24(3):617–32. DOI:10.1177/1362361319878080.

Stahmer AC, Aranbarri A, Drahota A, Reith S. (2017). Editorial: toward a more collaborative research culture: Extending translational research science from research to community and back again. Autism, 21(3), 259–61. Doi:10.1177/1362361317692950.

Sutherland R, Trembath D, Roberts J. Telehealth and autism: A systematic search and review of the literature. International Journal of Speech-Language Pathology. 2018;20:324–36. https://doi.org/10.1080/17549507.2018.1465123.

Vismara LA, McCormick CEB, Wagner AL, Monlux K, Nadhan A, Young GS. Telehealth parent training in the Early Start Denver Model: Results from a randomized controlled study. Focus on Autism Other Developmental Disabilities. 2018;33:67–79. https://doi.org/10.1177/1088357616651064.

Vismara LA, Colombi C, Rogers SJ. Can one hour per week of therapy lead to lasting changes in autism? Autism. 2009;13(1):93–115. doi.org/10.1177/1362361307098516.

Vismara LA, Young GS, Stahmer AC, Griffith EM, Rogers SJ. Dissemination of evidence-based practice: can we train therapists from a distance? J Autism Dev Disord. 2009;39:1636–51. https://doi.org/10.1007/s10803-009-0796-2.

Vismara LA, McCormick C, Young GS, Nadhan A, Monlux K. Preliminary findings of a telehealth approach to parent training in autism. JADD. 2013;43:2953–69. doi.org/10.1007/s10803-013-1841-8.

Wainer AL, Ingersoll BR. Increasing access to an ASD imitation intervention via a telehealth parent training program. J Autism Dev Disord. 2015;45:3877–90. https://doi.org/10.1007/s10803-014-2186-7.

Wetherby A, Prizant B. Communication and Symbolic Behavior Scales. Baltimore: Paul H. Brookes; 2003.

Download PDF

Journal Publication

published 05 Jan, 2022

Read the published version in Journal of Neurodevelopmental Disorders →

Review #1 received at journal
06 Mar, 2021
Editorial decision: Minor revision
06 Mar, 2021
Review #2 received at journal
27 Feb, 2021
Reviewer #2 agreed at journal
28 Dec, 2020
Reviewer #1 agreed at journal
10 Nov, 2020
Editor assigned by journal
08 Nov, 2020
Reviewers invited by journal
08 Nov, 2020
Submission checks completed at journal
08 Nov, 2020
Editor invited by journal
08 Nov, 2020
First submitted to journal
07 Nov, 2020

You are reading this latest preprint version

Feasibility of Delivering Parent-Implemented NDBI Interventions in Low Resource Regions: A Pilot Randomized Controlled Study

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Participants

Procedures

Training Procedures

Assessment Procedures

Parent and Child Assessments

Measures

Analysis

Results

Coaching Outcomes

Parent Fidelity Outcomes

Child Outcomes

Discussion

Implications

Conclusions

Abbreviations

Declarations

References

Status:

Journal Publication

Version 1