Caregiver characteristics
Due to lower caregiver response rates than anticipated, most organizations (n=5) began implementing P4P earlier than anticipated to ensure timely completion of post-implementation procedures, resulting in a smaller number of pre-implementation surveys than intended. Of the 832 caregivers contacted for the pre-implementation survey, 180 (21.6%) were surveyed. Meanwhile, 262 (28.7%) of the 914 caregivers contacted for the post-implementation survey completed the measure. As shown in Table 1, the pre-implementation group had a smaller proportion of caregivers identifying as Hispanic/Latine and preferring Spanish and a larger proportion of caregivers preferring Haitian Creole compared to the post-implementation group. No other demographic differences were detected between groups.
Care team and clinic characteristics
Survey respondents (N=27) were largely medical assistants (37%, n=10) and physicians (30%, n=8), with the remaining 33% (n=9) reporting a different role. Additional demographic factors (e.g., age, race) were not collected in the staff survey. Meanwhile, of the 25 interviewees, 32% (n=8) were pediatricians, 24% (n=6) were medical assistants, and 20% (n=5) were in managerial
Table 1. Pre-implementation (N=180) and post-implementation (N=262) caregiver demographics
Demographic
|
Pre-implementation
|
Post-implementation
|
|
|
n (%)
|
n (%)
|
p-value
|
Age
|
.20
|
18-24
|
23 (13%)
|
44 (17%)
|
|
25-34
|
92 (52%)
|
143 (55%)
|
|
35-44
|
56 (31%)
|
69 (26%)
|
|
45+
|
7 (4%)
|
6 (2%)
|
|
Gender
|
.63
|
Female
|
173 (97%)
|
250 (95%)
|
|
Male
|
6 (3%)
|
12 (5%)
|
|
Race/Ethnicity
|
.03
|
White
|
15 (8%)
|
7 (3%)
|
|
Black or African American
|
17 (9%)
|
20 (8%)
|
|
Hispanic/Latine
|
146 (81%)
|
228 (87%)
|
|
Other*
|
2 (1%)
|
7 (3%)
|
|
Relationship to Child
|
.71
|
Parents
|
177 (98%)
|
256 (98%)
|
|
Grandparents
|
3 (2%)
|
3 (1%)
|
|
Extended Family Member
|
0 (0%)
|
2 (0.8%)
|
|
Non-related Caregiver
|
0 (0%)
|
1 (0.4%)
|
|
Single Parent Household
|
.64
|
Yes
|
52 (29%)
|
68 (26%)
|
|
No
|
126 (71%)
|
192 (74%)
|
|
Language
|
.02
|
English
|
53 (29%)
|
74 (28%)
|
|
Spanish
|
122 (68%)
|
188 (72%)
|
|
Haitian Creole
|
5 (3%)
|
0 (0%)
|
|
Note: A small number of caregivers did not report demographics for the following variables: Age (pre-implementation n=2), gender (pre-implementation n=1), and single parent household (pre-implementation n=2, post-implementation n=2)
*American Indian or Alaska Native, Asian or Asian American, Native Hawaiian/Pacific Islander, Other.
positions, with the remaining 24% (n=6) reporting different roles. Respondents tended to be middle-aged and older (44%, n=11), female (52%, n=13), white (56%, n=14), and Hispanic/Latine (36%, n=9). Finally, of the 44 clinic observations, half (50%, n=22) occurred during 24-month WCCs, with the remaining observations evenly split across 18- and 36-month WCCs. Most visits were conducted in Spanish (59%, n=26) and English (34%, n=15) followed by Haitian Creole (7%, n=3).
Caregiver outcomes
As shown in Table 2, average subscale scores were high for both the pre- and post-implementation surveys. No differences were found between pre- and post-implementation respondents.
Table 2. Caregiver survey outcomes
Survey construct
|
Pre-implementation
Mean (SD)
|
Post-implementation
Mean (SD)
|
p-value
|
Behavioral Intention
|
3.50 (.49)
|
3.53 (.45)
|
.57
|
Attitudes
|
3.63 (.40)
|
3.68 (.37)
|
.20
|
Subjective Norms
|
3.34 (.52)
|
3.43 (.50)
|
.06
|
Perceived Behavioral Control
|
3.24 (.54)
|
3.23 (.52)
|
.89
|
Process evaluation
Supporting quotes for each theme are displayed in Table 3.
Table 3. Staff interview quotes
|
Theme
|
Subtheme
|
Quote
|
Adherence
|
Coverage
|
“I find myself talking about play in other visits where I'm not giving the packet a lot more, but I think having [the play kit] and being able to just model some of the behaviors is very helpful.”
|
Content
|
“I would talk about the different kinds of areas that we're looking at for development, fine motor skill, gross motor skills, and problem solving and social skills and how they're learning a lot of those skills as they're playing with toys. And also as [caregivers are] playing with them with the toys. Usually I'm playing with the toys a little bit with the kids back and forth at the same time.”
|
Implementation Moderators
|
Intervention complexity
|
“[I] thought it was well done. It wasn't too long, wasn't too short. And it was helpful.”
|
|
“It was pretty straightforward and easy. It was fun. I guess it wasn't overwhelming,”
|
Quality of delivery
|
“You have to know your audience. I think that as far as what goes on in the room that is what dictates what happens in the room discussions.”
|
Participant responsiveness
|
“Our [pediatrics] population is low income, so I think having different toys and books and things, they really appreciate.”
|
“The kids get happy...You go in with vaccines and stuff like that. So [the kit] gets them happy. And it makes them open up to you…[the caregivers] get a little bit more trustworthy. Like, oh, they're actually interested in us, you know, what my kid is going through and everything.”
|
“Sometimes with parents I feel like sharing and speaking to them about this project, I feel like some give you their attention and some don't…Thinking of ways to kind of make it exciting.”
|
Facilitation strategies
|
|
Usefulness for developmental conversations
|
“I also love making comments to the parent about how the kid is playing while we're doing the exam…It's really important that they're just playing with it and you're just making some positive reinforcement about how they're using their hands or what they're doing with it…So I think it can promote discussion that way about developing and what we're looking for.”
“This is a very positive message that encourages children and parents to interact, that does all the right things without necessarily making it a don't, don't, don't, don't, don't, sort of activity.”
|
Organized implementation systems
|
“I think the one thing I got concerned about was remembering to [log in EHR] so we can track it. So part of our workflows [is] nurses do it then I check to make sure it's in. So I think that that's been helpful, but other than that, I really have no other concerns about the flow of the day.”
|
“I think the next challenge I'll have is because we only give the blocks once, right? So my next challenge is going to be remembering to check previous physicals as we get five to six months into this to make sure that I haven't already given the blocks.”
|
[Place Table 3 here]
Adherence. Adherence is the extent to which a program is delivered as designed as measured by program content, coverage, frequency, and duration. Using information gathered through the interview, survey, and observational measures, two subthemes related to adherence, namely coverage (i.e., whether the target population was reached) and content (i.e., whether intervention components were implemented as planned), were found.
Coverage. According to survey data, 68% (n=15/22) of respondents reported occasionally, rarely, or never incorporating discussions about the importance of play into WCCs before completing P4P trainings. After completing the P4P training, this number reversed, with 58% (n=14/24) of survey respondents reporting that they now always discuss the importance of play in early childhood, which was supported in interview conversations. In addition, 33% (n=7/21) of survey respondents reported inviting other providers from their organization to complete the P4P training to expand its reach.
Content. Nearly all survey respondents somewhat or strongly agreed that they were excited about P4P (92%, n=24/26), found the program valuable (96%, n=25/26), and believe play is important to children’s development (96%, n=25/26). Most providers further demonstrated alliance with program goals. In 98% (n=43) of observations, providers successfully gave out the P4P kit, during which 60% (n=26) modeled play. Ninety-five percent of providers (n=42) discussed the importance of play to some degree and nearly two-thirds (61%, n=27) delivered a specific “prescription for play” to caregivers. These observations were consistent with interviews, with most respondents describing success in implementing the program as designed. However, staff surveys indicated more variability in adherence to program content, with only 45% (n=10/22) of respondents reporting that they model play using the kits at least most of the time.
Implementation moderators. Several implementation moderators that promoted program adherence, including intervention complexity, quality of delivery, participant responsiveness, and facilitation strategies, were identified across data sources.
Intervention complexity. Providers indicated that P4P fit nicely into their existing visits, with 89% (n=16/18) of survey respondents agreeing that the P4P content is relevant to their practice. In interviews, staff described the P4P training as a low-lift that enhanced their knowledge, further noting that the intervention itself was not time-consuming and complemented their workflows. Observations confirmed this by demonstrating that P4P took an average of 2.9 minutes to deliver, with the core messaging largely being built into standard anticipatory guidance.
Quality of delivery. In interviews, providers described tailoring their discussions about the benefits of play using education provided in the P4P training. From observations, the most frequently discussed topics during visits included encouragement to use simple toys and/or have screen-free time (n=28) and the role of play in language acquisition and speech development (n=28). Additional topics discussed included the relationship between play and brain development (n=24), social-emotional skills (n=23), and emotional regulation (n=14). However, several providers reported challenges with translating the training recommendations into practical play-based recommendations for caregivers upon interview and desired more practical guidance and sample scripts.
Participant responsiveness. A key facilitator to P4P implementation was its positive reception by families. In interviews, staff reported the toy was appreciated as a free resource. Observations further showed that children were often eased or comforted by the playfulness the program brought. Providers noted that this allowed them to better evaluate their patients’ developmental status because children were more likely to display physical evidence of developmental milestones, like stacking blocks, speaking, or interacting socially, when comfortable. Meanwhile, some providers expressed challenges with engaging caregivers in discussions about play upon interview, which may relate to different approaches in program delivery. Indeed, caregivers were seen to engage more actively in P4P when providers modeled play during observations. In 25% (n=11) of observations, families did not play with the P4P kit, and of those visits, 82% (n=9) did not have providers model play, which may have impacted caregiver engagement.
Facilitation strategies. Several subthemes related to facilitation strategies inductively emerged relating to the usefulness of the program for developmental conversations and the importance of organized systems for implementation.
Usefulness for developmental conversations. Providers appeared more willing to carry out the intervention if they recognized the importance of play in development and how P4P could be used as a complementary tool to support development. For example, providers reported often using the kit to observe how their patients engaged with the bricks to understand their developmental stage.
Organized implementation systems. Established processes, like electronic health record (EHR) tracking, workflows, and storage plans, facilitated P4P implementation. Those with established implementation systems and strategies were observed to be more likely to model play and discuss multiple benefits of play. In contrast, interviewees reported challenges with implementation when organizational systems were not clear, with others noting concerns about remembering to track kit distribution in the EHR. The importance of these organizational systems was further reflected in open-ended comments in the staff survey, with respondents expressing several challenges they faced in delivering P4P kits when organizational systems were not in place, like “Making sure each patient who qualifies for a kit gets identified” and “Running out of P4P kits.”