One size fits all? A latent Profile Analysis to Identify Care Professional Subgroups Based on Implementation Determinants

doi:10.21203/rs.3.rs-3951542/v1

Download PDF

Research Article

One size fits all? A latent Profile Analysis to Identify Care Professional Subgroups Based on Implementation Determinants

https://doi.org/10.21203/rs.3.rs-3951542/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Introduction: Recent research emphasizes the need for a more holistic approach to implementation science, ackonowledging complex interactions among implementation determinants and heterogeneity in context and care professionals (CPs). To verify this need, we aimed to identify distictive subgroups of CPs based on their unique profiles of implementation determinants concerning the Childcheck, a guideline facilitating early identification of child abuse based on parental characteristics. We also explored the influence of organization type on subgroups of CPs with specific implementation characteristics (subgroup membership) and assessed their relationship to CPs implementation level.

Methods: A total of 562 Dutch CPs in Mental Health Care (aMHC) and Forensic Care settings (Forensic MHC, Probation Service, and The Salvation Army) completed a self-reported questionnaire on Childcheck implementation determinants. We conducted Latent Profile Analysis to identify subgroups of CPs. The influence of organization type on subgroup membership was examined using Chi-Squared test and we explored the impact of subgroup membership on implementation levels using a one-way ANOVA.

Results: We identified five distinct subgroups. Subgroup A (Reporting Center for Child Abuse and Neglect (RCCAN) collaboration issues, 11.7%) faced issues related to the external organization, such as feedback and collaboration issues. Subgroup B (RCCAN collaboration and organizational issues, 5.0%) encountered challenges with both the external and internal organization, including issues with financial resources and formal agreements, resulting in the lowest implementation level. Subgroup C (Limited implementation issues, 9.4%) demonstrated relatively high ratings across determinants, achieving the highest implementation level. CPs in subgroup D (CP-client interaction issues, 37.7%) encountered challenges specifically in CP-client interaction. CPs in subgroup E (Indifferent attitudes towards implementation, 36.1%) expressed low to average retings and was predominantly represented by CPs in aMHC settings. This subgroup also reported a low to average implementation level.

Conclusions: This study highlights the importance of tailored implementation plans to address each subgroup's specific needs and challenges, instead of employing a one-size-fits-all approach. Latent Profile Analysis successfully revealed the variations in implementation determinants among CPs in aMHC and Forensic Care settings. Tailoring implementation strategies for these subgroups is key to successful guideline implementation and enhancing the well-being of vulnerable children and families.

Implementation Determinants

Childcheck

Mental Health Care

Forensic Care

Vulnerable children and families

Latent Profile Analyses

Recent implementation research has emphasized the need for a more holistic approach, recognizing the complex interactions among implementation determinants and the heterogeneity among care professionals working in adult Mental Health Care and Forensic Care.
This study shows that Latent Profile Analysis is a valuable method to identify subgroups of professionals, each characterized by unique implementation determinants, allowing for the development of tailored implementation strategies.
Addressing the specific needs of these subgroups can improve guideline implementation and enhance the well-being of vulnerable children and families.

In recent years, there has been a growing interest in implementation research within child and family care settings, particularly in determinants influencing the implementation of guidelines and interventions (1–5). The analyses in these studies have predominantly focused on the influence of individual determinants on implementation, such as resource availability, self-efficacy, and guideline complexity. Although each of these determinants is of importance and has an impact on the implementation of guidelines and interventions on its own, there is a compelling argument for adopting a more holistic approach in this research area (6, 7). Successful implementation is not a straightforward process determined by the presence or absence of determinants in isolation; rather, it depends on the complex interaction and synergist relationships between these determinants (7, 8). For example, while lack of time is often perceived as a barrier to implementation, it can result from various factors such as, insufficient leadership, low service priority, or inappropriate workflow. Effectively addressing the issue of time constraints in an implementation program requires targeted strategies tailored to the identified causes, rather than attempting to address the barrier in isolation (7). Furthermore, Care Professionals (CPs) and care settings can exhibit significant heterogeneity in implementation determinants and organizational context, necessitating implementation efforts tailored to their unique needs (9, 10). Identifying subgroups enables researchers and policymakers to understand the unique needs and challenges of different CPs, facilitating the creation of tailored strategies for each subgroup, increasing the likelihood of successful implementation. This subgroup-focused approach also improves resource allocation. By understanding the specific characteristics and needs of each subgroup, resources can be directed where they are most needed, ensuring efficient use and maximizing the impact of implementation efforts. However, in the current body of literature, there has been limited attention to quantitatively examining specific subgroups based on implementation determinants. While previous qualitative research has revealed some patterns and trends regarding implementation processes (11–13), there is a lack of in-depth quantitative approaches to systematically identify these subgroups.

This study presents a method to systematically capture the contextual variation or heterogeneity in implementation determinants across CPs. We performed latent profile analysis (LPA) (14) to identify groups of CPs, with unique profiles of implementation determinants. We used data collected during the implementation of the Childcheck, a guideline aimed at facilitating the early identification of child abuse based on parental characteristics, such as domestic violence, substance abuse, and suicide attempt or other severe psychiatric problems (15, 16). The Childcheck is part of the Model Protocol for Domestic Violence and Child Abuse and is mandatory for all professionals working with adult patients or children. This protocol guides CPs in effectively responding to signs of violence though a structured approach involving the following steps: 1) Identify the signs; 2) Consult with a colleague and, if necessary, with the Reporting and Advice Center for Domestic Violence and Child Abuse (RCCAN; in Dutch: Veilig Thuis); 3) Talk to the person(s) involved; 4) Assess whether domestic violence or child abuse has occurred. If in doubt, consult with the RCCAN; 5) Decide whether to arrange help yourself or report the case to the RCCAN. The Child Check is conducted during the first step, emphasizing the identification of signs based on parental characteristics rather than focusing solely on child-related factors. In 2018, the Dutch Ministry of Health, Welfare, and Sport commissioned the initiation of a Childcheck implementation impulse within adult Mental Health Care (aMHC) and Forensic Care (FC). aMHC and FC involve individuals dealing with complex mental health issues, co-occurring disorders, or past trauma and legal involvement. The vulnerability and complexity of these individuals may jeopardize the safety and well-being of their children (15). Improving the implementation of the Childcheck in these settings contributes to creating a safer environment for children through early detection of potential risks and the initiation of appropriate interventions. It acknowledges the importance of the family perspective in aMHCand FC, aligning with the responsibilities of CPs to ensure the safety and well-being of all parties involved.

The primary objective of this impulse was to facilitate the transition of the Childcheck from policy to practice by assisting, monitoring, and evaluating its implementation. One of the steps in the implementation impulses was to identify determinants influencing the implementation of the Childcheck. Using data from the implementation of the Childcheck within aMHC and FC, we aimed to:

Identify subgroups of CPs working in aMHC and FC based on their unique profiles of implementation determinants.

2a) Explore whether subgroup allocation is related to organization type.

2b) Assess how subgroup allocation is related to CPs’ implementation level.

For this study, we conducted retrospective, cross-sectional analyses to identify subgroups among CPs based on data related to determinants influencing the implementation of the Childcheck. The implementation impulses that were initiated by the ministry, were translated into implementation activities by a team of professionals, including researchers, domestic violence and child abuse policy officers, and statisticians. Additionally, an advisory group, comprised of policymakers and professionals from various aMHC and FC settings, played a key role by offering insights, feedback, and advice on the implementation process. The Haaglanden Medical Center (The Hague, The Netherlands) was responsible for executing the implementation impulses throughout The Netherlands. The Medical Ethics Committee at Leiden University Medical Center determined that the Dutch Medical Research Involving Human Subjects Act did not apply to our research proposal (proposal number WSC-2022-38). Our reporting follows the STROBE guidelines (Appendix A) (17).

Participants and procedures

Two distinct implementation impulse initiatives were developed for CPs working in Dutch aMHC and FC organizations. The aMHC implementation impulse represents an extension of a three-year implementation program (2016–2018) for aMHC organizations, involving 103 organizations. By the end of this program, 37 organizations either did not implement or did not adequately implement the Childcheck according to the recommended fundamental criteria. Consequently, these aMHC organizations were invited to participate in an additional implementation impulse in 2019 to enhance their implementation efforts. Some organizations that participated in the initial implementation program were unable to join the extended program due to mergers. Ten organizations joined the extended implementation program. Furthermore, we recruited other aMHC organizations that had not already taken part in the initial implementation program, of which two decided to participate. In total, twelve aMHC organizations participated. In the Netherlands, aMHC services includes diagnosis, treatment, and support for individuals and their family dealing with various mental health issues, such as depression, anxiety, and psychiatric disorders. Funding for aMHC is commonly sources from the Health Insurance Act.

The FC implementation impulse was initiated in 2020, and we recruited FC organizations, including Forensic MHC, Probation Service, and the Salvation Army. The FC aims to safely reintegrate offenders into society, recognizing that punishment alone is insufficient for those with mental disorders, intellectual disabilities, or addiction. Specifically, forensic MHC focuses on treating mental health issues related to criminal behavior, while Probation Service offers guidance for post-sentence reintegration and preventing recidivism. The Salvation Army provides various assistance services, including support for the homeless, substance abusers, and the socially vulnerable. In the Netherlands, government funding supports Forensic MHC and Probation Services, while The Salvation Army's financing depends on the type of care, whether it is funded by the municipality, government, or the Health Insurance Act. Thirty-two FC organizations decided to participate (i.e., 19 Forensic MHC, 10 Probation Services, and 3 the Salvation Army. In total, 44 aMHC and FC organizations joined the initiative (Appendix B).

Communication was facilitated through representatives of the respective organization. Throughout the two implementation phases in aMHC and FC, each representative distributed a questionnaire among CPs via mail, providing a link to an online platform (https://kindcheck-ggz.nl and https://kindcheck-forensisch.nl). Reminders were sent to the representatives regarding the distribution of the questionnaires. In aMHC organizations, questionnaires were distributed from February 2019 to November 2020 and from October 2020 to June 2022 in FC organizations. The implementation impulses were initially not designed with a research intention but rather aimed to evaluate, assist, and monitor the implementation of Childcheck. The questionnaires were entirely anonymous, with only the organization name and department being visible. Consequently, no individual informed consent was obtained. CPs were informed via the website about the implementation impuls and that data would be collected anonymously and shared with their respective organizations to optimize implementation of the Childcheck.

Questionnaire

We developed a questionnaire (Appendix C) to evaluate determinants influencing Childcheck implementation. This questionnaire is based on the theoretical and evidence-based Measurement Instrument for Determinants of Innovations (MIDI) framework developed by Fleuren et al.(18) for measuring determinants that influence the implementation of innovations. The MIDI included four categories evaluating determinants associated with the user (e.g., knowledge), the innovation (e.g., procedural clarity), the organization (e.g., financial resources) and the socio-political context (e.g., law and regulations). For the questionnaire development, we focused on the first three categories. Determinants relating to the socio-political context were not considered in the questionnaire since these determinants could not directly be influenced by the organization or CP. We deductively added items derived from evaluations with the project members and the advisory group.

Additionally, we included questions to evaluate the extent to which CPs adhere to Childcheck's recommendations (i.e., implementation level), comprising four items such as ‘Do you apply the Childcheck in the initial client meeting?’. The questionnaire comprises 37 questions distributed across four categories: 1) the user (the CP, twenty items), 2) the innovation (Childcheck, four items), 3) the organization (aMHC and FC organizations, nine items), and 4) the implementation level (four items). Items concerning implementation determinants used a 5-point Likert scale (1 - totally disagree to 5 - totally agree), with an additional ‘not applicable/I don't know’ option. Implementation level items were rated on a 4-point Likert scale (1 – never to 4 - always), except for one item using a 5-point scale (1 - totally disagree to 5 - totally agree). Prior to distribution, we conducted a review for clarity and time estimation. The total time needed to fill out the questionnaire was approximately 20–30 minutes.

Statistical Analyses

Data cleaning. We imported data to IBM SPSS Statistics 25 for Windows. Since all questions were mandatory, there were no missing values. The category ‘not applicable/I don’t know’ was recoded as 3 (neutral), and we adjusted the categories of the reverse-worded items. We excluded data from CPs who were not familiar with the Childcheck. The cleaned data was imported into RStudio version 4.3.1 for further analysis.

Reliability and scaling. We used Item Response Theory (IRT) to compute scales for all constructs that contained two or more items. This analysis was performed using the ‘mirt’ package (Appendix D) (19). In the IRT analysis, weighted person scores were computed for each participant based on their responses to all items within a specific construct. The IRT analysis assigns different weights to items, based on item discrimination (i.e., a-parameter, measuring an item's ability to distinguish trait levels) and threshold parameter (i.e., b-parameter, identifying trait level where response category choice changes). We applied the Generalized Partial Credit Model of IRT to the following determinants: coordinator, partnership and connection, client cooperation, descriptive norm, knowledge, professional obligation, outcome expectations, and implementation level. For each item within a construct, we evaluated both a- and b-parameters. Each construct included items with a-parameters above one and appropriately categorized b-parameters. We rescaled the resulting person scores for each construct to a range from 1 to 5 to align with the other determinants.

Based on the IRT analyses, we formed seven reliable construct, each effectively representing the following determinants: coordinator (two items), partnership and connections (four items), client cooperation RCCAN (two items), descriptive norm (two items), knowledge (two items), professional obligation (four items), and outcome expectations (two items). The construct implementation level was formed by three items; the item (‘I conduct the Childcheck in accordance with step 1 of the Reporting Code’) was not included in the construct due to poor discrimination, inconsistent location parameters, and poor fit to the model. In total, 22 determinants divided over three domains were used in the LPA analyses: the innovation (compatibility, observability, procedural clarity, relative priority), the organization (access to knowledge, coordinator, financial resources, formal ratification, partnership and connection, time, client cooperation Childcheck, client cooperation RCCAN), and the professional (communication skills, descriptive norm, general skills, routine, implementation needs, knowledge, outcome expectations, professional obligation, relationship client, social support).

Subgroup identification. We conducted a LPA in Rstudio using the ‘tidyLPA’ package (Appendix E) (20). LPA is a statistical technique used to identify unobserved (latent) subgroups or profiles within a heterogenic population by examining patterns in observed variables (14). Specifically, in this study, subgroup membership is determined by CPs’ response patterns to questionnaire items. We applied Model 1 LPAs with one to ten profiles, assuming that variances are equal across profiles and covariances are set to zero. We assessed model fit using the ‘MClust’ package (21) and selected the optimal model based on fit indices and interpretability (22). To compare the relative fit among competing models, we used the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC), with lower values indicating a better fit to the data. We also examined changes in fit indices; note that AIC may continuously decrease with large sample sizes, and a high BIC could suggest an overly complex model (23). Additionally, we assessed the models’ entropy, indicating the clarity of profile distinction, with scores ranging from 0 to 1 (optimal) (24). Finally, we interpreted the models by examining the mean ratings of implementation determinants within each class. This involved identifying the unique characteristics of each class and gaining insight into the practical significance and applications of these classes in the context of Childcheck implementation.

Relationship with organization type and implementation level. To assess the relationship between organization type (i.e., aMHC, Forensic MHC, Probation Services, and the Salvation Army) and subgroup membership, we performed a chi-square test. To account for multiple comparisons, we applied Bonferroni corrections to control for Type I errors. Our hypothesis was that organization type would be associated with group membership (significant level was set at α = .05 and α = .0025 with Bonferroni correction). We also explored the impact of subgroup membership on implementation levels using a one-way ANOVA with a significance level set at α = .05. Post-hoc pairwise comparisons were conducted using Tukey's Honestly Significant Difference method to identify specific group differences when the ANOVA showed significance. Our hypothesis was that there would be significant differences in implementation levels among the identified subgroups.

In total, 603 CPs completed the questionnaire (aMHC: 204, forensic MHC: 198, Probation Service: 161, and the Salvation Army: 40). Forty-one cases were excluded due to unfamiliarity with the Childcheck, resulting in a final dataset of 562 cases for the subsequent analysis (aMHC: 193, forensic MHC: 180, Probation Service: 159, and the Salvation Army: 30).

Subgroup identification

We assessed the fit indices of various models, ultimately selecting the model that best represented distinctive and interpretable subgroups within the data (Appendix F). Both the AIC and BIC exhibited a significant decrease in the five-profile model, followed by an increase. Entropy values were deemed most favorable for the model with five profiles. Based on the fit indices and interpretability, a model with five profiles was considered the best, having an entropy of 0.91. Table 1 provides estimates for all implementation determinants, and Fig. 1 shows a visual representation of the subgroups.

Table 1

Mean ratings of implementation determinants in the five identified subgroups and the total group.
Determinant		RCCAN collaboration issues	RCCAN collaboration and organizational issues	Limited implementation issues	CP-client interaction issues	Indifferent attitudes towards implementation	Total
		n = 66, 11.7%	n = 28, 5.0%	n = 53, 9.4%	n = 212, 37.7%	n = 203, 36.1%	n = 562
Innovation	Compatibility	4.21	2.56	3.86	4.23	3.21	3.75
	Observability	1.19	1.19	3.74	3.33	2.85	2.83
	Procedural clarity	4.14	2.94	3.36	4.45	3.47	3.88
	Relative priority	3.20	4.18	3.34	2.39	2.84	2.83
Organization	Access to knowledge	3.96	2.56	3.96	4.04	2.95	3.56
	Coordinator	2.87	2.09	3.21	3.63	2.90	3.16
	Financial resources	3.31	1.34	3.80	3.85	3.06	3.37
	Formal ratification	4.18	1.98	3.99	4.35	3.17	3.76
	Partnership & connections	1.96	1.76	3.73	3.74	3.04	3.17
	Time	4.01	2.10	4.30	4.11	2.96	3.61
Professional	Client cooperation Childcheck	3.33	4.59	3.56	2.31	2.80	2.85
	Client cooperation RCCAN	4.27	4.51	2.59	2.89	3.25	3.24
	Communication skills	2.79	3.68	3.60	2.31	2.58	2.66
	Descriptive norm	3.94	3.15	4.16	3.66	3.05	3.50
	General skills	4.16	3.12	3.58	4.46	3.54	3.94
	Implementation needs	2.16	3.11	2.72	2.28	3.27	2.70
	Knowledge	3.85	2.23	3.21	4.23	2.94	3.53
	Outcome expectations	2.26	1.88	3.60	3.35	2.85	2.99
	Professional obligation	3.76	3.10	3.14	4.20	4.08	3.95
	Relationship client	3.03	3.94	3.93	2.32	2.68	2.77
	Routine	3.85	1.49	3.60	3.92	2.55	3.27
	Social support	3.90	1.52	3.94	4.32	3.32	3.73
Note. RCCAN = Reporting Center for Child Abuse and Neglect; CP = Care Professional; Bold values represent the lowest value for each determinant or a value lower than 2.50: Italic values represent the highest value for each determinant.

Subgroup A (n = 66, 11.7%), was characterized by low mean ratings on determinants relating to the RCCAN. CPs in this profile expressed low confidence in and perceived low client satisfaction with the assistance offered by the RCCAN (2.26 and 1.19, respectively). They also scored low in determinants such as collaboration and communication with RCCAN (1.96). However, the Childcheck was compatible with CPs' current practices (4.21), formal agreements concerning the implementation of the Childcheck were formulated by their organization (4.18), and CPs were provided with enough time (4.01) and had access to knowledge (3.96). CPs also considered clients as cooperative concerning conversations about RCCAN (4.27). Subgroup A was labeled as ‘RCCAN collaboration issues’.

Subgroup B (n = 28, 5.0%) shared similarities with subgroup C with low mean ratings on determinants related to the RCCAN. Additionally, profile B was marked by low mean ratings on determinants associated with the internal organization, including concerns about formal agreements (1.98) and the presence of various resources (coordinator = 2.09, financial resources = 1.34, and time = 2.10). Routine and support from colleagues or supervisors was also rated as low (1.49 and 1.52, respectively). However, CPs in this profile rated determinants related to client cooperation as high (4.59 and 4.51) and indicated that applying the Childcheck did not interfere with other activities (4.18). We labeled subgroup B as ‘RCCAN collaboration and Organizational Issues’.

Subgroup C (n = 53, 9.4%) was characterized by overall high to average ratings, except for client cooperation concerning conversations about RCCAN (2.59). CPs in this profile reported to be provided with sufficient time (4.30) and perceived that the Childcheck was applied by their colleagues (4.16). Subgroup C was labeled as ‘Limited implementation issues’.

Subgroup D (n = 212, 37.7%), was characterized by overall high ratings on the determinants. However, CPs in this profile encountered issues when integrating the Childcheck into practice. They reported that applying the Childcheck interfered with their other activities (2.39), that they lacked communication skills (2.31), and were concerned that applying the Childcheck might harm their relationship with clients (2.32). Additionally, they considered clients as poorly cooperative concerning the Childcheck (2.31). We labeled subgroup D as ‘CP-client interaction issues.

Subgroup E (n = 203, 36.1%), was characterized by overall low to average ratings and was accordingly labeled as ‘Indifferent attitudes towards implementation’. In this subgroup, CPs did not express particularly positive or negative opinions regarding the Childcheck or its implementation determinants.

[Figure 1]

[Table 1]

Relationship with organization type and implementation level Organization. The distribution of organizations within subgroups is shown in Fig. 2 and the results of the chi-square test with Bonferroni corrections are presented in Table 2. The results show a significant association between organization type and group membership among CPs (χ²(12, n = 562) = 263.09, p < 0.001). More specifically, CPs within aMHC were found to be more often present in the ‘Indifferent attitudes towards implementation’ subgroup (Std. Res = 15.4, p < 0.001), and less often to be present in the other subgroups (Limited implementation issues: Std. Res=-5.4, p < 0.001; CP-client interaction issues: Std. Res=-6.6, p < 0.001; RCCAN collaboration issues: Std. Res=-5.9, p < 0.001; RCCAN collaboration and organizational issues: Std. Res=-3.5, p < 0.05). CPs within Forensic MHC were more often present in the ’Integration issue’ subgroup (Std. Res = 4.1, p < 0.001) and less often present in the ‘Indifferent attitudes towards implementation’ subgroup (Std. Res=-7.5, p < 0.001). While CPs within Probation Service were more often present in the ’Integration issue’ subgroup (Std. Res = 3.1, p < 0.05) and ‘RCCAN collaboration issues’ (Std. Res = 3.4, p < 0.05), they were less often present in the ‘Indifferent attitudes towards implementation’ subgroup (Std. Res=-7.1, p < 0.001). CPs within The Salvation Army were more often present in the ‘Limited implementation issues’ subgroup (Std. Res = 3.4, p < 0.05).

Table 2

Chi-square tests for subgroup membership and organizations with Bonferroni correction.
		Subgroups
Org		RCCAN collaboration issues	RCCAN collaboration and organizational issues	Limited implementation issues	CP-client interaction issues	Indifferent attitudes towards implementation	Total
aMHC	Obs	2	1	1	38	151	193
	Exp	22.7	9.6	18.2	72.8	69.7
	Res	-4.3	-4.3	-4.3	-4.3	-4.3
	Std. Res	-5.7	-3.5	-5.2	-6.4	15.0
Forensic MHC	Obs	31	13	21	90	25	180
	Exp	21.1	9.0	17.0	67.9	65.0
	Res	2.1	2.1	2.1	2.1	2.1
	Std. Res	2.8	1.7	1.2	4.1	-7.5
Probation Service	Obs	30	9	23	75	22	159
	Exp	18.7	7.9	15.0	60.0	57.4
	Res	2.6	2.6	2.6	2.6	2.6
	Std. Res	3.3	0.5	2.6	2.9	-6.9
The Salvation Army	Obs	3	5	8	9	5	30
	Exp	3.5	1.5	2.8	11.3	10.8
	Res	-0.3	-0.3	-0.3	-0.3	-0.3
	Std. Res	-0.3	3.0	3.3	-0.9	-2.3
Total		66	28	53	212	203	562
Note. Standardized residuals in bold are those that exceeded +/- 3.0 (p < 0.0025); aMHC = Mental Health Care; RCCAN = Reporting Center for Child Abuse and Neglect; Care Professional.

Implementation level. Overall mean implementation level was 2.94 [1.00–5.00] (median = 3.04; IQR = 2.21–3.67]. Boxplots for implementation levels for each subgroup are shown in Fig. 3 and the results of the ANOVA and pairwise comparisons are presented in Table 3. The ANOVA revealed a significant overall association between subgroup membership and implementation level (F(4, 557) = 37.4, p < 0.001). Post hoc tests showed a significant lower mean implementation level in the ‘RCCAN collaboration and organizational issues’ subgroup compared to all other subgroups (Mean Diff.= -1.58 (RCCAN collaboration issues), -2.02 (Limited implementation issues), -1.58 (CP-client interaction issues), and − 0.83 (Indifferent attitudes towards implementation), all p < 0.001). CPs in the ‘Limited implementation issues' subgroup had a significant higher mean implementation level compared to the CPs in the ‘RCCAN collaboration and organizational issues’ (Mean Diff.=2.02, p < 0.05), ‘CP-client interaction issues’ (Mean Diff.=-0.44, p < 0.05), and the ‘Indifferent attitudes towards implementation’ subgroup (Mean Diff.=1.19, p < 0.05).

Table 3

One-way ANOVA for subgroup membership and implementation level with pairwise comparison
Source	Sum of Squares		Df	Mean Sq.		F-value		p-value
Class	140.63		4	35.16		37.21		< 0.001
Residual	526.25		557	0.94
Post Hoc Tukey’s HSD
					Mean Diff.	95% CI			Adjusted
Profile 1		Profile 2			Profile 2 − 1	Lower	Upper		p-value
RCCAN collaboration issues		RCCAN collaboration and organizational issues			-1.58	-2.17	-0.97		< 0.001
		Limited implementation issues			0.44	-0.04	0.94		0.09
		CP-client interaction issues			0.00	-0.37	0.38		1.00
		Indifferent attitudes towards implementation			-0.75	-1.12	-0.37		< 0.001
RCCAN collaboration and organizational issues		Limited implementation issues			2.02	1.40	2.64		< 0.001
		CP-client interaction issues			1.58	1.05	2.12		< 0.001
		Indifferent attitudes towards implementation			0.83	0.29	1.37		< 0.001
Limited implementation issues		CP-client interaction issues			-0.44	-0.85	-0.03		< 0.05
Limited implementation issues		Indifferent attitudes towards implementation			-1.19	-1.60	-0.78		< 0.001
CP-client interaction issues		Indifferent attitudes towards implementation			-0.75	-1.01	-0.49		< 0.001
Note. 95% CI = 95% Confidence Intervals; RCCAN = Reporting Center for Child Abuse and Neglect; CP = Care Professional.

This study aimed to identify subgroups among CPs based on determinants influencing the implementation of the Childcheck within aMHC and FC. Five distinct subgroups were identified, each with their unique profile of implementation determinants. Subgroup A (RCCAN collaboration issues) had low mean ratings in determinants related to the RCCAN, such as collaboration, communication, and client assistance. Subgroup B (RCCAN collaboration and organizational issues) was similar to Subgroup A but had additional low ratings for internal organization determinants, like formal agreements and various resources. CPs in Subgroup B also showed low mean ratings for routine, suggesting that performing the Childcheck has not become a regular practice for them. Subgroup C (Limited implementation issues) exhibited overall relative average to high ratings. CPs in subgroup D (CP-client interaction issues) faced some difficulties integrating the Childcheck into practice, including a lack of communication skills and concerns about client relationships. CPs in subgroup E (Indifferent attitudes towards implementation) expressed average opinions, neither strongly positive nor negative. Most CPs were classified into the ‘CP-client interaction issues’ subgroup, followed by the ‘Indifferent attitudes towards implementation’ subgroup. This latter subgroup was predominantly represented by CPs working in aMHC settings. The 'Limited implementation issues' subgroup demonstrated the highest level of implementation, while the 'RCCAN collaboration and organizational issues' subgroup exhibited the lowest implementation level.

Considering implementation theories, we observed a noteworthy alignment between identified subgroups and the domains of the Consolidated Framework for Implementation Research (CFIR) (25), highlighting the consistency between practical outcomes and theoretical foundations. This alignment not only strengthens the validity of the CFIR framework but also underscores its practical relevance in implementing the Childcheck in both aMHC and Forensic MHC settings. It offers researchers, policymakers, and care professionals a solid framework to understand specific implementation challenges and develop targeted intervention strategies, thereby enhancing Childcheck implementation.

The identified subgroups are based on the co-occurrence of determinants, aligning with patterns found in prior qualitative research. For example, in the 'CP-client interaction issues' subgroup, we observed the co-occurrence of communication skills, client relationships, and client cooperation. In prior studies, CPs expressed concerns about potential aggressive reactions or damaging their client relationships, leading to reducing parental cooperation, when addressing suspected child abuse. However, such concerns might be caused by lack of communication skills (12, 13). Similarly, the determinants characterizing the 'RCCAN collaboration issues' subgroup have been previously recognized as co-occurrent (11, 13). CPs expressed a lack of confidence in follow-up care, with the care offered perceived as inadequate or too slow, potentially exacerbating the child's situation after reporting. These concerns might be influenced by CPs facing challenges related to RCCAN, including a lack of feedback and unclear communication, and often find themselves not taken seriously.

Our study's findings are comparable with a previous study using LPA to identify subgroups based on pre-implementation determinants (26). However, interpreting similarities requires careful consideration, acknowledging distinctions between the two studies concerning the type of innovation (HIV pre-exposure prophylaxis versus the Childcheck guideline) and implementation phases (pre-implementation versus post-implementation). Furthermore, Piper et al. focused on organizational readiness for implementation from the perspective of professionals or administrators, while our study centered on the professionals themselves. Piper et al. identified six distinct profiles, with the 'Highest Capacity for Implementation' subgroup being similar to our ‘Limited implementation issues’ subgroup. They reported overall high mean ratings on determinants, which corresponds to our subgroup. They also identified a subgroup named the 'Resource-Strained Group,' which faced obstacles concerning the internal and external organization, such as limited resources, weak leadership engagement, poor implementation climate, and external partnerships, corresponding to our subgroup 'RCCAN collaboration and organizational issues'. Moreover, both groups showed the lowest scores on outcome measures, namely implementation readiness and implementation level, emphasizing a substantial impact of internal and external organizational determinants on the implementation process. Both studies identified a specific subgroup with neither strongly positive nor negative ratings. Notably, Piper and colleagues were unable to identify a subgroup comparable to our 'CP-client interaction issues' subgroup due to the pre-implementation nature of their study.

Practical implications

The identification of distinct subgroups of CPs allows for the development of tailored implementation strategies. Instead of employing a one-size-fits-all approach, organizations can customize their implementation plans to address each subgroup's specific needs and challenges. For example, in the aforementioned ‘Integration issue’ subgroup, CPs might benefit from improving client communication and local consensus discussions to reflect on why the Childcheck is important, rather than a distraction from their “real work”. Meanwhile, the ‘RCCAN collaboration issues’ subgroup could benefit by building partnerships to facilitate information sharing, collaborative problem-solving, and the development of a shared vision and goals related to the implementation of the Childcheck (27). A Cochrane Review found that tailored strategies improved CPs’ implementation into practice (28). Successful implementation of guidelines like Childcheck leads to early identification and intervention for children at risk. This, in turn, not only enhances their well-being and reduces the risk of long-term problems (15, 29–31), but also plays a role in constraining associated societal costs (30, 32).

The ‘Indifferent attitudes towards implementation’ subgroup primarily comprised CPs working in aMHC settings and demonstrated a low to average implementation level. The term ‘Indifferent attitudes towards implementation’ implies a degree of disinterest among these CPs when confronted with new initiatives or changes. This indifference may have stemmed from the substantial challenges faced by the Dutch aMHC in recent years. The decentralization in the Dutch aMHC system in 2015 resulted in a fragmented care environment, influenced by budgetary constraints and the delegation of responsibilities to municipalities (32). The decentralization introduced complexity by involving different levels of government and diverse funding structures. Consequently, this complexity may contribute to increased bureaucracy and administrative burdens, leaving professionals with less time and energy for implementing new initiatives. Additionally, despite the introduction of an action program in 2017 to reduce waiting lists in mental healthcare (33), these lists have not decreased as expected. By mid-2022, the waiting list had increased to 80,000 individuals, with approximately 52% surpassing the specified target duration of fourteen weeks. Concurrently, the persistent personnel shortage reached 7% of vacant positions in 2022, with an expected continued rise over the next decade (34). These challenges might have collectively contributed to the observed indifferent attitudes towards Childcheck implementation among CPs in aMHC settings. However, for a deeper understanding of CPs' indifferent attitudes towards implementing the Childcheck within aMHC, additional qualitative research is essential. Qualitative methods facilitate an in-depth exploration of attitudes, behaviors, and experiences, allowing researchers to delve into the specific contexts and situations influencing indifferent attitudes towards Childcheck implementation.

Another noteworthy detail for discussion is that in the majority of aMHC settings, the implementation impulse began in 2016 and was later extended in 2019. This extension provided aMHC settings with additional time to enhance their implementation efforts. It's important to note that the questionnaires were distributed between 2019 and 2020. In contrast, for FC settings, the implementation impulse started in 2020, and the questionnaires were distributed between 2020 and 2021. This discrepancy resulted in a time gap of 3 to 4 years for aMHC settings, compared to a maximum of 1 year for the FC settings, and have introduced potential differences in experiences and attitudes to the Childcheck implementation among CPs within aMHC compared to FC.

Strengths and limitations

This study has several strengths. To the best of our knowledge, it is among the first to take a holistic approach to implementation research, focusing on determinants that CPs perceived as influencing guideline implementation. LPA offers a detailed understanding of how distinct groups of CPs view and experience the determinants affecting the implementation of the Childcheck, offering meaningful insights for practice. Additionally, the study’s sample size of 562 participants exceeds the recommended minimum for LPA, enhancing the robustness of subgroup classification (35). Moreover, the high entropy of the five-profile model (0.91) indicates well-defined and easily distinguishable profiles, enhancing the validity and interpretability of the subgroup classifications. Last, the use of the MIDI in combination with input from project members and the advisory group provides a well-established, theoretical- and practice-based framework for evaluating implementation determinants.

Limitations should be noted as well. First, the study's reliance on organizational representatives for the distribution of questionnaires introduces a potential limitation in terms of generalizability. The effectiveness of the questionnaire distribution was contingent upon the varying levels of effort and diligence exhibited by these representatives. Second, and potentially as a consequence of the preceding limitation, within the aMHC, two organizations collectively represented about 80% of the total CPs. Since we identified a subgroup predominantly represented by CPs working in aMHC settings (i.e., the 'Indifference attitudes towards implementation' subgroup), we investigated whether this identification was primarily influenced by these two organizations. We examined the distribution of individual organizations across all classes and observed a notable presence of CPs from various organizations in the identified subgroup. Therefore, the identification was not limited to the well-represented organizations but was based on a likely widespread pattern within the aMHC. Next, we did not obtain data on CPs’ background characteristics, making it impossible to investigate whether factors such as gender, age, or work experience influenced the allocation of CPs in the different subgroups. Second, the study relied on self-report data and utilized a questionnaire with reverse-worded items, potentially introducing response bias that may have affected the identification of latent profiles in LPA. Furthermore, this study focused on aMHC and FC settings. Future research could explore if similar subgroup dynamics exist in other settings where the Childcheck is implemented, such as the emergency department, ambulance services and General Practices. Last, we were unable to obtain informed consent, since the implementation impulses were not originally established with a research intention but rather to evaluate, assist, and monitor the implementation of Childcheck. Nevertheless, the study ensured the anonymity of CPs and adhered to ethical guidelines to protect their privacy.

LPA is a valuable method to capture the heterogeneity in implementation determinants among CPs in aMHC and FC settings. We identified five distinct subgroups, each characterized by its unique set of implementation determinants. Interaction processes between CPs and clients posed significant challenges for a majority of CPs when implementing the Childcheck in practice and should be considered when developing a tailored implementation program. Additionally, considering the low implementation level, CPs facing challenges related to the RCCAN, organizational resources, leadership, and support should not be overlooked, despite being the smallest subgroup. Qualitative research is needed to gain a deeper understanding of the indifferent attitudes towards implementing the Childcheck among CPs in aMHC settings. Recognizing and addressing the specific needs of different CP subgroups, organizations can take more effective steps towards achieving successful guideline implementation and, ultimately, improving the lives of vulnerable children and families.

aMHC Adult Mental Health Care

CFIR Consolidated Framework for Implementation Research

CP Care Professional

FC Forensic Care

IRT Item Respons Theory

LPA Latent Profile Analysis

MIDI Measurement Instrument for Determinants of Implementation

RCCAN Reporting Center for Child Abuse and Neglect

Ethics approval and consent to participate

The Medical Ethics Committee of the Leiden University Medical Center, decided that the rules laid down in the Dutch Medical Research Involving Human Subjects Act (in Dutch: ‘Wet Medisch-wetenschappelijk Onderzoek met mensen’) did not apply to the research proposal (proposal number WSC-2022-38). The questionnaire was sent via an online platform and the data were processed without identifiers. We were unable to obtain informed consent, since the implementation impulses were not originally established with a research intention but rather to evaluate, assist, and monitor the implementation of the Childcheck. Nevertheless, the study ensured anonymity of CPs and adhered to ethical guidelines to protect their privacy.

Consent for publication

Not applicable.

Availability of data and materials

All data supporting the conclusions of this study are included in the paper and its additional files. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors have no competing interests to declare.

Funding

This study was funded by ZonMw, the Netherlands Institute for Health Research and Development (grant numbers 729220001 and 741500001).

Authors' contributions

EMD assisted in the development of the implementation impulse, was involved in the study design, analysed and interpreted the data, and wrote the initial draft and final manuscript. MRC assisted in the development of the implementation impulse, the design of the study, the interpretation of the data and critically revised the manuscript. JCKdJ and RMJJvdK assisted in the interpretation of the data and critically revised the manuscript. HMD and ILLG developed and executed the implementation impulse. All authors read and approved the final manuscript.

Acknowledgements

The authors wish to acknowledge all representatives for their effort during the implementation impulse and all participants for their time completing the surveys.

Konijnendijk AA, Boere-Boonekamp MM, Fleuren MA, Haasnoot ME, Need A. What factors increase Dutch child health care professionals’ adherence to a national guideline on preventing child abuse and neglect? Child abuse & neglect. 2016;53:118 – 27.
Diderich HM, Dechesne M, Fekkes M, Verkerk PH, Pannebakker FD, Velderman MK, et al. Facilitators and barriers to the successful implementation of a protocol to detect child abuse based on parental characteristics. Child Abuse Negl. 2014;38(11):1822–31.
Schalkwijk AA, Nijpels G, Bot SD, Elders PJ. Health care providers’ perceived barriers to and need for the implementation of a national integrated health care standard on childhood obesity in the Netherlands–a mixed methods approach. BMC Health Serv Res. 2016;16(1):1–10.
Pannebakker NM, Fleuren MA, Vlasblom E, Numans ME, Reijneveld SA, Kocken PL. Determinants of adherence to wrap-around care in child and family services. BMC Health Serv Res. 2019;19(1):1–8.
Asada Y, Lin S, Siegel L, Kong A. Facilitators and barriers to implementation and sustainability of Nutrition and physical activity interventions in early childcare settings: a systematic review. Prev Sci. 2023;24(1):64–83.
Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2015;11(1):1–39.
Garcia-Cardenas V, Perez-Escamilla B, Fernandez-Llimos F, Benrimoj SI. The complexity of implementation factors in professional pharmacy services. Res Social Administrative Pharm. 2018;14(5):498–500.
Nilsen P. Making sense of implementation theories, models, and frameworks. Implementation Science 30,: Springer; 2020. p. 53–79.
Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44:177–94.
Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front public health. 2019;7:3.
Konijnendijk AA, Boere-Boonekamp MM, Haasnoot‐Smallegange RM, Need A. A qualitative exploration of factors that facilitate and impede adherence to child abuse prevention guidelines in Dutch preventive child health care. J Eval Clin Pract. 2014;20(4):417–24.
Louwers EC, Korfage IJ, Affourtit MJ, De Koning HJ, Moll HA. Facilitators and barriers to screening for child abuse in the emergency department. BMC Pediatr. 2012;12(1):1–6.
Schols MW, De Ruiter C, Öry FG. How do public child healthcare professionals and primary school teachers identify and handle child abuse cases? A qualitative study. BMC Public Health. 2013;13(1):1–16.
Oberski D. Mixture models: Latent profile and latent class analysis. Modern statistical methods for HCI. 2016:275 – 87.
Diderich HM, Fekkes M, Verkerk PH, Pannebakker FD, Velderman MK, Sorensen PJ, et al. A new protocol for screening adults presenting with their own medical problems at the Emergency Department to identify children at high risk for maltreatment. Child Abuse Negl. 2013;37(12):1122–31.
Diderich HM, Fekkes M, Dechesne M, Buitendijk SE, Oudesluys-Murphy AM. Detecting child abuse based on parental characteristics: Does The Hague Protocol cause parents to avoid the Emergency Department? Int Emerg Nurs. 2015;23(2):203–6.
Von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Bull World Health Organ. 2007;85:867–72.
Fleuren MA, Paulussen TG, Van Dommelen P, Van Buuren S. Towards a measurement instrument for determinants of innovations. Int J Qual Health Care. 2014;26(5):501–10.
Chalmers RP. mirt: A multidimensional item response theory package for the R environment. J Stat Softw. 2012;48:1–29.
Rosenberg JM, Beymer PN, Anderson DJ, Van Lissa C, Schmidt JA, tidyLPA. An R package to easily carry out latent profile analysis (LPA) using open-source or commercial software. J Open Source Softw. 2019;3(30):978.
Scrucca L, Fop M, Murphy TB, Raftery AE. mclust 5: clustering, classification and density estimation using Gaussian finite mixture models. R J. 2016;8(1):289.
Johnson SK. Latent profile transition analyses and growth mixture models: A very non-technical guide for researchers in child and adolescent development. New Dir Child Adolesc Dev. 2021;2021(175):111–39.
Nylund-Gibson K, Choi AY. Ten frequently asked questions about latent class analysis. Translational Issues Psychol Sci. 2018;4(4):440.
Jung T, Wickrama KA. An introduction to latent class growth analysis and growth mixture modeling. Soc Pers Psychol Compass. 2008;2(1):302–17.
Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):1–16.
Piper KN, Haardörfer R, Escoffery C, Sheth AN, Sales J. Exploring the heterogeneity of factors that may influence implementation of PrEP in family planning clinics: a latent profile analysis. Implement Sci Commun. 2021;2(1):1–14.
Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):1–14.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Reviews. 2010(3).
Vliek L, Overbeek G, Orobio de Castro B. Effects of Topper Training on psychosocial problems, self-esteem, and peer victimisation in Dutch children: a randomised trial. PLoS ONE. 2019;14(11):e0225504.
Thielen FW, Ten Have M, de Graaf R, Cuijpers P, Beekman A, Evers S, et al. Long-term economic consequences of child maltreatment: a population-based study. Eur Child Adolesc Psychiatry. 2016;25:1297–305.
Sawyer AM, Borduin CM, Dopp AR. Long-term effects of prevention and treatment on youth antisocial behavior: A meta-analysis. Clin Psychol Rev. 2015;42:130–44.
Romeo R, Knapp M, Scott S. Economic cost of severe antisocial behaviour in children-and who pays it. BJPsych. 2006;188(6):547–53.
The Ministry of Health WaS. Actieplan NZa wachttijden in de zorg. The Hague 2017.
Boumans J, Kroon H, van der Hoek B. Ggz uit de knel. Utrecht: Trimbos instituut; 2023.
Spurk D, Hirschi A, Wang M, Valero D, Kauffeld S. Latent profile analysis: A review and how to guide of its application within vocational behavior research. J Vocat Behav. 2020;120:103445.

Download PDF

Reviewers agreed at journal
06 Mar, 2024
Reviewers invited by journal
02 Mar, 2024
Editor assigned by journal
13 Feb, 2024
First submitted to journal
11 Feb, 2024

You are reading this latest preprint version

One size fits all? A latent Profile Analysis to Identify Care Professional Subgroups Based on Implementation Determinants

Status:

Version 1

Abstract

Figures

Contributions to the literature

Introduction

Methods

Participants and procedures

Questionnaire

Statistical Analyses

Results

Subgroup identification

Discussion

Practical implications

Strengths and limitations

Conclusion

Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1