Ethics
The online survey study was approved by the Internal Review Board of the University of Marburg (approval number: 2019-58k). Participants received study information and provided informed consent before they were able to access the survey. Data were collected anonymously. All raw data were stored securely at the Department of Clinical Child and Adolescent Psychology at Philipps University in Marburg, Germany.
Participants
Eligible participants were licensed psychotherapists and psychiatrists for adults, children and adolescents as well as psychotherapists and psychiatrists enrolled in postgraduate training to obtain such a license. No exclusion criteria were applied. The link to the online survey was clicked 2.417 times. Overall, 913 participants continued after informed consent. Of these, 863 met the inclusion criteria (i.e., profession). A total of 261 participants were excluded due to drop-out before completion of the EBPAS-36, two due to implausible answers (for example being 99 years old), one due to conspicuous response patterns in EBPAS-36 (e.g., straight-lining despite reverse coded items). Of the remaining 599 participants, 502 were female (83.8 %) and their age ranged from 23 to 82 years (M = 36.62, SD = 11.26). Roughly half of the sample (56.26 %) stated being in postgraduate training to become psychotherapists or psychiatrists. 42.7 % of the German psychotherapists have ever or are currently working in science. Further information on profession are presented in Table 1.
Table 1.
Demographics and information on profession
Therapy orientation
|
%
|
Professional group
|
%
|
Current occupation
|
%
|
Cognitive Behavior therapy (CBT)
Psychodynamic psychotherapy (PDT)
PDT and psychoanalytic therapy
CBT and systemic therapy
Other
|
74.5
14.9
4.3
2.2
4.1
|
Psychotherapist in training
Licensed psychotherapist
Child and adolescent psychotherapist
Child and adolescent psychotherapist in training
Other
|
40.7
26.9
16.2
15.4
0.8
|
Outpatient practice
Psychiatric hospital
Clinic for psychosomatic medicine
Rehabilitation clinic/center
Psychiatric day-clinic
University
Other
|
64.1
15.1
6.5
5.0
4.5
2.2
2.6
|
Procedure
Translation. The translation was carried out in accordance with the WHO recommendations (www.who.int/substance_abuse/research_tools/translation/en/). The EBPAS-36 (43) was translated into German by the second author (JT) and back-translated by the bilingual English-speaking senior author (AB). The back-translation was reviewed by the original authors of the scale (MR, GAA) who provided feedback to assure the items represented the meaning and original constructs. The original and back-translated versions were then reviewed in a consensus meeting of the translating authors. Their consensus version was then reviewed by a group of German clinical psychotherapists (in training) and researchers as well as a graduate linguist for comprehensibility and wording. Their revisions were discussed and considered by the translating authors in a second consensus meeting, resulting in a final German version of the scale (see supplemental material 1).
Recruitment and data collection. All data were collected online via an openly accessible online survey, using the scientific survey platform SoSci Survey (www.soscisurvey.de). The survey was online and accessible from 14th November 2019 to 27th April 2020. The link to the online survey was widely distributed via e-mail lists of professional psychotherapy organizations that all licensed psychotherapists are members of, universities, training institutes, and psychiatric in- and outpatient institutions as well as Facebook groups of psychotherapists and psychiatrists. As an incentive to participate in the study, participants were free to enter into a raffle for a voucher of 500 Euros for a wellness-weekend or a popular events and ticket agent. On the first page of the survey, potential participants received full study information and were required to provide consent before they were able to proceed with the survey. On the last page of the survey, participants were asked whether they wanted to receive information about the study results or to take part in the raffle. If they chose one or both of these options, they were linked to a separate page where they could register their e-mail addresses independently of their survey answers.
Measures
Demographics and information on training and profession. Participants were asked for their age, gender and nationality. Additionally, they completed questions concerning their education and occupation, e.g. their university degree, license status, therapy orientation, and current occupation.
Evidence Based Practice Attitudes Scale (EBPAS-36D). The EBPAS-36 is a brief version of the EBPAS-50, an expanded version of the original 15-item instrument developed to assess mental health and social service providers’ attitudes towards adopting EBP (43). The 36 items of the EBPAS-36 load on 12 subscales of three items each: Requirements (α = .89), Appeal (α = .69), Openness (α = .75), Divergence (α = .65), Limitations (α = .82), Fit (α = .68), Monitoring (α = .77), Balance (α = .65), Burden (α = .81), Job security (α = .89), Organizational support (α = .85), and Feedback (α = .76). Respondents are asked to rate their agreement with statements on a 5-point Likert scale ranging from 0 (‘not at all’) to 4 (‘to a very great extent’). Most items are worded in such a way that a higher total score indicates a more positive attitude towards the adoption of EBP; 15 items are scored reversely. A mean of the subscales can be computed to create a total scale. The German instrument, including scoring instructions, can be found in the supplemental materials 1 and 2.
Implementation Climate Scale (ICS). The ICS is an 18-item instrument measuring the implementation climate in organizations and work groups (53). The original English version was translated into German by the first author (KS) and back-translated by the bilingual English-speaking senior author (AB). In order to adapt the scale for psychotherapists in private practice, a parallel version was constructed that captures the implementation climate in the health system. Respondents are asked to rate their agreement with statements describing how the respondents perceive the climate in the institution they work at with regard to the implementation of evidence-based interventions. A 5-point Likert scale ranging from 0 (‘not at all’) to 4 (‘to a very great extent’) was used. Six subscales can be calculated: Focus on EBP, Educational Support for EBP, Recognition for EBP, Rewards for EBP, Selection for EBP, and Selection for Openness towards EBP. Means of the subscales are computed to create a total scale. In the present study, the internal consistency for the ICS regarding organizations was Cronbach’s α = .91 for the total scale and between α = .77 (Selection for Openness) and α = .92 (Focus on EBP) for the subscales. For the ICS with respect to the health system, Cronbach’s α was = .90 for the total scale and between α = .77 (Rewards for EBP) and α = .88 (Educational Support for EBP) for the subscales. (A separate manuscript for this measure is in preparation).
Intention Scale for Providers (ISP). The ISP is a 70-item instrument assessing individual behavioral intentions for EBP use (54) based on the theory of planned behavior (55). The original English version was translated to German by the first author (KS) and back-translated by the bilingual English-speaking senior author (AB). Responses are given on 7-point and 4-point rating scales. Seven subscales can be calculated. Direct measurement scales of attitudes (A-D, 5 items, α = .69), subjective norms (SN-D, 3 items, α = .85), perceived behavioral control (PBC-D, 4 items, α = .69) and behavioral intention (BI-D, 4 items, α = .89) are generated by calculating the average subscale scores. Indirect measurement scales of attitudes (A-ID, 22 items, α = .80), subjective norms (SN-ID, 18 items, α = .89) and perceived behavioral control (PBC-ID, 14 items, α = .89) are created by multiplying and summing up the belief and influence items (e.g., Normative Beliefs and Motivation to Comply).
Global assessments. As a subjective self-assessment measure, participants were asked to rate their interest in EBP on visual analogue scales for nine questions (see supplemental material 3). A total score was computed (α = .84). At the end of the survey, participants were asked to rate the honesty of their responses (‘How honestly did you answer the questions of this study?’) and their self-reported tendency towards social desirability when answering the survey (‘Did social desirability play a role in the survey?’) on visual analogue scales.
Statistical analysis
All statistical analyses were performed using IBM SPSS 26 for Windows (Chicago, IL, USA). For the confirmatory factor analysis (CFA), SPSS AMOS version 26.0.0 was used. P values < .05 were set as thresholds for statistical significance in all analyses. For the EBPAS-36D, means were computed if there was a maximum of one missing item per scale. Otherwise, respondents would have been excluded from analyses. For item analyses, item difficulties, corrected item-whole correlations and Cronbach’s alpha if item is deleted were calculated. To obtain internal reliability coefficients of the scales and subscales, Cronbach’s alpha was calculated. Values above .70 are regarded as acceptable, higher than .80 as good, higher than .90 as excellent. To assess the factorial validity of EBPAS-36D, the total sample was randomly divided into two samples: With the first one, we conducted an exploratory factor analysis (EFA), followed by a confirmatory one (CFA) with the other half. Differences between both samples regarding age, gender distribution and the EBPAS-36D total scale and subscales were examined with independent t-tests. The suitability of data for EFA was assessed with the Kaiser-Meyer-Oklin (KMO) sample adequacy measure (56, 57) and Bartlett’s test (58). To determine the number of components for the EFA, Horn's parallel analysis and Velicer's MAP test were conducted using the SPSS programs available online (https://people.ok.ubc.ca/brioconn/nfactors/nfactors.html) and the results compared (59). Since parallel analysis of principal factor analysis tends to over-extract factors (60), parallel analysis of principal component analysis was conducted with raw data permutation and 1000 datasets. The EFA was conducted using principal axis factoring analysis with promax correlated factors rotation method. Subsequently, a CFA was conducted to test and compare the original 12-factor structure of EBPAS-36 against a second-order factor structure derived by the EFA, merging the EFA components 4 (Constraints by the institution), 5 (Monitoring) and 6 (Burden) into one second-order factor, and another second-order factor solution that was proposed by Rye et al. (37). Maximum likelihood estimations were used. Since the Mardia-test for multivariate normal distribution is significant (z = 19.16) and all variables exceed either the limits for skewness or for excess as postulated by West et al. ((61), skewness < 2, excess < 7), an increased χ2 value was expected and the Bollen-Stine bootstrap procedure (1000 samples) was performed. The chi-square test statistic was conducted as model fit index. Since this statistic is dependent on sample size and might lead to rejection of plausible models with increasing sample sizes (62), the χ2/df ratio was computed. Smaller values indicate better model fit, with values of 3 being regarded as acceptable and vales of 2 or smaller being regarded as good (62). Additionally, the root mean square error of approximation (RMSEA), the standardized root mean squared residual (SRMR), the comparative fit index (CFI) and the parsimony-adjusted comparative fit index (PCFI) were reported as fit indices. RMSEA values less than .06, SRMR values less than .08, and CFI values greater than .95 indicate good model fits (63). The Akaike Information Criterion (AIC) was computed to compare the models. Smaller scores indicate a better model fit. To assess the convergent validity of EBPAS-36D, Pearson correlation coefficients were calculated between the EBPAS-36D total scale and the direct and indirect scales of attitudes (A-D, A-ID) of the ISP as well as the interest in EBP score. According to Cohen (64), r = .50 indicates high correlations, r = .30 medium correlations, r = .10 low correlations. To test whether the EBPAS-36D total scale is an incremental predictor of the direct scale of behavioral intention to use EBP of the ISP (BI-D), a hierarchical linear regression analysis with the method ENTER was conducted. Gender and age (block 1), having ever worked in science (block 2), and the EBPAS-36D total scale (block 3) were successively included in the regression model to assess incremental improvements of model fit. Mean differences across gender and professional groups on the EBPAS-36D were assessed with independent t-tests. Pearson coefficients were calculated to assess correlations between age as well as demographic/professional variables and the EBPAS-36D. Lastly, Pearson correlations between ICS and EBPAS-36D were assessed.