Study design
We conducted a cross-sectional survey based on the model and questionnaire developed by Paré et al. 15. As shown in Figure 1, the Paré model consists of ten latent constructs or variables: Vision clarity, Change appropriateness, Change efficacy, Top-management support, Presence of an effective champion, Organizational history of change, Organizational politics and conflicts, Organizational flexibility, Collective self-efficacy and Organizational readiness. In the model, Organizational readiness is referred as an endogenous latent variable because it is essentially an outcome variable which the other nine (referred to as exogenous latent variables) measure. The nine exogenous variables fall under 4 facets similar to those discussed by Holt 11.
All latent variables are measured on four Likert-scale questionnaire items (referred to as manifest or indicator variables), except Presence of a champion which is measured on three items. This makes a total of 39 questionnaire items. In our study, the scale was 5-point, with 5 = strongly agree, 4 = agree, 3 = neither agree nor disagree, 2 = disagree, 1 = strongly disagree.
We also added sections for comments to encourage participants to give more details to explain why they scored the organization the way they did. Respondent characteristics including age, gender, tenure, computer usages, and prior EHR experience were also collected since these affect readiness 13,18. Additional material 1 shows the questionnaire.
To minimize the effect of ordering of the items 19, we made five versions of the questionnaire containing exactly the same items of which the order has been shuffled randomly using the free online list randomizer 20. The questionnaires were then printed and distributed sequentially, one from each of the five versions by the first author. Each participant was given a questionnaire once, to fill in and return immediately.
Figure 1: Research model (adapted from Paré et al. 15). Solid arrows show paths from indicator variables (questionnaire item) to the latent variables. Dashed arrows show paths from exogenous latent variable to the endogenous latent variable.
Setting
The study was conducted at the Uganda Cancer Institute (UCI) 21, a 100-bed tertiary public cancer hospital in Kampala, Uganda. The UCI receives about 5000 new cancer patients per year from Uganda and neighboring countries. Clinical documentation is done on paper. However, three years ago the UCI procured an off-the-shelf EHR called Clinic Master which currently is only being used for patient registration, appointments scheduling, and retrospective capture of some clinical details such as diagnosis and treatment, as well as for tracking paper files. Only a few of the staff directly interact with the EHR, mostly the biostatisticians and data entry clerks. The system has provisions for capturing free-text clinical notes, as well as billing, ordering of lab investigations, etc., but these functionalities are not yet being used. Efforts are ongoing to customize Clinic Master to suit the exact needs of the users with regards to cancer care workflow, as well as considerations to switch to a different system altogether.
Participants
UCI staff who are directly involved in patient care or directly use the EHR, were included in the study. There are approximately 250 of these staff, but not all were on site during the survey period (September to October 2018), e.g. due to study leave or other travels, hence one hundred and seventy-five questionnaires were distributed. Staff who normally do not handle clinical data, e.g., cleaners, drivers and other support staff were excluded.
Using G*Power 22 v3.1.9.2, the calculated minimum sample size required to detect a small effect size (R2 of 0.3) in our model where the maximum number of predictors (or arrows pointing at a latent variable) is 9, at a significance level 5% and statistical power of 80%, is 62 cases. Alternatively, following the rule of thumb in 23, the minimum sample size for our model is 90 – i.e. 10 times the maximum number of predictors.
Data analysis
Double data entry was done using Epi Data v4.4.2.1 24 by two independent data clerks and any transcription errors were resolved. We performed descriptive statistics using SPSS v24 25.
For model analysis, we used the R statistical environment 26, specifically the plspm package v0.4.9 27, to perform structural equation modeling (SEM) using the partial least squares (PLS) method. We reverse-coded negatively phrased indicator variables to correct their direction with respect to the latent construct, and removed all cases with missing values in any of the 39 indicator variables (questionnaire items, Additional material 1) since the PLS algorithm requires complete cases. Details of SEM and PLS are provided in Additional material 2, and the R code is provided in the additional files.
We tested our model using measures as described in 23. Table 1 shows the measures for validating the measurement model, i.e. loadings or communalities for indicator reliability, cross loadings for discriminant reliability, Dillon-Goldstein’s rho for composite reliability, and average variance extracted (AVE) for convergent validity.
We tested the structural model using the R2 (also called the coefficient of determination) for the endogenous latent variables, as well as the path coefficients for the exogenous latent variables. The R2 indicates the amount of variance in the endogenous latent variable that is explained by the exogenous latent variables. R2 values <0.3 are considered low, between 0.3 and 0.6 moderate, and above 0.6 are high.
We also conducted sentiment analysis of the comments from the survey using the R package sentimentr 28, to determine the overall polarity i.e. how negative or positive respondents felt about the UCI’s readiness for change.
As a way of triangulation, we used the mean score of each indicator variable and the sentiment score of the corresponding comment to calculate as correlation (Pearson’s r). Similar to model analysis, we also reverse-coded negatively phrased indicator variables for correlation analysis.
Lastly, we conducted deductive content analysis of the comments 29 using the R package RQDA 30 to derive perceived benefits or reasons to implement the EHR as well as action points to get the organization ready.