3.1 Participant characteristics
The research was conducted on two separate samples of healthcare professionals at the Hassan II University Hospital in Fez who utilize Electronic Health Records (EHRs). The first sample (N = 164) underwent exploratory factor analysis, while the second (N = 368) was examined through confirmatory factor analysis. Both samples had similar sociodemographic characteristics, as detailed in Table 1. The average age was 28.34 ± 4.53 for the first sample and 29.21 ± 5.33 for the second sample. The gender distribution showed that 61% of participants in the first sample were female, compared to 62.5% in the second sample. Regarding their clinical positions, the majority of EHR users in both samples were physicians (92.7% vs. 86.1%). Similarly, a significant portion of respondents in both samples worked at the Specialties Hospital of Hassan II University Hospital in Fez (75.6% vs. 78%). Furthermore, nearly half of the healthcare professionals in both samples (53.7% vs. 47.3%) had 1 to 4 years of experience using the EHR at the Hassan II University Hospital in Fez. Moreover, the majority of respondents in both samples had not received training on the use of the Electronic Health Record (86.6% vs. 85.1%).
Table 1 Sociodemographic characteristics of the participants
|
|
Sample 1 (N = 164)
|
|
Sample 2 (N = 368)
|
|
|
Mean ± SD
|
N (%)
|
|
Mean± SD
|
N (%)
|
|
Age a,b
|
28,34 ± 4,53
|
29,21 ± 5,33
|
|
Gender c
|
|
|
|
|
Female
|
|
100 (61%)
|
|
230 (62,5%)
|
Male
|
|
64 (39%)
|
|
137 (37,2%)
|
Clinical position
|
|
|
|
|
Physician
|
|
152 (92,7%)
|
|
317 (86,1%)
|
Nurse
|
|
9 (5,5%)
|
|
44 (12%)
|
Health technician
|
|
3 (1,8%)
|
|
7 (1,9%)
|
Hospital
|
|
|
|
|
Specialties Hospital
|
|
124 (75,6%)
|
|
287 (78%)
|
Oncology Hospital
|
|
23 (15%)
|
|
35 (9,5%)
|
Mother and Child Hospital
|
|
17 (10,4%)
|
|
46 (12,5%)
|
Year of EHR system use
|
|
|
|
|
< 1
|
|
30 (18,3%)
|
|
63 (17,1%)
|
1 - 4
|
|
88 (53,7%)
|
|
174 (47,3%)
|
5 - 9
|
|
36 (22,0%)
|
|
87 (23,6%)
|
5 - 9
|
|
10 (6,1%)
|
|
44 (12%)
|
> 10
|
|
30 (18,3%)
|
|
63 (17,1%)
|
Training in the use of EHR d,e
|
|
|
|
|
Yes
|
|
18 (11%)
|
|
45 (12,2%)
|
No
|
|
142 (86,6%)
|
|
313 (85,1%)
|
a: Tree missing values for N = 164.
b: Hight missing values for N = 368.
c: One missing value for N = 368.
d: Four missing values for N = 164.
e: Nine missing values for N = 368.
|
|
|
|
|
3.2 Exploratory Factor Analysis
Before proceeding with the factor analysis, we assessed the adequacy of the sampling using the Kaiser–Meyer–Olkin (KMO) test. The overall KMO value was 0.866, which is above the threshold of 0.7 suggested by Kaiser [45] and Carricano et al. [46]. Furthermore, the determinant of the correlation matrix was 4.673E-1, exceeding the threshold of 1E-05, which allows us to reject the presence of multicollinearity among the variables according to Field [47]. Additionally, Bartlett's test of sphericity (χ2 = 3413.379, df = 595, p < .001) showed that the correlations between the items were statistically significant, supporting the appropriateness of conducting exploratory factor analysis.
The scree-plot test (Fig. 1) and Velicer's MAP test confirmed the relevance of a seven-factor solution, with eigenvalues exceeding 1, explaining 65.80% of the total variance of the measured variables. This percentage is considered acceptable as it surpasses the thresholds recommended by Gorsuch [48] and Hair et al. [49].
Factor analysis, with principal axis factoring as the extraction method and Promax rotation, yielded an initial structure of the factor matrix (Table 2). An initial loading threshold of at least 0.30 was used. Items that did not surpass this threshold or showed significant loadings on multiple factors were excluded from all factors [42]. After each iteration, the examination of the rotated factor matrix revealed significant factor loadings and changes in the communalities' values. If an observed variable was not significant, it was removed from the measurement model. Each time a variable was removed, the model was revised and run again. This process was repeated multiple times until a structured rotated factor matrix was achieved, with all communalities (factor loadings) of the remaining observed variables exceeding 0.20 [50].
Therefore, items SQ3 (with a communality below 0.2), SQ4 (exhibiting too low factor loading, i.e., below 0.30), and CP1 (having significant cross-loading on two different factors) were excluded from this model. Despite losing three items in the factor composition, the refined model retained the seven-factor structure from our research model version (System Quality, Information Quality, IT Service Quality, Organization, Environment, User Satisfaction, and Clinical Performance). Thus, the factor names were retained, and the seven-factor model explained 68% of the variance (Table 2), which is considered acceptable according to Child [50].
Internal Consistency
The questionnaire's reliability was assessed in terms of internal consistency by calculating Cronbach's alpha coefficient for each dimension and item-total (corrected) correlations for each item (Table 2). Overall, the values are satisfactory, with all alpha coefficients exceeding the threshold of 0.7, thus reinforcing the psychometric qualities of the internal reliability of the questionnaire [51]. Furthermore, the minimum item-total correlation calculated was 0.416 (Table 2), surpassing the 0.30 threshold [39]. According to both methods of assessing internal consistency, all constructs and items in our questionnaire demonstrated sufficient and acceptable internal consistency.
3.3 Confirmatory Factor Analysis
Convergent Validity
The results of the confirmatory factor analysis of the first-order measurement model showed that the lowest factor loadings were 0.54, observed for items ORG1 and CP8. Furthermore, all standardized regression coefficients (Fig. 2) exceeded the recommended threshold of 0.50, as indicated by [39].
Table 2 Factor loadings, communalities (h2), item-total correlation, and Cronbach's alpha of the measurement tool
Item label
|
Factors
|
H2
|
Item-total correlation
|
Alpha
|
CP
|
IQ
|
US
|
ISQ
|
ENV
|
ORG
|
QS
|
|
|
|
|
CP3
|
,902
|
|
|
|
|
|
|
,853
|
,789
|
,870
|
CP 4
|
,889
|
|
|
|
|
|
|
,833
|
,796
|
|
CP 2
|
,739
|
|
|
|
|
|
|
,678
|
,706
|
|
CP 5
|
,659
|
|
|
|
|
|
|
,560
|
,657
|
|
CP 6
|
,613
|
|
|
|
|
|
|
,465
|
,636
|
|
CP 8
|
,450
|
|
|
|
|
|
|
,459
|
,570
|
|
CP 7
|
,350
|
|
|
|
|
|
|
,338
|
,454
|
|
IQ3
|
|
,850
|
|
|
|
|
|
,728
|
,789
|
,883
|
IQ4
|
|
,832
|
|
|
|
|
|
,767
|
,791
|
|
IQ5
|
|
,772
|
|
|
|
|
|
,598
|
,699
|
|
IQ1
|
|
,672
|
|
|
|
|
|
,538
|
,675
|
|
IQ2
|
|
,590
|
|
|
|
|
|
,537
|
,646
|
|
US2
|
|
|
,879
|
|
|
|
|
,755
|
,797
|
,891
|
US5
|
|
|
,745
|
|
|
|
|
,720
|
,792
|
|
US4.
|
|
|
,730
|
|
|
|
|
,642
|
,712
|
|
US1
|
|
|
,712
|
|
|
|
|
,611
|
,655
|
|
US3
|
|
|
,622
|
|
|
|
|
,651
|
,718
|
|
ISQ2
|
|
|
|
,826
|
|
|
|
,697
|
,742
|
,839
|
ISQ3
|
|
|
|
,771
|
|
|
|
,636
|
,652
|
|
ISQ1.
|
|
|
|
,757
|
|
|
|
,726
|
,714
|
|
ISQ4
|
|
|
|
,673
|
|
|
|
,667
|
,731
|
|
ISQ5
|
|
|
|
,531
|
|
|
|
,406
|
,444
|
|
ENV2
|
|
|
|
|
,860
|
|
|
,659
|
,646
|
,735
|
ENV3
|
|
|
|
|
,753
|
|
|
,594
|
,626
|
|
ENV4
|
|
|
|
|
,561
|
|
|
,392
|
,457
|
|
ENV1.
|
|
|
|
|
,422
|
|
|
,345
|
,445
|
|
ORG2
|
|
|
|
|
|
,821
|
|
,716
|
,670
|
,767
|
ORG3.
|
|
|
|
|
|
,741
|
|
,703
|
,681
|
|
ORG4
|
|
|
|
|
|
,460
|
|
,259
|
,416
|
|
ORG1
|
|
|
|
|
|
,436
|
|
,452
|
,525
|
|
SQ1
|
|
|
|
|
|
|
,812
|
,789
|
,611
|
,756
|
SQ2
|
|
|
|
|
|
|
,544
|
,436
|
,611
|
|
Eigenvalue
|
11,102
|
2,668
|
2,054
|
1,720
|
1,497
|
1,424
|
1,385
|
|
|
|
Variance (total = 68%)
|
34,69%
|
8,34%
|
6,42%
|
5,37%
|
4,68%
|
4,45%
|
4,33%
|
|
|
|
Abbreviations. SQ: System Quality; IQ: Information Quality; ISQ: IT Service Quality; ORG: Organization; ENV: Environment;
US: User Satisfaction; CP: Clinical Performance.
|
Furthermore, the reliability and convergent validity of the questionnaire were also confirmed, with high and acceptable values for composite reliability (CR) (0.75-0.89) and average variance extracted (AVE) (0.51-0.63), respectively (Table 3). As a result, the entire factor analysis process was validated, and the measurement tool adapted satisfactorily to the data.
Table 3. Construct validity and reliability
|
CR
|
AVE
|
MSV
|
MaxR(H)
|
Latent Constructs
|
|
|
|
|
|
SQ
|
IQ
|
ISQ
|
ORG
|
ENV
|
US
|
CP
|
SQ
|
0,75
|
0,60
|
0,25
|
0,75
|
0,77
|
|
|
|
|
|
|
IQ
|
0,89
|
0,63
|
0,47
|
0,90
|
0,47***
|
0,79
|
|
|
|
|
|
ISQ
|
0,84
|
0,57
|
0,38
|
0,85
|
0,41***
|
0,61***
|
0,76
|
|
|
|
|
ORG
|
0,75
|
0,51
|
0,35
|
0,79
|
0,27***
|
0,49***
|
0,46***
|
0,71
|
|
|
|
ENV
|
0,76
|
0,52
|
0,10
|
0,84
|
0,21**
|
0,24***
|
0,27***
|
0,312***
|
0,72
|
|
|
US
|
0,87
|
0,58
|
0,47
|
0,88
|
0,50***
|
0,69***
|
0,59***
|
0,587***
|
0,28***
|
0,76
|
|
CP
|
0,89
|
0,59
|
0,44
|
0,95
|
0,43***
|
0,46***
|
0,34***
|
0,551***
|
0,22***
|
0,66***
|
0,77
|
* p < 0.05; ** p < 0.01; *** p < 0.001
Abbreviations. AVE: the square root of the average variance extracted, CP: Clinical Performance, CR: Composite reliability, ENV: Environment, IQ: Information Quality, ISQ: Information technology Service Quality, MSV: Maximum Shared Variance, ORG: Organization, SQ: System Quality, US: User Satisfaction
|
|
Discriminant Validity
Discriminant validity was evaluated following the Fornell and Larcker [44] criterion, which involves checking if the average variance extracted (AVE) is greater than the intercorrelations values between latent variables. In Table 3, the values in bold represent the square root of the AVE for each factor, while the other values correspond to inter-correlations between latent variables. The most important correlation observed among factors was 0.66 (between US and CP), while the lowest value among the square roots of the AVE values was 0.71. Therefore, we can affirm the discriminant validity of all factors in the model because the diagonal values of the matrix exceeded the values located outside the diagonal in the corresponding rows and columns.
Moreover, the maximum shared variance (MSV) was lower than the average variance extracted (AVE). Discriminant validity can also be assessed using the HTMT test. Thus, a value of the HTMT test below 0.85 or 0.90 indicates good discriminant validity [43]. As indicated in Table 4, all values in the matrix are below 0.85, confirming the discriminant validity between all factors in the proposed model.
In conclusion, all conditions required to ensure the validity of the constructs in the overall measurement model have been met. Reliability (measured by Cronbach's alpha and Joreskog's Rho) is very satisfactory. Convergent validity (assessed by average variance extracted) as well as discriminant validity (evaluated according to the Fornell and Larcker criterion and the HTMT test) of the constructs are all acceptable.
Table 4 Discriminant validity: Heterotrait-Monotrait (HTMT) Criterion results
|
System Quality
|
Information Quality
|
IT Service Quality
|
User Satisfaction
|
Organization
|
Environment
|
Clinical Performance
|
System Quality
|
1
|
|
|
|
|
|
|
Information Quality
|
0,483
|
1
|
|
|
|
|
|
IT Service Quality
|
0,431
|
0,614
|
1
|
|
|
|
|
User Satisfaction
|
0,280
|
0,504
|
0,463
|
1
|
|
|
|
Organization
|
0,269
|
0,284
|
0,299
|
0,374
|
1
|
|
|
Environment
|
0,495
|
0,699
|
0,556
|
0,612
|
0,319
|
1
|
|
Clinical Performance
|
0,436
|
0,557
|
0,398
|
0,625
|
0,294
|
0,716
|
1
|
Assessment of the Overall Measurement Model's Fitness.
To assess the quality of the current model, we will use three categories of fit indices, namely absolute fit, parsimony, and incremental fit indices. These fit indices are considered acceptable if their values meet the following statistical fit thresholds: the chi-square to degrees of freedom ratio (χ2/df) should be less than 3, the comparative fit index (CFI) should be greater than 0.95, the values of the normed fit index (NFI), Tucker Lewis index (TLI), and goodness-of-fit Index (GFI) should be equal to or greater than 0.90, the adjusted goodness-of-fit index (AGFI) should be greater than 0.85, and the root mean square error of approximation (RMSEA) should be less than 0.05 [39, 52].
Based on these acceptance thresholds, some values in the initial model M1 do not conform to acceptable levels (Table 5). To improve the quality of these fit indices, we made some modifications, primarily by removing items (ISQ5 = 0.41; ORG4 = 0.44; ENV1 = 0.43; CP7 = 0.42) that had standardized loading values below the threshold of 0.5 recommended by Hair et al. [39]. Additionally, we introduced some covariances between the errors of items that exhibited high modification indices. Table 5 summarizes the transition from model M1 to model M5 with the modifications made. Ultimately, the overall measurement model M5 proved to fit the needed thresholds well and thus offers a better fit to the empirical data than the other models.
Table 5 Goodness-of-fit statistics for the modified primary-order CFA models
|
|
|
Absolute Fit
|
|
Incremental Fit
|
|
Parsimonious Fit
|
Model
|
Modification
|
RMSEA
|
GFI
|
AGFI
|
|
TLI
|
CFI
|
NFI
|
|
χ2/df
|
M1
|
Initial model
|
,06
|
,84
|
,81
|
0,89
|
,90
|
,84
|
2,38
|
M2
|
M1 + removed items with loading <0,4 (ISQ5; ORG4; ENV1; PC7).
|
,06
|
,88
|
,85
|
,92
|
,93
|
,88
|
2,17
|
M3
|
M2+ correlated errors of CP6 and CP8 (MI= 48.92)
|
,05
|
,89
|
,86
|
,93
|
,94
|
,89
|
2,01
|
M4
|
M3 + correlated errors of ISQ1 and ISQ2 (MI= 20,41)
|
,05
|
,89
|
,87
|
,94
|
,95
|
,90
|
1,92
|
M5
|
M4 + correlated errors of IQ2 and IQ5 (MI=13,77)
|
,05
|
,90
|
,87
|
,94
|
,95
|
,90
|
1,87
|
Criterion for goodness of fit
|
≤.05
|
≥ .90
|
≥ .85
|
≥ .90
|
≥ .90
|
≥ .90
|
<3
|
Abbreviations: CFI, Comparative Fit Index; TLI, Tucker and Lewis’s Index of fit; NFI, Normed Fit Index; IFI, Incremental Fit Index; RMSEA, Root Mean Square Error of Approximation; MI, Modification Index; M, Model; ISQ, IT Service Quality; ORG, Organization; ENV, Environment; IQ, Information Quality; CP, Clinical Performance.
|