Electronic monitoring versus manual paper-based monitoring for hand hygiene compliance: A comprehensive health economic assessment analysis

doi:10.21203/rs.3.rs-3872269/v1

Background: Monitoring hand hygiene compliance (HHC) of healthcare providers (HCPs) in healthcare facilities is critical for hand hygiene (HH) promotion. However, less is known about the cost and effectiveness of different HHC monitoring tools. In this study, we aimed to compare various health economic indicators corresponding to electronic system-based monitoring (ESM) and manual paper-based monitoring (MPM) for HHC to provide evidence-based advice for HHC monitoring measures targeted selecting.

Methods: A before and after self-controlled prospective study in 40 clinical departments with 4,524 HCPs was conducted from December 2022 to January 2023 (MPM implementation phase) and March 2023 to May 2023 (ESM implementation phase). The cost-effectiveness, cost-efficiency, the extent of the Hawthorne effect, and indirect cost-benefit of the two monitoring methods were compared.

Results: Compared with ESM, every 1% increase in the HHC of MPM resulted in a 39,881.62 CNY reduction in cost. Acceptability of ESM was higher when the cost was 40,000 CNY or less. The cost-efficiency ratio of MPM in all departments (155,775.56 CNY) was higher than that of ESM (36,796.76 CNY). The Hawthorne effect of MPM of HHC in all departments (43.99%) was more pronounced than that of ESM (35.69%), with a statistically significant difference(p < 0.01).

Conclusions: ESM exhibited notable advantages over MPM in terms of cost-effectiveness, cost-efficiency, cost-benefit, and the Hawthorne effect.

hand hygiene

hand hygiene compliance

health economic evaluation

Healthcare-associated infections (HAIs) have become a globally recognized public health issue. The occurrence of HAIs, on the one hand, affects the recovery of patients, prolongs hospital stays, increases patients’ financial burden, and poses a threat to their physical health and life safety, and on the other, it leads to a higher incidence of medical disputes, reduces hospital bed turnover rates, and wastes a significant amount of healthcare resources [1]. Hand hygiene (HH) is the most basic, direct, affordable, and effective method for reducing HAIs [2]. The global hand hygiene compliance (HHC) rate among HCPs in healthcare facilities is reportedly suboptimal, ranging from 20% to 40%, which falls significantly short of the HH standards (90%–95%) for HCPs during medical activities established by the World Health Organization (WHO). The first global strategy for infection prevention and control (IPC) placed special emphasis on the requirement for continual surveillance of HH indicators to facilitate prompt feedback [3]. Additionally, it recommended that monitoring the HHC of HCPs in healthcare facilities is a crucial element of HH promotion programs [4]. The inconsistent distribution of regional healthcare resources globally has resulted in varying risks of HAIs among different disease groups within healthcare facilities. A critical issue when it comes to monitoring HH is selecting the most efficient monitoring tool for a particular situation, taking into account the available workforce, healthcare resources, acceptable HAI rates, and cost-effectiveness.

Currently, the most widely accepted approaches for HH monitoring around the globe are direct observation and indirect observation [5]. Direct observation, that is, “trained observers directly observing HHC” is still regarded as the “gold standard” for HHC monitoring and is the one of the most reliable measures of evaluating HHC [6–7]. This monitoring method is fairly straightforward and can be implemented regardless of the size or existing structure of the hospital. However, it is prone to the Hawthorne effect and is overly tedious and time-consuming, making it difficult to obtain accurate results. Also, this approach makes it challenging to conduct large-scale surveys [8]. Indirect monitoring includes but is not limited to estimating the consumption of paper towels, hand sanitizers, and soap, among other products, for maintaining HH or assessing the needs based on the nursing operation database. These methods require less time and resources compared with direct observation but have some biases, such as a lack of evaluation of patient factors and workload. In recent years, with the continuous development of technology, intelligent monitoring systems have been widely used in healthcare facilities and are considered potential alternative solutions for measuring and improving HHC. Previous studies documented the usage of electronic badges with alcohol vapor sensors to monitor whether HCPs practiced HH [9]. Some researchers have used smart rings worn on the fingers to monitor HHC. Compared with the most common manual paper-based direct observation method, using information systems for HHC monitoring has prominent advantages such as sustainability, reduced Hawthorne effect, saved human resources, improved data collection efficiency, and increased data traceability. Further investigations should be performed to evaluate the practicality, cost, acceptability, and cost-effectiveness of HHC monitoring information systems, even though they provide a novel way of accurately monitoring HHC.

This study was conducted to compare various health economic indicators of electronic system monitoring (ESM) and manual paper-based monitoring (MPM) for HHC and to utilize the results to furnish healthcare institutions with varying levels of resources and infection risk with HHC monitoring evidence-based advice.

Study design

A before and after self-controlled prospective study in 40 clinical departments with 4,524 HCPs of hospital, China, was conducted from December 2022 to January 2023 (MPM implementation phase) and March 2023 to May 2023 (ESM implementation phase; Fig. 1). MPM analysis relied on handwritten HHC records, with data being analyzed manually. ESM was accomplished using a self-developed app (REDACTED) embedded in nursing operation mobile including the following functions: HHC monitoring, automatic statistics, inquiry, analysis, reminders, and so on. This study has been approved by the Clinical Research Ethics Committee of Zhongnan Hospital of Wuhan University ( approval number: 2023136K).

The cost-effectiveness, cost-efficiency, Hawthorne effect, indirect cost–benefit analysis of the two monitoring methods were compared.

Basic data

Basic data included information concerning beds and HCPs of 40 clinical departments. In addition, these 40 clinical departments were divided into three levels of high risk (11 clinical departments), medium risk (23 clinical departments), and low risk (six clinical departments) according to the previous literature [11–13] (sTable 1).

Costs

The cost of ESM included the cost of software, electricity, and labor cost. ① the cost of developing software (C₁): the development cost of the software was 50,000 CNY, the use cycle is at least three years; our study lasted three months, and the average payment per department was 31.25 CNY; ② the cost of electricity (C₂): a full charge required one degree of electricity, and according to the power supply standard of the hospital, the cost of a full charge of the device was 0.8 CNY [14], where the charging time of device was once a month; ③ labor cost (C₃): According to the investigation and statistics of Wuhan Municipal Bureau of Human Resources and Social Security, the average monthly salary of employees in 2022 was 8,845 CNY [15], and the hourly labor cost was about 40 CNY.

The cost of MPM included the cost of paper and the labor cost. ① cost of paper: the cost of an A4 paper was estimated to be 0.04 CNY. According to the design of WHO HHC questionnaire [16], it took two sheets of paper to conduct an MPM; ② labor cost (C₃). The costs and calculation of the two monitoring methods are summarized in Table 1.

Table 1

Cost calculation of two monitoring methods
Monitoring method	Risk levels for different departments	No. of departments	Time consumed (hour)	No. of paper (piece)	Total cost (CNY)
ESM	Total number of departments	40	411.14	/	17,702.92
	High-risk departments	11	113.07	/	C₁(31.25/department)×11 + C₂(0.8/kilowatt hour)×3(month) + C₃(40/person/hour)×113.07
	Medium-risk departments	23	245.56	/	C₁(31.25/department)×23 + C₂(0.8/kilowatt hour)×3(month) + C₃(40/person/hour)×245.56
	Low-risk departments	6	52.51	/	C₁(31.25/department)×6 + C₂(0.8/kilowatt hour)×3(month) + C₃(40/person/hour)×52.51
MPM	Total number of departments	40	545.50	169	21,826.68
	High-risk departments	11	178.00	53	C₄(0.04/piece)×53 + C₃(40/person/hour)×178.00
	Medium-risk departments	23	296.66	93	C₄(0.04/piece)×93 + C₃(40/person/hour)×303.33
	Low-risk departments	6	70.84	23	C₄(0.04/piece)×23 + C₃(40/person/hour)×70.84

HHC Monitoring Effectiveness

HHC includes hand wash and alcohol-based hand rub. HHC means that HPWs adhere to the established HH guidelines and the steps, time, and scope of HH are in line with regulations. HHC cost-effectiveness was quantified by monitoring HH action and opportunity by respective part-time IPCs from clinical departments and based on formula 1 [17]:

HHC monitoring effectiveness = $\frac{\text{H}\text{H} \text{a}\text{c}\text{t}\text{i}\text{o}\text{n} }{\text{H}\text{H} \text{o}\text{p}\text{p}\text{o}\text{r}\text{t}\text{u}\text{n}\text{i}\text{t}\text{y}}$×100% (formula 1)

HHC Monitoring Efficiency

Cost-efficiency was evaluated by time consumed on HHC monitoring and investigated by 10 full-time IPCs from departments of IPC and nursing. HHC was quantified through ESM by directly inputting the observed number of HH actions, and the system automatically generated the result of HHC and uploaded the same to the computer. The use of MPM required the observers to record the observed and actual HH moments in a paper observation sheet and calculate the results manually. HHC monitoring efficiency (formula 2) was estimated as the time spent for each monitoring moment:

HHC monitoring efficiency = $\frac{\text{T}\text{h}\text{e} \text{n}\text{u}\text{m}\text{b}\text{e}\text{r} \text{o}\text{f} \text{m}\text{o}\text{n}\text{i}\text{t}\text{o}\text{r}\text{i}\text{n}\text{g} \text{m}\text{o}\text{m}\text{e}\text{n}\text{t}\text{s}}{\text{T}\text{o}\text{t}\text{a}\text{l} \text{t}\text{i}\text{m}\text{e} \text{c}\text{o}\text{n}\text{s}\text{u}\text{m}\text{e}\text{d}}$×100%(formula 2)

Hawthorne effect

The Hawthorne effect refers to a change in behavior as a motivational response to the interest, care, or attention received through observation and assessment [18]. The Hawthorne effect may deliberately increase HH when the observer perceives that he/she is being observed during HH monitoring. In this study, the HHC during the period when HCPs were not observed, i.e., the unobservation period of different risk departments was derived from previous references (sTable 2). The extent of the Hawthorne effect was calculated using formula 3 [19]:

Hawthorne effect = $\frac{\text{T}\text{h}\text{e} \text{n}\text{u}\text{m}\text{b}\text{e}\text{r} \text{o}\text{f} \text{H}\text{H}\text{C}(\text{d}\text{u}\text{r}\text{i}\text{n}\text{g} \text{t}\text{h}\text{e} \text{o}\text{b}\text{s}\text{e}\text{r}\text{v}\text{a}\text{t}\text{i}\text{o}\text{n} \text{p}\text{e}\text{r}\text{i}\text{o}\text{d}-\text{d}\text{u}\text{r}\text{i}\text{n}\text{g} \text{t}\text{h}\text{e} \text{u}\text{n}\text{o}\text{b}\text{s}\text{e}\text{r}\text{v}\text{a}\text{t}\text{i}\text{o}\text{n} \text{p}\text{e}\text{r}\text{i}\text{o}\text{d}）}{\text{T}\text{h}\text{e} \text{n}\text{u}\text{m}\text{b}\text{e}\text{r} \text{o}\text{f} \text{H}\text{H}\text{C} \text{d}\text{u}\text{r}\text{i}\text{n}\text{g} \text{t}\text{h}\text{e} \text{o}\text{b}\text{s}\text{e}\text{r}\text{v}\text{a}\text{t}\text{i}\text{o}\text{n} \text{p}\text{e}\text{r}\text{i}\text{o}\text{d}}$×100% (formula 3)

Indirect benefit

Indirect benefit was evaluated in terms of disease burden due to HAIs during the monitoring period. HAI rates were estimated as the number of new infections in the total number of hospitalized patients in a certain period of time (formula 4):

HAI rates = $\frac{\text{T}\text{h}\text{e} \text{n}\text{u}\text{m}\text{b}\text{e}\text{r} \text{o}\text{f} \text{n}\text{e}\text{w} \text{i}\text{n}\text{f}\text{e}\text{c}\text{t}\text{i}\text{o}\text{n}\text{s} \text{i}\text{n} \text{t}\text{h}\text{e} \text{s}\text{a}\text{m}\text{e} \text{p}\text{e}\text{r}\text{i}\text{o}\text{d}}{\text{T}\text{o}\text{t}\text{a}\text{l} \text{n}\text{u}\text{m}\text{b}\text{e}\text{r} \text{o}\text{f} \text{h}\text{o}\text{s}\text{p}\text{i}\text{t}\text{a}\text{l}\text{i}\text{z}\text{e}\text{d} \text{p}\text{a}\text{t}\text{i}\text{e}\text{n}\text{t}\text{s}}$×100% (formula 4)

Data on HAI rates in this study were exported from the hospital information system of hospital. HAI rates without HH of different risk departments were estimated from references (sTable 3). The average disease burden per case of HAIs was 39,800 CNY estimated from references [24].

Outcomes

Cost-effectiveness ratio, cost-efficiency ratio, Hawthorne effect, and indirect cost–benefit ratio are the main outcomes and target effects and based on formulas 5 to 9:

Cost-effectiveness ratio = $\frac{\text{C}\text{o}\text{s}\text{t}}{\text{E}\text{f}\text{f}\text{e}\text{c}\text{t}\text{i}\text{v}\text{e}\text{n}\text{e}\text{s}\text{s}}$×100% (formula 5);

Cost-efficiency ratio = $\frac{\text{C}\text{o}\text{s}\text{t}}{\text{E}\text{f}\text{f}\text{i}\text{c}\text{i}\text{e}\text{n}\text{c}\text{y}}$×100% (formula 6);

Cost–benefit ratio (CBR) = $\frac{\text{B}\text{e}\text{n}\text{e}\text{f}\text{i}\text{t}}{\text{C}\text{o}\text{s}\text{t}}$×100% (formula 7);

$\text{B}\text{e}\text{n}\text{e}\text{f}\text{i}\text{t}$ = $\frac{(\text{H}\text{A}\text{I} \text{r}\text{a}\text{t}\text{e}\text{s} \text{d}\text{u}\text{r}\text{i}\text{n}\text{g} \text{t}\text{h}\text{e} \text{s}\text{u}\text{r}\text{v}\text{e}\text{y} \text{p}\text{e}\text{r}\text{i}\text{o}\text{d}-\text{H}\text{A}\text{I} \text{r}\text{a}\text{t}\text{e}\text{s} \text{w}\text{i}\text{t}\text{o}\text{u}\text{t} \text{H}\text{H})\times \text{D}\text{i}\text{s}\text{e}\text{a}\text{s}\text{e} \text{b}\text{u}\text{r}\text{d}\text{e}\text{n} \text{d}\text{u}\text{e} \text{t}\text{o} \text{H}\text{A}\text{I}\text{s}}{\text{C}\text{o}\text{s}\text{t}}$×100% (formula 8);

Incremental cost-effectiveness ratio=$\frac{{\text{C}\text{o}\text{s}\text{t}}_{\text{M}\text{P}\text{M}}-{\text{C}\text{o}\text{s}\text{t}}_{\text{E}\text{S}\text{M}}}{{\text{E}\text{f}\text{f}\text{e}\text{c}\text{t}\text{i}\text{v}\text{e}\text{n}\text{e}\text{s}\text{s}}_{\text{M}\text{P}\text{M}}-{\text{E}\text{f}\text{f}\text{e}\text{c}\text{t}\text{i}\text{v}\text{e}\text{n}\text{e}\text{s}\text{s}}_{\text{E}\text{S}\text{M}}}$×100% (formula 9).

Data and Analyses

Cost calculation was recorded into Microsoft Excel by two research members, and the formula was set in Excel and calculated to avoid any computational errors. SPSS (version 16, SPSS Inc) software was used for statistical analysis, and the chi-square test was used to detect any statistical differences within and between groups. Based on the previous studies and combined with the data characteristics of this study, the decision tree model [10] was used to analyze cost-effectiveness. The decision node of the decision tree model was HHC monitoring, and the branches of the opportunity node were ESM and MPM. The nodes of these two branches are divided into HHC and its lack thereof, i.e., no-HHC. The decision tree model was performed using TreeAge Pro 2022 (Fig. 2).

Comparison of cost-effectiveness of the two monitoring methods among different departments

The total cost spent on ESM for the 40 departments (17,702.92 CNY) was 4,123.76 CNY lower than that of MPM (21,826.68 CNY). The HHC of MPM (80.16%) was higher than that of ESM (69.82%), and the difference was statistically significant (p < 0.01; Table 2). The HHC of ESM in high-, medium-, and low-risk departments was 61.33%, 76.82%, and 56.59%, respectively (p < 0.01). The HHC of MPM in high-, medium-, and low-risk departments was 78.79%, 83.42%, and 65.85%, respectively (p < 0.01).

In high- and medium-risk departments, the cost-effectiveness ratio of ESM (7,977.90 CNY and 13,794.60 CNY, respectively) was lower than that of MPM (9,039.61 CNY and 14,549.05 CNY, respectively), indicating that the average cost of ESM was lower when improving HHC. On the contrary, in low-risk departments, the cost-effectiveness ratio of ESM (3,910.77 CNY) was higher than that of MPM (3,899.06 CNY), indicating that the average cost of ESM was higher. Compared with ESM, the incremental cost of MPM in all departments was 4,123.76 CNY, the incremental effectiveness was 10.34%, and the incremental cost-effectiveness ratio was 39,881.62 CNY. Every 1% increase in the HHC compliance of MPM was found to result in a 39,881.62 CNY reduction in cost. However, in low-risk departments, every 1% increase in the HHC of MPM was noted to result in a 3,828.29 CNY reduction in cost. As shown in Fig. 3, both ESM and MPM are undominated strategies. Our tornado analysis revealed that the cost of MPM was the only factor that had the largest impact of the overall strategy (Fig. 4).

Table 2

Cost-effectiveness analysis and incremental cost-effectiveness analysis of the two monitoring methods among the different risk departments.
Total	Cost (CNY)	Effectiveness (%)	P (Effectiveness)	P (Effectiveness)	Cost-effectiveness ratio (CNY)	Cost-effectiveness ratio-IQR (CNY)	Incremental cost (CNY)	Incremental effectiveness (%)	Incremental cost-effectiveness ratio (CNY)
Total
MPM	21,826.68	3,807/4,749 (80.16%)	<0.05		27,228.89	599.77 (421.18, 903.77)	4,123.76	10.34	39,881.62
ESM	17,702.92	8,173/11,706 (69.82%)			25,355.49	290.16 (290.16 ,762.19)
High risk
MPM	7,122.12	832/1,056 (78.79%)	<0.05	>0.05 (vs MPM of medium risk)	9,039.61	785.89 (489.94, 1,135.21)	2,228.95	17.46	12,766.04
ESM	4,893.17	2,400/3,913 (61.33%)		<0.05 (vs ESM of medium risk)	7,977.90	395.97 (251.49, 840.38)
Medium risk
MPM	12,137.05	2,521/3,022 (83.42%)	<0.05	<0.05 (vs MPM of low risk)	14,549.05	521.90 (407.39, 832.16)	1,540.31	6.60	23,338.03
ESM	10,596.74	5,176/6,738 (76.82%)		<0.05 (vs ESM of low risk)	13,794.60	462.40 (245.74, 749.38)
Low risk
MPM	2,567.51	376/571 (65.85%)	>0.05	<0.05 (vs MPM of high risk)	3,899.06	605.35 (481.04, 802.75)	-354.50	9.26	-3,828.29
ESM	2,213.01	597/1,055 (56.59%)		>0.05 (vs ESM of high risk)	3,910.77	411.94 (350.59 ,673.18)

The cost-effectiveness acceptability curve (CEAC, Fig. 5) illustrates that when the cost of all departments, namely, high-risk departments, medium-risk departments, and low-risk departments was 40,000 CNY, 15,000 CNY to 20,000 CNY and 5,000 CNY to 10,000 CNY, respectively, the choice of ESM had a higher cost-effectiveness.

Comparison of cost-efficiency analysis of using two monitoring methods among different departments

Between the two monitoring methods, the efficiency of ESM (48.11%) in all departments was higher than that of MPM (14.20%), and the difference was statistically significant (p < 0.01). The cost-efficiency ratio of MPM in all departments (155,775.56 CNY) was higher than that of ESM (36,796.76 CNY), indicating that ESM was efficient and low-cost. In high-risk departments, the maximum gap in the cost-efficiency ratio of the MPM and ESM were 72,013.35 CNY and 8,858.02 CNY, respectively (Table 3).

Table 3

Cost efficiency analysis of using two monitoring methods among the different risk departments.
Total	Cost (CNY)	Efficiency (%)	P (Effectiveness)	P (Effectiveness)	Cost-efficiency ratio (CNY)	Cost-efficiency ratio-IQR (CNY)
Total
MPM	21,826.68	4,649/32,730 (14.20%)	<0.05		155,775.56	4,001.20 (3,967.57, 7,618.27)
ESM	17,702.92	11,706/24,331 (48.11%)			36,796.76	426.95 (316.65, 864.62)
High risk
MPM	7,122.12	1,056/10,680 (9.89%)	<0.05	<0.05 (vs MPM of medium risk)	72,013.35	6,741.71 (4,001.20, 8,002.40)
ESM	4,893.17	3,913/7,084 (55.24%)		<0.05 (vs ESM of medium risk)	8,858.02	373.00 (340.00, 718.31)
Medium risk
MPM	12,137.05	3,022/18,200 (16.60%)	<0.05	<0.05 (vs MPM of low risk)	73,144.76	4,001.20 (3,367.87, 6,935.13)
ESM	10,596.74	6,738/14,223 (47.37%)		<0.05 (vs ESM of low risk)	22,370.15	469.00 (247.48, 893.11)
Low risk
MPM	2,567.51	571/3,850 (14.83%)	<0.05	<0.05 (vs MPM of high risk)	17,312.95	4,001.20 (3,751.20, 4,173.37)
ESM	2,213.01	1,055/3,024 (34.89%)		<0.05 (vs ESM of high risk)	6,342.82	396.50 (314.20, 482.25)

Comparison of the extent of Hawthorne effect of using two monitoring methods among different departments

The extent of Hawthorne effect of MPM of HHC in all departments (43.99%) was higher than that of ESM (35.69%), and the difference between the two monitoring methods was noted to be statistically significant (p < 0.01). Among departments with different risk levels, the extent of Hawthorne effect of MPM (59.45%) and ESM (47.90%) in high-risk departments were all higher than those in other departments, and no statistical difference existed between the two monitoring methods in high-risk departments (p = 0.940). The extent of Hawthorne effect of MPM (41.46%) and ESM (36.20%) in medium-risk departments was the lowest, and there existed a statistical difference between the two monitoring methods (p < 0.01; Table 4).

Table 4

Hawthorne effect analysis of using the two monitoring methods among different departments.
Total	HHC during observation (%)	HHC during no observation (%)	Hawthorne effect (%)	P (Hawthorne effect)	P (Hawthorne effect)
Total
MPM	3,807/4,749 (80.16%)	2,132/4,749 (44.90%)	43.99%	<0.05
ESM	8,173/11,706 (69.82%)	5,256/11,706 (44.90%)	35.69%
High risk
MPM	832/1,056 (78.79%)	337/1,056 (31.95%)	59.45%	>0.05	<0.05 (vs MPM of medium risk)
ESM	2,400/3,913 (61.33%)	1,256/3,931 (31.95%)	47.90%		<0.05 (vs ESM of medium risk)
Medium risk
MPM	2,521/3,022 (83.42%)	1,481/3,022 (49.01%)	41.46%	<0.05	<0.05 (vs MPM of low risk)
ESM	5,176/6,738 (76.82%)	3,302/6,738 (49.01%)	36.20%		<0.05 (vs ESM of low risk)
Low risk
MPM	376/571 (65.85%)	200/571 (35.00%)	46.85%	<0.05	>0.05 (vs MPM of high risk)
ESM	597/1,055 (56.59%)	369/1,055 (35.00%)	38.15%		<0.05 (vs ESM of high risk)

Comparison of cost-benefit analysis of the two monitoring methods among different departments

When ESM was used as the HHC monitoring approach, the HAI rates (1.39%) in all departments were higher than that when MPM was used (1.34%), but no statistical difference was observed (p = 0.562). When comparing the CBR of MPM and ESM, the average CBR of ESM in all departments (665.44 CNY) was lower than that of MPM (2,454.37 CNY), indicating that in the current analysis, the CBR of ESM was better than MPM. Among departments with different risk levels, the CBR difference between the two monitoring methods in medium-risk departments was found to be the greatest. The average CBR of ESM (744.73 CNY) was 2,560.72 CNY lower than that corresponding to MPM (3,305.45 CNY; p < 0.01; Table 5).

Table 5

Cost-benefit analysis of using the two monitoring methods among the different risk departments.
Total	Cost (CNY)	HAI rates with hand hygiene (%)	HAI rates without hand hygiene (%)	P (HAI rates)	P (HAI rates)	Reduced disease burden (CNY)	CBR (CNY)	CBR-IQR (CNY)
Total
MPM	21,826.68	467/34,863 (1.34%)	1,813/34,863 (5.2%)	<0.05		53,570,800	2,454.37	23,147.48 (6,893.53, 91,204.45)
ESM	73,534.92	443/31,816 (1.39%)	1,654/31,816 (5.2%)			48,197,800	665.44	19,753.51 (8,580.36, 47,236.66)
High risk
MPM	7,122.12	232/7,864 (2.95%)	747/7,864 (9.54%)	<0.05	<0.05 (vs MPM of medium risk)	20,497,000	2,877.94	19,975.62 (9,944.58, 69,970.16)
ESM	20,246.97	166/7,855 (2.11%)	749/7,855 (9.54%)		<0.05 (vs ESM of medium risk)	23,203,400	1,146.02	12,854.52 (7,296.18, 30,604.40)
Medium risk
MPM	12,137.05	203/24,715 (0.82%)	1,211/24,715 (4.9%)	<0.05	<0.05 (vs MPM of low risk)	40,118,400	3,305.45	23,386.44 (5,262.27, 126,390.33)
ESM	42,700.14	239/21,193 (1.13%)	1,038/21,193 (4.9%)		>0.05 (vs ESM of low risk)	31,800,200	744.73	22,711.16 (12,525.18, 79,744.09)
Low risk
MPM	2,567.51	32/2,284 (1.40%)	62/2,284 (2.7%)	>0.05	>0.05 (vs MPM of high risk)	1,194,000	465.04	36,705.09 (17,038.61, 71,270.98)
ESM	10,587.81	38/2,768 (1.37%)	75/2,768 (2.7%)		<0.05 (vs ESM of high risk)	1,472,600	139.08	31,127.70 (19,209.28, 43,404.11)
CBR, cost benefit ratio

The WHO Hand Hygiene Research Agenda for 2023–2030 underscores the need to advance HH research and increase the effectiveness of HHC monitoring through the use of information technology in the next two decades [29]. HHC monitoring is widely recognized as a highly challenging task that requires expertise and training, as this determines the accuracy of the monitoring results, and it also demands human resources and time investment, as this determines the completeness of the monitoring and the timeliness of feedback. This study marked the initial examination of a comprehensive health economic analysis to evaluate the utilization of ESM and MPM in HHC monitoring. The findings revealed that ESM outperformed MPM in terms of cost-effectiveness, cost-efficiency, cost–benefit, and the extent of Hawthorne effect.

In terms of cost-effectiveness, the total cost of ESM was lower than that of MPM. Particularly in high- and medium-risk departments, the average cost of improving HHC using ESM was lower than that of MPM. In line with this study is the observation that most commonly used information technology systems for HHC motoring can significantly improve HHC among HCPs (OR = 3.06, p < 0.001) [30]. With continued improvement of electronic monitoring systems, combining electronic monitoring with observational methods may provide the best information as part of a multimodal strategy to improve and sustain HHC rates among HCWs. Moreover, this study further distinguished the cost-effectiveness of clinical departments of different risk levels. The high-risk settings such as intensive care units has the stricter regulations for HAI prevention and control with an increasing HH opportunities moments. Thus the cost-effective advantage of ESM is more significant [31]. However, in low-risk departments, MPM resulted in higher cost effectiveness [32]. Undeniably, the disadvantages of electronic monitoring in combination with direct observation include, among others, the cost of installation [33]. Hence, MPM were recommended to be applied in low-risk medical departments.

As for cost-efficiency, ESM was noted to be superior to MPM, especially in high- and medium-risk departments, where their efficiency advantage was even more pronounced. We theorized that this could be attributed to the higher-risk divisions, such as the intensive care unit, executing more intrusive procedures, accommodating patients with weakened immune systems, and consuming a great amount of resources in the hospital. The staff in this department were short-handed (with a nurse-to-patient ratio of 2.5 to 3:1 [34]), and there were shortages of equipment, drugs, and other resources, all of which put an extra strain on HHC, making it necessary to keep a close eye on personnel and time expenditure. ESM could help managers better understand the implementation, promptly identify problems and non-compliant behavior, and avoid waste and misuse of resources in high-risk departments. Moreover, through analysis of HH data, resource allocation could be optimized, and work efficiency can be improved [35]. This study’s analysis of cost-efficiency is consistent with the actual clinical demands. ESM could simultaneously monitor the HHC of multiple individuals online, and the data could be automatically analyzed, provided with feedback, and traced, thereby saving a significant amount of workforce and time costs. Kardaś-Słoma [36] also remarked that ESM can automatically collect data, improving the accuracy and timeliness of monitoring. The data collected through an ESM can be further analyzed and used for trend prediction. This analysis can help identify hotspots and high-risk periods for HH issues, enabling targeted preventive measures to improve HHC and correctness.

The Hawthorne effect, which refers to the alteration of behavior on being aware that one is being observed, was chosen as an evaluation indicator for monitoring biases. ESM, being discreetly conducted on nursing mobile devices, offers a certain level of secrecy when conducting compliance monitoring. On the other hand, MPM relies on paper records, making the Hawthorne effect more pronounced. Therefore, ESM was more effective in controlling the Hawthorne effect. Interestingly, it was found that HHC was lower under ESM compared with that under MPM, which may be attributed to a weaker Hawthorne effect associated with ESM. Casaroto’s research [37] in the ICU context demonstrated an HHC rate of 56.3% under MPM and 51.0% under ESM, with the Hawthorne effect being the influencing factor. Another research group [38] demonstrated a downward trend in HHC rates among HCPs after the introduction of intelligent monitoring systems. This phenomenon mainly occurs when direct observation and ESM are used simultaneously to measure HHC. Direct observation produces the Hawthorne effect, resulting in higher measurement results compared with those obtained through intelligent monitoring systems. These findings suggest that information technology-enabled monitoring provides a more accurate reflection of the actual situation, enabling the establishment of more precise baseline data for subsequent HHC interventions. Electronic HHC motoring systems can monitor HHC on all work shifts without a Hawthorne effect and provide significantly more data regarding HHC [39].

Finally, a comparison of the occurrence of HAIs during the application of the two monitoring methods was conducted. Although no significant difference was observed in the occurrence of HAIs between ESM and MPM during the study period, the cost-effectiveness of ESM outweighed that of MPM when considering the cost and the burden of patient diseases caused by HAIs. Likewise, Salinas-Escudero [40] observed that within one month of implementing ESM, the number of infections decreased by 46–79 individuals, resulting in cost savings of $308,927 to $546,795 for preventing HAIs.

However, it is important to note some limitations of this study. First, a large number of studies focused on using electronic monitoring systems to monitor HHC, including application-assisted direct observation, camera-assisted observation, sensor-assisted observation, and real-time locating system [39]. We cannot evaluate the health economics of all electronic monitoring systems. Hence, the findings cannot be extrapolated to other electronic monitoring systems. However, we intend to conduct an exploratory health economic evaluation in terms of tools for monitoring HHC and provide a reference for future research. Second, the cost of ESM did not consider the potential repair costs resulting from device damage in the later stages. Long-term prospective observational studies are also being conducted to provide a comprehensive health economic assessment of the long-term application of ESM in HHC.

In conclusion, for high-risk departments with multiple HH moments and higher requirements for HHC monitoring frequency and coverage, ESM exhibits notable advantages over MPM in terms of cost-effectiveness, cost-efficiency, cost–benefit, and the Hawthorne effect.

ESM

Electronic system-based monitoring

MPM

Manual paper-based monitoring

HHC

Hand hygiene compliance

HCPs

Healthcare providers

HAIs

Healthcare-associated infections

HH

Hand hygiene

WHO

World Health Organization

IPC

Infection prevention and control

IQR

Interquartile range

CEAC

Cost-effectiveness acceptability curve

CBR

Cost benefit ratio

Ethics approval and consent to participate

This study has been approved by the Clinical Research Ethics Committee of Zhongnan Hospital of Wuhan University ( approval number: 2023136K). Informed consent was obtained from all patients who met the inclusion criteria.

Consent for publication

All authors approved the final manuscript and the submission to this journal.

Availability of data and materials

Data is provided within the manuscript or supplementary information files.

Competing interests

The authors declare that they have No competing financial interests exist.

Funding

This study was supported by the National Natural Science Foundation of China (52178080) and National Institute of Hospital Administration Management of China (GY2023049).

Authors’ contributions

W and F designed this study and protocol development; C and L were responsible for data collection; C and H were responsible for data analysis; C, L, C and K conducted the manuscript writing; P, C, C, T, and Y critically revised the manuscript; W and F provided final approval for this version to be published; all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Acknowledgements

Not applicable.

Ray-Griffith SL, Wendel MP, Stowe ZN, Magann EF: Chronic pain during pregnancy: a review of the literature. Int J Womens Health. 10: 153-164 (2018).
Bearman G, Doll M, Cooper K, Stevens MP: Hospital Infection Prevention: How Much Can We Prevent and How Hard Should We Try? Curr Infect Dis Rep. 21: 2 (2019).
World Health Organization: Global report on infection prevention and control. Available at: https:/apps.who.int/iris/handle/10665/354489 (2009). Accessed 8 June 2023.
Wang C, Jiang W, Yang K, Yu D, Newn J, Sarsenbayeva Z, Goncalves J, Kostakos V: Electronic Monitoring Systems for Hand Hygiene: Systematic Review of Technology. J Med Internet Res. 23: e27880 (2021).
Boyce JM, Laughman JA, Ader MH, Wagner PT, Parker AE, Arbogast JW: Impact of an automated hand hygiene monitoring system and additional promotional activities on hand hygiene performance rates and healthcare-associated infections. Infect Control Hosp Epidemiol. 40: 741-747 (2019).
World Health Organization: WHO guidelines on hand hygiene in health care. Geneva: World Health Organization. pp. 1–270 (2009).
Stewardson A, Sax H, Longet-Di Pietro S, Pittet D: Impact of observation and analysis methodology when reporting hand hygiene data. J Hosp Infect. 77: 358-9 (2011).
Boyce JM: Electronic monitoring in combination with direct observation as a means to significantly improve hand hygiene compliance. Am J Infect Control. 45: 528-535 (2017).
Kardaś-Słoma L, Lucet JC, Perozziello A, Pelat C, Birgand G, Ruppé E, Boëlle PY, Andremont A, Yazdanpanah Y: Universal or targeted approach to prevent the transmission of extended-spectrum beta-lactamase-producing Enterobacteriaceae in intensive care units: a cost-effectiveness analysis. BMJ Open. 7: e017402 (2017).
Alenzi EO: Cost-effectiveness analysis of polycystic ovary syndrome management and the risk of gestational diabetes in pregnant women: a decision-tree model. Expert Rev Pharmacoecon Outcomes Res. 21: 995-999 (2021).
Su X, Zong CX, Wang X, Sun XG, Lv FY, Sun L: Application of risk assessment system in classifed management of healthcare-associated infection in different departments. Chinese Journal of Infection Control. 21: 844-852 (2022).
Peng C, Jiang B: Risk assessment in the management of healthcare-associated infectionn in primary care hospitals. Modern Nurse. 21: 844-852 (2022).
Gan YJ, Lu YY, Liang DY: Risk assessment on healthcare-associated infection in different clinicadepartments based on rank-sum ratio method. Chinese Journal of Infection Control. 17: 418-422 (2018).
National Development and Reform Commission: Notice of the National Development and Reform Commission on Issuing the Guiding Opinions on the Trial Implementation of Ladder Tariffs for Residential Household Electricity. Available at: https://zfxxgk.ndrc.gov.cn/web/iteminfo.jsp?id=19692 (2023). Accessed 29 June 2023.
Wuhan Municipal Bureau of Human Resources and Social Security: Wuhan Municipal Bureau of Human Resources and Social Security average monthly salary for employees in 2022. Available at: https://verywind.cn/ee/vjfyy2f73jjyxv22fe2.html (2023). Accessed 23 June 2023.
Specification of hand hygiene for healthcare workers WS/T 313-2019. Chinese Journal of Infection Control. 19: 93-98 (2020).
Wang Y, Yang J, Qiao F, Feng B, Hu F, Xi ZA, Wu W, Ni ZL, Liu L, Yuan Y: Compared hand hygiene compliance among healthcare providers before and after the COVID-19 pandemic: A rapid review and meta-analysis. Am J Infect Control. 50: 563-571 (2022).
Sedgwick P, Greenwood N: Understanding the Hawthorne effect. BMJ. 351: h4672 (2015).
Hagel S, Reischke J, Kesselmeier M, Winning J, Gastmeier P, Brunkhorst FM, Scherag A, Pletz MW: Quantifying the Hawthorne Effect in Hand Hygiene Compliance Through Comparing Direct Observation With Automated Hand Hygiene Monitoring. Infect Control Hosp Epidemiol. 36: 957-62 (2015).
El-Saed A, Noushad S, Tannous E, Abdirizak F, Arabi Y, Al Azzam S, Albanyan E, Al Jahdalil H, Al Sudairy R, Balkhy HH: Quantifying the Hawthorne effect using overt and covert observation of hand hygiene at a tertiary care hospital in Saudi Arabia. Am J Infect Control. 46: 930-935 (2018).
Bruchez SA, Duarte GC, Sadowski RA, Custódio da Silva Filho A, Fahning WE, Belini Nishiyama SA, Bronharo Tognim MC, Cardoso CL: Assessing the Hawthorne effect on hand hygiene compliance in an intensive care unit. Infect Prev Pract. 2: 100049 (2020).
Kwok YLA, Juergens CP, McLaws ML: Automated hand hygiene auditing with and without an intervention. Am J Infect Control. 44: 1475-1480 (2016).
Yang XD, Huang YY, Huang Q, Hu Y: Study on the effect of hand hygiene behavior intervention on hand hygiene implementation rate of medical staff in rehabilitation department. Modern Nurse. 09: 132-134 (2016).
Lin Z, Zhu XQ, Chen ZN, Huang QQ, Jiang H, Zhang GQ, Huang GY, Meng LY: Evaluation of direct economic burden induced by nosocomial infection ofpatients with tumor-related diseases based on DRG. Infect Control Hosp Epidemiol. 39: 701-709 (2018).
Wang J, Liu F, Tartari E, Huang J, Harbarth S, Pittet D, Zingg W. T.: The Prevalence of Healthcare-Associated Infections in Mainland China: A Systematic Review and Meta-analysis. Infect Control Hosp Epidemiol. 39: 701-709 (2018).
Qiu J, Wu L, Yang YJ, Luo J, Kang H, Zhang ZJ. Effect of sanitary wipes use on nosocomial infection rates in intensive care units. Chinese Journal of Infection Control. 39: 278-281 (2022).
Ding LL: A Nosocomial Infection Point-Prevalence Survey in Departments of InternaMedicine and Surgery:Results and Analysis. Chinese Journal of Infection Control. 9: 1268-1270 (2008).
Zhao XP, Cao LZ, Yao MH: Investigation and analysis of nosocomial infections of in-patients fromneurological rehabilitation department. Chinese Journal of Infection Control. 24: 4824-4825+4832 (2014).
World Health Organization: WHO research for hand hygiene in health care 2023–2030: summary. Available at: https://www.who.int/publications/i/item/9789240073715 (2023). Assessed 5 May 2023.
Lin TY, Lin CT, Chen KM, Hsu HF: Information technology on hand hygiene compliance among health care professionals: A systematic review and meta-analysis. J Nurs Manag. 29: 1857-1868 (2021).
Edmisten C, Hall C, Kernizan L, Korwek K, Preston A, Rhoades E, Shah S, Spight L, Stradi S, Wellman S, Zygadlo S: Implementing an electronic hand hygiene monitoring system: Lessons learned from community hospitals. Am J Infect Control. 45: 860-865 (2017).
Ward MA, Schweizer ML, Polgreen PM, Gupta K, Reisinger HS, Perencevich EN: Automated and electronically assisted hand hygiene monitoring systems: a systematic review. Am J Infect Control. 42: 472-8 (2014).
Boyce JM: Electronic monitoring in combination with direct observation as a means to significantly improve hand hygiene compliance. Am J Infect Control. 45: 528-535 (2017).
Møller-Sørensen H, Korshin A, Mogensen T, Høiby N: New technology markedly improves hand-hygiene performance among healthcare workers after restroom visits. J Hosp Infect. 92: 337-9 (2016).
Casaroto E, Generoso JR Jr, Serpa Neto A, Prado M, Gagliardi GM, de Menezes FG, Gonçalves P, Hohmann FB, Olivato GB, Gonçalves GP, Xavier N, Fernandes Miguel M, Edmond MB, Marra AR: Comparing human to electronic observers to monitor hand hygiene compliance in an intensive care unit. Antimicrob Steward Healthc Epidemiol. 2: e161 (2022).
Kardaś-Słoma L, Lucet JC, Perozziello A, Pelat C, Birgand G, Ruppé E, Boëlle PY, Andremont A, Yazdanpanah Y: Universal or targeted approach to prevent the transmission of extended-spectrum beta-lactamase-producing Enterobacteriaceae in intensive care units: a cost-effectiveness analysis. BMJ Open. 7: e017402 (2017).
Casaroto E, Generoso JR Jr, Serpa Neto A, Prado M, Gagliardi GM, de Menezes FG, Gonçalves P, Hohmann FB, Olivato GB, Gonçalves GP, Xavier N, Fernandes Miguel M, Edmond MB, Marra AR. Comparing human to electronic observers to monitor hand hygiene compliance in an intensive care unit. Antimicrob Steward Healthc Epidemiol. 2: e161 (2022).
Boyce JM, Laughman JA, Ader MH, Wagner PT, Parker AE, Arbogast JW: Impact of an automated hand hygiene monitoring system and additional promotional activities on hand hygiene performance rates and healthcare-associated infections. Infect Control Hosp Epidemiol. 40: 741-747 (2019).
Wang C, Jiang W, Yang K, Yu D, Newn J, Sarsenbayeva Z, Goncalves J, Kostakos V: Electronic Monitoring Systems for Hand Hygiene: Systematic Review of Technology. J Med Internet Res. 23: e27880 (2021).
Salinas-Escudero G, la Rosa-Zamboni D, Carrillo-Vega MF, Gamiño-Arroyo AE, Toledano-Toledano F, Ortega-Riosvelasco F, Granados-García V, Villa-Guillén M, Garduño-Espinosa J Cost-effectiveness analysis of a hand hygiene monitoring system in a tertiary pediatric hospital in Mexico. Front Public Health. 11: 1117680 (2023).

No competing interests reported.

Supplementaty.docx

Electronic monitoring versus manual paper-based monitoring for hand hygiene compliance: A comprehensive health economic assessment analysis

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Methods

Study design

Basic data

Costs

HHC Monitoring Effectiveness

HHC Monitoring Efficiency

Hawthorne effect

Indirect benefit

Outcomes

Data and Analyses

Results

Comparison of cost-effectiveness of the two monitoring methods among different departments

Comparison of cost-efficiency analysis of using two monitoring methods among different departments

Comparison of the extent of Hawthorne effect of using two monitoring methods among different departments

Comparison of cost-benefit analysis of the two monitoring methods among different departments

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1