The Health Information Management System and HIV/Aids Monitoring: Insights From Ethiopia

doi:10.21203/rs.3.rs-2756791/v1

Download PDF

Research Article

The Health Information Management System and HIV/Aids Monitoring: Insights From Ethiopia

https://doi.org/10.21203/rs.3.rs-2756791/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 26 Apr, 2024

Read the published version in South African Journal of Information Management →

Version 1

posted

You are reading this latest preprint version

Background

A well-performing Health Information System (HIS) provides timely, complete, accurate and easily retrievable data. It also reflects the impact of guidelines and policies on the functioning of the health system. However, an HIS in Low- and Middle-Income Countries (LMIC), including Ethiopia, is highly complex and influenced by pressures from donors, politics and technical factors. Hence, these countries experience persistent challenges in producing quality data and difficulties using HMIS data from their HISs. This study aimed to evaluate how HMIS was perceived and utilized in HIV/AIDS monitoring in Ethiopia, and views regarding the influence of technical, organizational factors and behavioural factors on the use of HMIS.

Methods

A qualitative evaluative case study using focus group discussions (8) with data producers and users was conducted in selected public hospitals and health centres in Addis Ababa. A purposive critical case sampling was used to recruit participants. Data were collected from 64 participants made up of data clerks, HMIS heads, health information technicians, voluntary counselling and testing (VCT), Prevention of Mother to Child Transmission (PMTCT) and Antiretroviral Treatment (ART) heads, medical directors and disease prevention heads. Qualitative analytical techniques using ATLAS.ti Version 8 were used for data analysis, and codes, categories, and themes were validated with the participants.

Result

Key findings revealed that HIV/AIDS-specific indicators, ICT and other related resources were critical barriers to the successful use of the HMIS. Participants believed these technical issues impacted the quality of data adversely and, subsequently, the conversion of that data to information and using it to monitor the HIV/AIDS programme’s performance. Ethiopian guidelines emphasize the benefits of DHIS2 in ensuring the production of good-quality data. However, the system will not produce useful data without clear and meaningful indicators. Timely reporting was hampered by parallel reporting systems such as DHIS2 and SmartCare, a lack of interoperability between these systems, and a hybrid system consisting of paper documents and electronic data.

Conclusion

Technical factors such as data collection tools, the HMIS design, computer software, and reporting formats are all strategic decisions taken by the organization, which in turn influence the behavior of users and the use of HMIS. The health facilities did not process information as expected. However, staff performed the HMIS tasks with the tools available as they tried to make sense of data coming out of the HMIS. Challenges, such as a lack of human resources, specialized skills, and support, could be improved to effectively use the HMIS for generating useful information for HIV monitoring.

Technical determinants

Health Information System

Health Management Information System

HIV/AIDS

Monitoring & Evaluation

A well-performing HIS provides timely, complete, accurate and easily retrievable data. It also reflects the impact of guidelines and policies on the functioning of the health system and population health [1]. The HIS has multiple uses, such as enabling planning, supporting patient and health facility management, and supporting global reporting. However, the information from such a system will be of little value if it is not available in formats that meet the needs of multiple users, namely policymakers, planners, managers, healthcare providers, communities, and individuals [2].

According to the WHO, country monitoring is the basis for regional and global monitoring of priority health issues. M&E is needed to report health-related Sustainable Development Goals (SDGs) progress. Countries need functional surveillance mechanisms and accountability to ensure that priority health programmes are implemented as planned against stated objectives and desired results [3]. The MEASURE Evaluation identified that strengthening was required for the HIV M&E system [4]. Too complex indicators challenge the effectiveness of monitoring the HIV programme [5].

The HMIS has a critical role in monitoring and evaluating a health system, as it integrates data from all healthcare levels and provides information for managing programmes, facilities, policies, and resources [6]. Ending the AIDS epidemic will stimulate wider international health and development efforts, indicating what can be realized through evidence-based action and multi-sectorial partnerships [7]. HIV providers’ preferences and clinical workflows reflected through an integrated system intended for this purpose should provide evidence-based information. This will permit its usage in specific contexts, for specific users to solve specific questions, thereby improving data torrents quality and efficiency, leading to upgraded patient care and health outcomes [8].

Globally, most of the eastern Mediterranean region countries have national health information system plans. Still, some plans have limited scope, particularly in data collection, analysis, and capacity-building [9]. HISs in low- and middle-income countries (LMICs) are highly complex and influenced by pressures from donors and political and administrative role players. Therefore, accountability and good governance are imperative [10].

In the southern and eastern African regions, Botswana struggled to create a functional HIS over the years. Substantial challenges happened in all aspects of the system, undermining growth. Prolonged challenges such as the absence of central coordination, feeble leadership, poor policy and regulatory frameworks, and insufficient resources were observed [11]. A study undertaken in Kenya indicates that most health information systems collect aggregate data that do not allow patient-level quality of care evaluation [12]. Poorly developed health management information systems in Uganda resulted in poor data quality. Considerable IT and other related resources are needed to bring them to appropriate levels [13].

Studies in Ethiopia reveal technical and behavioural factors related to reporting, such as poor documentation, poor competency (a skills gap), and lack of experience, resulting in poor data quality and being below the national targets [14]. This study focused on how the system was used to generate quality data. Therefore, this paper reports on perceptions and use of HMIS in HIV/AIDS monitoring and evaluation in Ethiopia.

The study recognized the need to understand how participants viewed the impact of contextual factors on the use of HMIS. Performance of Routine Information System Management (PRISM) framework from Aqil, Lippeveld and Hozumi, revised by MEASURE Evaluation was adopted. The framework is guided by technical, organizational, and behavioural components to measure performance [15–16].

Study setting

The study was conducted in two settings, namely hospitals and health centres, referred to as health facilities, and governed by the Addis Ababa regional health bureau. Key to the setting were three factors: HIV/AIDS service, HMIS implementation and accessibility. Three public hospitals and six health centres were chosen as the research site. Hospitals were purposefully selected based on the burden of HIV cases, experience with HMIS implementation and HIV/AIDS management. Health centres were selected mainly due to their long-term experience with HIV management and varying levels of ART HIV/AIDS burden.

Design

A qualitative evaluative case study was conducted to explore and describe how data generators and users utilized HMIS to generate relevant and reliable data for HIV/AIDS monitoring. This approach permitted the examination of data management processes, thus giving insights into the complexities of generating quality and useful data from HMIS. Therefore, HMIS use was the unit of analysis or the case. Health facilities represented a natural setting. Yin regards the goals of a case study as understanding a social phenomenon and real-life events such as organizational and managerial processes [17]. Yazan contends that researchers have yet to fully agree on the case study’s design and implementation [18].

The population for this study was professionals operating at public health facilities, using the country’s HIS and the HMIS, such as those who mainly produce data (HMIS staff like data clerks, HIT, and the HMIS/M&E focal person) and others who mainly use data for the HIV/AIDS programme monitoring and evaluation (clinicians and the facility management team). A non-probability purposive critical case sampling was used to recruit well-informed and experienced participants who worked on the HMIS and mainly gave service, support and decision-making in the HIV/IDS programme. The rationale for using critical case sampling was that a few informants could be the source of theoretical saturation, thus, being decisive in explaining health management information system processes in these facilities [19]. Specific criteria were also set for recruitment, and only professionals working in the health facilities stipulated above were included in the research sample. The recruitment of research participants continued until data saturation. Therefore, the sample size was not predetermined.

Data Collection Instruments/tools

The interview guide was developed in English and then translated into Amharic. During translation, the researcher and language professional translated it independently and compared different versions for consistency and similarity of wording [20]. The main focus of questioning was on their perceptions of HMIS and how it was used to generate relevant and reliable data for HIV and AIDS monitoring and participants’ views regarding the influence of technical, organizational and behavioural factors on the use of HMIS.

Data Collection Procedure

Semi-structured focus group interviews were conducted with 64 participants over eight groups from December 01, 2018, up to March 31, 2019. Focus groups were particularly suited for this study because they are flexible, inexpensive, elaborative and capable of producing rich data [21]. The participants who were willing to participate in the study signed informed consent forms, which outlined the purpose and processes of the study. The researcher outlined the rules for the discussions, such as that only one person was allowed to talk at a time, mobile phones had to be put on silent, and participants had to raise a hand when needing to speak.

Data collection only started after ethical clearance was obtained from the university and the regional health bureau granted permission. A central grand tour question was planned to allow probing.

Researchers considered participants’ comfort, ease of access to the venue, and levels of interruption during interviews [22]. The meetings were conducted in a suitable venue that provided privacy and was convenient for most participants.

Field notes were taken by an assistant who signed a confidentiality agreement with the researcher to ensure the ethical principles were observed. Based on the participation, each group interview took roughly 1–2 hours. Follow-up sessions were conducted until data saturation was achieved. Finally, eight FGDs were conducted with 64 participants from nine public health facilities. The interviews were audio recorded.

Data analysis

The audio-recorded data were transcribed verbatim and translated into English by the language professional and the researcher. The transcripts were numbered separately for each FGD. Data producers’ focus groups were numbered and labelled: FG1 data producers to FG4 data producers. Data users’ focus group transcripts were also numbered one to four and labelled: FG1 data users to FG4 data users. MMR verified the transcripts, and after preliminary analysis, saturation was determined, and data from all groups were merged.

Thematic analysis was carried out using ATLAS.ti. Version 8 software to develop broad topics condensed into descriptive codes. Coded sections were repeatedly read, cross-checked to derive meanings, and aligned with the research objectives. The codes were categorized into themes, subthemes and categories that represented participants’ views on HMIS and technical, organizational and behavioral factors that influence the use of HMIS for HIV and AIDS monitoring and evaluation.

The analysis process was theory-driven and pattern-matching, following key concepts of the PRISM framework. The themes which emerged were cross-checked to derive meaning and were aligned with the research aim. The researchers reassessed and revised the codes, categories and themes until a consensus was reached.

Ethical Consideration

Ethics approval was granted by the University of South Africa (UNISA)( HSHDC/ 792/2017). Permission to conduct the study was granted by the Addis Ababa Health Bureau. Written informed consent was obtained from all participants. Privacy and confidentiality were assured throughout the study; the rights of participants were protected, and the integrity of the study was maintained.

Trustworthiness

The study used the Lincoln and Guba model (1985) criteria for trustworthiness: credibility, dependability, transferability, and confirmability [23]. There was prolonged engagement with participants during focus group interviews until data saturation occurred. They communicated in the language in which they felt comfortable. Dependability was ensured by double-checking data and summarising key findings for participants to confirm they were accurate reflections. A detailed description of the process used, such as the procedure and purpose of selecting participants, the method and length of the data collection procedure, data analysis, and presentation, was produced to enhance transferability. MMR assessed the research methods and procedures. Participants’ verbatim statements enhanced dependability.

This study intended to examine how the HMIS generates relevant and reliable data and explore stakeholders’ views regarding technical and organisational factors that influence the use of HIV/AIDS health information.

Two themes with related categories emerged from the data: the organization and structure of HMIS in Ethiopia and the use of routine health information for HIV/AIDS monitoring.

Biographical Data Of Participants

Sixty-four participants, comprising 33 data producers and 31 data users, participated in eight focus group discussions. They were all recruited from nine health facilities. Nineteen were male, and 45 were female. The average age of participants was 32, and the level of education was higher in the data users group than in data producers; 33 had a diploma, and 30 had a BSc degree. Finally, one had a master’s degree. The experience level in the HMIS ranged from one year to ten years, respectively. More data producers received training on data quality checks, reporting systems and format use. Experience in using HMIS ranged from 2.03 years (medical directors) to 5.25 years (voluntary counselling and testing heads), respectively.

Theme 1: Organization And Structure Of Hmis In Ethiopia

Use of the HMIS at the facility level

Data users understood that the HMIS is a management function used to provide users with reliable information to support decisions regarding the performance of the HIV/AIDS programme and to manage local factors that influence service utilisation. Data producers confirmed and said that routine facility reporting is critical for effective data management of antiretroviral therapy. Data flow within the facilities follows routine reporting processes, with each group having specific roles.

“HMIS is implemented in every activity in the facility and includes vital registration, providing information on program related factors. A comprehensive software is being utilized and the recently implemented DHIS2 is a huge resource for data management” (FG4 Data users).

“Mostly data clerks do data capturing and review for consistencies. We do paperwork and produce reports and there is (an) HMIS officer who also analyses the data” (FG2 Data users).

“Voluntary counselling and testing registers and tally sheets are the main HMIS record format available to capture data. Summary sheets for daily work are used to provide monthly reports” (FG1 Data users).

Electronic and paper-based data reporting

Participants indicated that currently, the reporting uses both manual and electronic inputs. As mentioned earlier, some units, such as voluntary counselling and testing and prevention of mother-to-child transmission, still use paper-based registers and tally sheets. On the other hand, data clerks use SmartCare software. The HMIS focal persons enter all data in the DHIS2 (Web-District Health Information System). However, various challenges related to paper-based and parallel reporting were expressed. Most believe manual reporting compromises quality, takes time, and increases the risks of data entry errors. Some believe that paper-based reporting is still maintained due to software incompatibility with the reporting system. Others contended that the paper-based system could be a backup during power shortages or system failures.

“We are manually working on some reports to HMIS. We use prevention of mother to child transmission register and tally sheet. The register is (an) MNCH integrated part that is used in antenatal care” (FG2 Data users).

‘We use SmartCare software and it has more detailed information” (FG4 Data Producers).

“Doing manual may result in a lot of mistakes, quality will be compromised… paper-based reporting system takes time through each level of reporting and takes months to reach federal level” (FG1 Data Producers).

“Paper-based system is needed for backup purposes, in case software gets corrupted. Hard copy is used when there is power cut” (FG3 Data producers).

Data-capturing system

Most participants indicated that using the correct data capture and recording tools is important for producing useful information in HIV/AIDS monitoring. Data from tallies and registers are collected and entered into the DHIS2 system in the HMIS room. Routine data is collected daily using paper-based recording (registers and tally sheets). They found it useful in providing information for planning, costing and research, which is also used to evaluate the performance of the HIV/AIDS programme.

“We need to follow specific formats and registers when we capture data. HIV-related data are collected mostly in (the) antiretroviral therapy department at the data clerk level, then sent to (the) HMIS unit for compilation” (FG2 Data producers).

“Register and tallies are collected from each department to enter [the data] in the DHIS2 [system] in the HMIS room.” (FG3 Data producers).

“We find it useful in that information from HMIS can be used for planning, budgeting, monitoring and research…” (FG4 Data users).

Challenges with reporting

Participants raised several issues related to the generation of quality reports. Some were related to health professionals, timeliness, reporting formats and misunderstandings between health professionals and the HMIS unit. They also emphasised the mismatch between the software and report format regarding data elements. In addition, they lamented the outdated software, which does not match the level of performance needed for accurate data analysis, especially medication records. Participants recommended appropriate software to replace manual reporting. Most participants mentioned that the prevention of mother-to-child transmission reporting format changes regularly, and it can cause confusion and errors among those who are not trained.

The following quotes support the findings:

“This software reports mostly the type of drugs only. But viral load data is being done manually in formats … the printing from [Firms/Suppliers] takes from 3–6 months and that causes delays” (FG3 Data Producers).

“Software has problems in wrongly reducing the number of reports. DHIS requires (an) update in data analysis by method, ages, and categories.” (FG4 Data Producers).

“Provision of medical service for communicable disease report is still in hard copy and has drawback…reporting electronically is better” (FG1 Data Producers).

“In PMTCT (prevention of mother-to-child transmission) unit, the ministry supporting partner has their own reporting format. The data are not integrated and errors happen due to different types of reporting formats” (FG2 Data users).

Management of updates

Almost all data producers believed that SmartCare needs updating, and they also need regular training on updates. Data such as viral load, cohort chart and appointments were sometimes lost, influencing everything from client dosages to tracking lost to follow-up cases. The updates were perceived as necessary to enhance capacity and confidence in data management processes. However, they lamented frequent updates in routine data collection:

“…It is good to provide training parallel with new updates. …all facility problems are related to updates and training. Specific training needs to be done” (FG1 Data Producers).

“We need to be skilled in updates on indicator display, use of the system to track distribution and use of resources…refresher training needs to be planned in every year by the Health Bureau” (FG4 Data Producers).

“The prevention of mother-to-child transmission format is frequently updated. For (a) new untrained person, it will be difficult to understand and work with” (FG2 Data users).

Usability of the system

Most users expressed that the HMIS is very useful for registrations using specific data elements such as age and disease codes. They found it helpful in providing information for planning, costing and research, which is also used to evaluate the performance of the HIV/AIDS programme. However, they suggested it can also be complicated, considering the number of inexperienced users. The main issues were the system’s complexity, related disease classification, and too many indicators. These issues affect the efficiency of generating quality data and continuous use of information.

The following quotes support the findings:

“HMIS supports the use (of) information for various reasons, such as having the right data based on accurate registrations. When reporting, disease classification and age data support (the) measurement of the disease’s prevalence. We find it useful in that information from HMIS can be used for planning, budgeting, monitoring and research…” (FG4 Data users).

“After HMIS was started, new terminology was brought and resulted in many disease codes … Staffs have difficult(ty) to comprehend all the disease codes. In the past, HMIS reporting format was simple and less paperwork” (FG3 Data users).

It Infrastructure And System

All groups emphasized that the IT infrastructure needed to be designed to support data management processes, including the use of information for monitoring and evaluation. Currently, the system lacks functionalities, especially appointments, CD4 (Cluster of Differentiation 4) counts and viral load. Participants reported that some facilities had no computers, and staff had to report using manual formats. On the other hand, the data producers highlighted issues with connectivity.

The following verbatim quotes illustrate the findings:

“The system does not have a functionality to save (the) appointment, CD4 [Cluster of Differentiation 4] count and viral load data. It does not produce the report accordingly. A child who is overweight will take (an) adult drug regimen, but the system is rigid and doesn’t recognize and record INH data.” (FG3 Data users).

“The other part, we don’t have (the) computer system to report …we requested one with (the) software application. HMIS staffs (sic) are expected to save reports on compact disk and flash if there is (a) lack of internet service to send the report…we have started to use (a) data display chart for monitoring the patient load....” (FG4 Data users).

Data Quality Assurance

All participants agreed that data generated from health facilities should be of high quality to use information effectively. They believed that electronic reporting and the HMIS help in maintaining quality data. They described various ways adopted to assess quality. Data clerks mainly collect, compile, analyse and validate data from different sources. The validation takes place after every shift and monthly and is usually done by two people to check the accuracy and consistency for control. They also use a lot of quality assurance sampling to select a sample of files to check the quality, using HIV/AIDS indicators. The general agreement among them was that lot quality assurance sampling ensures efficiency at different levels. However, they also indicated that quality is affected by the level of job satisfaction of the staff.

“Electronic HMIS supports effective communication and quality. What is required is sending updated information, which is very important” (FG1 Data Producers).

“The data will be checked according to the proportion of files by comparing the previous and current month [and if the] difference is huge or exaggerated, then indicators will be checked” (FG3 Data Producers).

“We work with monthly data that come from the registers. We review to see how many clients are on treatment. We compile data at the end of our shift, identify improper recording and sometimes we find incompatible data from the recordings” (FG1 Data users).

“There are things to be done to control data quality, like lot quality assurance sampling, by selecting 12 indicator data sets, we check data source documents such as tally and register for accuracy, that is, differences and similarities” (FG4 Data Producers).

Theme 2: Use of routine health information for HIV/AIDS monitoring

Ministry-level Data-driven Decisions

Participants acknowledged the usefulness of the HMIS in planning and mobilising resources at the sub-city health office, city health bureau and ministry of health levels. They described the data flow from facilities to the ministry level and recognised the significance of policy formulation based on data emanating from facilities. The Ministry of Health uses the information to manage healthcare programmes and the overall health systems. Most participants were aware of their important role in improving population health outcomes and viewed the HMIS as a system that enabled them to make a contribution.

They elaborated as follows:

“It is important to provide the ministry with useful data to enable them to set national targets and review the indicators. The city health bureau has its own targets in terms of resources allocation. We need to be mindful of different levels of information needs” (FG3 Data users).

“Management team looks at data and take(s) actions to improve the service, for example, counselling, training, and staff” (FG4 Data users).

Case management

Participants confirmed that data have proven very vital in managing HIV cases. It gives much information that helps with managing drug therapy. Case managers provide an essential service. Lost cases are being followed up using specific algorithms, testing is monitored, and plans are developed based on the results. The guidelines provide definitions regarding tracing, and clients are followed within prescribed days of missing treatment. The plans mostly involve the initiation of treatment and counselling.

“In (the) antiretroviral therapy department, we have a case manager, who will follow the linked cases every week. On (a) monthly basis, they will call the lost cases, find out reasons for not following the treatment” (FG3 Data users).

“The test and treat algorithm helps the adherence team to decrease defaults and lost cases. We are able to see if we are reaching our targets” (FG3 Data users).

“Patients are followed up based on the information received from other units. During reporting time, (an) HMIS team evaluates the facility performance in terms of targets. In January, HMIS was revised, the medical director or HMIS focal person usual(ly) does the assessments to identify gaps” (FG4 Data users).

Specific Indicators For The Hiv/aids Programme

Participants recognised the significance of indicators in information use and linked them to the programme’s performance. Specific departments have unique indicators, for example, prevention of mother-to-child transmission, ART, and voluntary counselling and testing. The annual plans are used as benchmarks to monitor the level of achievement by performance review teams. This allows them to self-correct and learn from each other.

“The indicators are number of pregnant women attending at least one ANC visit at a prevention of mother to child transmission site and acceptance of HIV testing, percentage of pregnant women testing positive for HIV, number of women testing positive who receive ART prophylaxis” (FG2 Data users).

“Indicators are useful to measure implementation level of the HIV/AIDS programme and to determine resources required, for us, we report to the ministry who take actions on supply of resources” (FG4 Data users).

Performance monitoring

Participants confirmed the existence of some monitoring processes in the units, such as monthly review meetings. The composition of teams was similar across facilities. Departmental heads participate in teams chaired by the medical directors. Major items for review are testing, initiation levels and adherence to ART. The HMIS reports form the basis for discussions using various forms, indicators, and logbooks. They all agreed that monitoring is a team effort that includes inputs from different departments.

“In (the) prevention of mother to child transmission department, there is (a) cohort monitoring chart system for checking (the) number of clients, defaulters, fatality rate, and children diagnosed and lost…. The reasons/causes are discussed in our monthly meetings” (FG2 Data users).

“In my hospital, we hold these review meetings to make sure that we use data from HMIS effectively and use annual plans for monitoring. The HMIS focal person makes sure that HMIS reports are done timely and completely for each department to check their performance. Finance and HR heads are also represented” (FG4 Data users).

Specialized Skills

All participants agreed that adequate knowledge and competence in the HMIS are critical. They indicated that one of the biggest challenges in healthcare is inadequate human resources, knowledge, and lack of specialized software skills, which greatly impacts the quality of information using HMIS protocols.

“We didn’t have a data manager for the past three years, who will do displays and analyse data for the facility. …there is high staff turnover due to retirement and separation. Every year, staff rotates and no one is fixed in one place” (FG2 Data producers).

“We have one HIT person for all departments in this facility and he has difficulty of supporting every room. 4–5 HIT persons are expected to be assigned. In the past, even one person was not assigned, which created problems in sending quality reports” (FG1 Data producers).

Culture Of Information Use

Data users from various units expressed different perceptions on the promotion of the culture of information use. They described processes that are being followed at the facility level as an indication of some level of promoting a culture of information use. However, the majority believed that much more could be done by the management as, currently, there are no incentives for information use. There are no guidelines, and training is inadequate for users to have a holistic understanding of the purpose of the HMIS.

“The information I get from HMIS staff is beneficial; we work together to make sure that information is used, it is not perfect, but there is a steady progress from where we were” (FG2 Data users).

“Data clerks are very helpful in providing information for (the) compilation of reports and tracking the HIV/AIDS program progress. However, it would be good to have them participate in our tracking activities as well” (FG3 Data users).

“Information use is at the start level and I cannot say we are fully utilising data for decision-making. We do support staff. Recent implementation of DHIS2 may assist in getting data quickly to use for monitoring the program” (FG4 Data users).

Gimbel et al. state that HMISs are planned to serve all users at each health system level with reliable information to support decisions [24]. The study identified the significance of an in-depth understanding of the HMIS for programme follow-up and information. Participants indicated that the design of the HIS was critical in the successful implementation of the HMIS. The assumption was that the HMIS understanding is a foundation for proper HMIS use and implementation through management inducing staff to embrace an innovative HMIS understanding [25].

Findings revealed that Ethiopia uses a hybrid model comprising paper-based and electronic reporting. There were various views. Some expressed that the use of paper-based reporting did not support timely reporting, and others indicated that the paper trail provided a reliable backup, especially during power failures that seemed to be frequent. The system seemed hampered by reporting formats, the load, and the tools used to capture data. The non-alignment between the software and paper-based generated data seemed to be an issue, especially in reporting certain data elements. The software did not accurately capture the level of performance. Routinely collected health service data must be reported timely to higher levels, facilitating evidence-based decision-making at all levels of the health pyramid, especially at the point of collection [26]. MEASURE Evaluation indicates that the continuous usage of paper-based systems results in poor data quality regarding availability, reliability, completeness, and reporting timeliness, affecting health service delivery [27]. Mozambique also reported the use of paper and electronic systems reporting formats, the incompatibility of reporting formats and a shortage of skilled professionals as the main challenges that could be improved through standardization of reporting [28].

The involvement of external agencies also seemed to be an issue. For example, the Ministry of Health [Ethiopia] had its mandates for the data management process, but external agencies also bring software incompatible with the local IT infrastructure. Some weaknesses were highlighted in the systems updates. More specifically, SmartCare needed to be updated to avoid data loss. Participants believed that capacity building should be effected every time there is an update or change in reporting formats. Participants perceived HMIS as very useful in generating data for managerial functions, including evaluating the performance of the HIV/AIDS programme at their level. Findings reveal that having a usable system can significantly help practitioners to implement the HMIS successfully, especially if they are knowledgeable about the disease classification system and indicators. In addition, this system needs to be compatible with the SmartCare software. HIS will face challenges in any country if fragmented and incompatible with other donor-funded systems. Umezuruike, Nwankwo and Kareyo confirm that a lack of standards for various software and hardware tools is a recipe for incompatibility among myriads of devices applicable to health information technology [29]. Therefore, management might need to review the current reporting formats and software. In developing countries, the HIS exists with many parallel central programmes, which leads to the disintegration of the HIS [30].

ICT and other related resources were recognized as critical for efficient and effective monitoring of a health programme such as HIV/AIDS. Timely, complete, and accurate information and a functional IT system are critical to producing quality data. Having reliable data on the health system’s performance supports effective strategies to implement and measure interventions [31]. Some functionalities, such as clients’ appointments, viral load and CD4 count, were missing, and manual reporting closed that gap. Braa et al. report that antiretroviral therapy reporting is complex and paper-based systems make it even more difficult to handle [32].

The purpose of HMIS is to generate useful quality data. Timely, accurate, error-free reporting and data uploading are considered the quality benchmarks of an ideal HMIS [33]. The significance of electronic reporting using systems such as DHIS2 was acknowledged. However, they also believed that user satisfaction needed to be considered. They used various mechanisms to validate data, including using LQAS. They also expressed various levels of confidence in conducting data quality checks. The study assumed that capacity building in data quality checks would be beneficial. Facilities need to have an adequate number of qualified employees and infrastructure to produce quality data [34]. This will support the analysis, interpretation, and use of information at the district and facility levels [35]. Cheburet and Odhiambo-Otieno argue that the LOAS has great potential for monitoring health programmes to improve data quality [26].

Most participants acknowledged the importance of guidelines concerning understanding definitions of indicators to support the use of information. Key indicators generate data, such as the number of individuals tested and counselled for HIV and who received their test results. In the prevention of mother-to-child transmission department, the focus seemed to be on the number of HIV-infected women on HIV care and using a modern family planning method in addition to all the indicators used at the antiretroviral therapy department. The interconnected roles between data producers and users supported using data to manage cases.

This study contends that when data users and producers collaborate, they become more attentive to the data collection processes, methods, data sources, and data quality. They can analyse and interpret data cooperatively to respond to programmatic questions [36]. The antiretroviral therapy department generates data on key indicators, such as patients currently on ARVs, differentiated by the type of regimen, age category, and sex. All these units used the HMIS to generate relevant data and plan activities accordingly. The HMIS team evaluated the performance according to targets.

Users require different information in varying details to track the HIV/AIDS programme performance. The performance review meetings reflected some commitment to data quality and developing a culture of information. Lippeveld and Hagan agree that there must be a plan at every health system level to create a culture of data-driven decisions [37]. Different units tracked their performance routinely through data generated by the HMIS. This was achieved by comparing their plans with data. Participants perceived these meetings as vital and their roles as significant for the continued use of the HMIS in monitoring the programme’s performance. Collaboration across various government agencies, specifically data management units, is essential to allow this sharing and use of data [38].

The review meetings provided a platform to collaborate and monitor the programme performance using various tools. Silas found the importance of monitoring performance and following up on information usage through performance review meetings [39]. Alshamari indicates that DHIS2 utilization is critical for properly implementing data management and facilitating the use of information in HIV/AIDS monitoring [40]. Findings show that HMIS users need specialized skills. Increasing the number of skilled health information management professionals in the HMIS strengthens the health system [41]. Different users will require different types of information in varying details to support decision-making. The study found that at the facility level, users needed information to manage issues related to the programme and resources. Lloyd, Collie, McInnes, King, Lollback and Garland found that the major challenge is to apportion resources for training and to advance the necessary skills regarding information in ideal formats [42].

The study revealed that the sustainability of the HMIS depends on all processes affecting organizational information culture. Harikumar agrees that advancing an information culture cultivates programme implementation [43].

Participants acknowledged some support in the HMIS task. However, they lamented their inadequacy. They would like incentives for information use and more adequate guidelines for users to understand the bigger set of HMIS fully. Wude et al. indicate that a perceived culture of information enhances routine health information utilisation [44]. Ojo and Popoola also confirm that management and leadership advocacy and an organization’s commitment to promoting a culture of information affects the success of the health information system [45].

Ethiopia uses a hybrid model of paper-based and electronic reporting. The use of paper-based reporting was deemed to be beneficial during power or connectivity failures. However, it appeared that paper-based reporting did not support the significant dimension of data quality: timeliness of reporting. For information to be effectively used in monitoring and evaluation, data must reach intended recipients on time. HMIS was well perceived and understood. The main challenge appeared to be the inadequacy of information technology infrastructure. The study found that DHIS2 helped provide information for planning, costing, researching and monitoring the performance of the HIV/AIDS programme. However, this system needed to be compatible with the existing SmartCare software. HIS would face challenges in any country if fragmented and incompatible with other donor-funded systems. The findings leaned towars showing the local experience of the HMIS as is itself an innovation for the organization implementing it. Such information reforms require well-structured change management.

The complexity of reporting formats, interoperability of software used, non-alignment with paper-based tools, the DHIS2, HIV/AIDS-specific indicators, ICT and other related resources were highlighted as critical barriers to the successful use of the HMIS. Participants believed these technical issues impacted the quality of data adversely and, subsequently, the conversion of that data to information and using it to monitor the programme’s performance. The actual impact on producing useful information still needs to be quantified. Developing countries, including Ethiopia, need to implement mechanisms to facilitate systems integration to achieve universal coverage and reach the Sustainable Development Goals, especially those related to providing infrastructure to support health systems. The importance of indicators to monitor the programme’s performance to show whether it is achieving its intended objectives is acknowledged. The common fact is the need for clear indicators with a simple disease classification system for improved data management and use in HIV/AIDS monitoring.

The study showed that the culture of health information was perceived as lacking. However, users did use the information for annual work plans and setting priorities. Therefore, the gaps identified need to be addressed through adequate training and capacity building, especially after HIS updates. In addition, users need to believe that the organization promotes the culture of information use and creates an enabling environment that encourages staff engagement. The study notes that performance monitoring teams played a role in ensuring the quality of HIV/AIDS data and utilization of HMIS data for decision-making. There was a clear interplay between technical and behavioural factors. Data showed that data analysis, inerpreation and presentation need technical skills and knowledge. Therefore, there is a need to design appropriate skills development programmes to ensure staff is capacitated adequately.

In conclusion, the study acknowledges that the HMIS is a management tool that can be used at any health system level. Its use is influenced by the information needs of each stakeholder group, in this case, data producers and users. The study concludes that the interpretation and meaning users had of the HMIS in Ethiopia influenced how they used it and, subsequently, how they perceived support from the organization. The importance of high-quality data in monitoring and evaluating the HIV/AIDS program is acknowledged.

AIDS

Acquired Immunodeficiency Syndrome

ART

Antiretroviral Therapy

DHIS2

District Health Information System2

Focus Group

FGD

Focus Group Discussion

HIS

Health Information System

HIV

Human Immunodeficiency Virus

HMIS

Health Management Information System

ICT

Information and Communication Technology

Information Technology

M&E

Monitoring and Evaluation

Acknowledgements

We acknowledge all participants, the health centre and hospital directors and translators for their valuable study support. We would also like to acknowledge Prof. Phil Botha’s contribution to editing the manuscript.

Authors’ contributions

BED conceived and designed the study. BED conducted fieldwork, collected data, analysed data, and wrote the manuscript. MMR designed the study, advised and supervised data analysis and reviewed the manuscript. BED and MMR read and approved the final manuscript.

Funding

This research received a specific grant from the University of South Africa for data collection and analysis.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available but on reasonable request from the corresponding author.

Ethics approval and consent to participate

This study was approved by the Institutional Review Boards of the Departmental Higher Degrees Committee of the Department of Health Studies University of South Africa Ethical Clearance Committee for Research on Human Subjects (HSHDC/ 792/2017) and Addis Ababa City Administration Health Bureau (A/A/H/B/574/227). Also, a support letter was written by the University of South Africa Addis Ababa Regional Office to the Addis Ababa City Administration Health Bureau. Support letters from the above organizations were given to the respective hospital and health centre directors. Written Informed consent was also obtained from each participant, and confidentiality was guaranteed. All the methods were performed in accordance with the relevant guideline and regulation.

Consent for publication

Not applicable.

Competing interests

The authors declare that we have no competing interests.

Ministry of Health [Ethiopia]. Overview of the health sector. 2013. http://www.moh.gov.et/overviewsector. Accessed 25 Jun 2018.
World Health Organization. Monitoring and evaluation of health system strengthening. 2010. http://www.who.int/healthinfo/HSS_MandE_framework_Nov_2009.pdf. Accessed 22 May 2017.
World Health Organization. World health statistics: Monitoring health for the SDGs. 2016. https://www.who.int/docs/default-source/gho-documents/world-health-statistic-reports/world-heatlth-statistics-2016.pdf. Accessed 22 Jun 2021.
MEASURE Evaluation. A Case Study to Measure National HIV Monitoring and Evaluation System Strengthening. Côte d’Ivoire. 2014. https://www.measureevaluation.org/resources/publications/sr-14-102. Accessed 18 Oct 2018.
Gloyd S, Wagenaar BH, Woelk GB, Kalibala S. Opportunities and challenges in conducting secondary analysis of HIV programmes using data from routine health information systems and personal health information. Journal of the International AIDS Society. 2016; 19(5) Suppl 4:20847. https://onlinelibrary.wiley.com/doi/full/10.7448/IAS.19.5.20847.
Qazi MS, Ali M. Health Management Information System utilization in Pakistan: Challenges, pitfalls and the way forward. Bioscience Trends. 2011; 5(6):245–254. https://www.ncbi.nlm.nih.gov/pubmed/22281538. Accessed 21 Feb 2018.
Joint United Nations Programme on HIV/AIDS. Guidance on Capacity Building for HIV Monitoring and Evaluation. Switzerland, Geneva: UNAIDS; 2014.
Mgbere O, Khuwaja S, Vigil KJ, Patel SM, Wang J, Essien EJ. Sources of clinical information used in HIV care and treatment: Are providers’ choices related to their demographic and practice characteristics? Health Informatics Journal. 2018; 25(4):1572–1587. https://journals.sagepub.com/doi/10.1177/1460458218788906.
World Health Organization. a. Health facility and community data toolkit. Geneva, Switzerland: WHO. 2014. http://www.who.int/healthinfo/facilityinformationsystems/FacilityCommunityDataToolkitfinal.pdf. Accessed 27 Apr 2017.
Koumamba AP, Bisvigou UJ, Ngoungou EB, Diallo G. Health information systems in developing countries: Case of African countries. BMC Med Inform Decis Mak. 2021; 21:232. https://doi.org/10.1186/s12911-021-01597-5.
Seitio-Kgokgwe O, Gauld RD, Hill PC, Barnett P. Development of the National Health Information Systems in Botswana: Pitfalls, prospects and lessons. Online Journal of Public Health Informatics. 2015; 7(2). doi:10.5210/ojphi.v7i2.5630.
Gathara DN. The use of routine case record data to evaluate quality of inpatient hospital care for newborns and children in Kenya. PhD thesis, University of Amsterdam. 2015. http://dare.uva.nl/personal/pure/en/publications/the-use-of-routine-case-record-data-to-evaluate-quality-of-inpatient-hospital-care-for-newborns-and-children-in-kenya(02bd5816-9b6a-4522-b894-e9b12b5ee58f).html. Accessed 11 Dec 2021.
Tashobya CK, Da Silveira VC, Ssengooba F, Nabyonga-Orem J, Macq J, Criel B. Health systems performance assessment in low-income countries: learning from international experiences. Globalization and Health. 2014; 10(5). https://globalizationandhealth.biomedcentral.com/articles/10.1186/1744-8603-10-5. Accessed 22 Jun 2018.
Endriyas M, Alano A, Mekonnen E, Ayele S, Kelaye T, Shiferaw M, Misganaw T, Samuel T, Hailemariam T, Hailu S. Understanding performance data: health management information system data accuracy in Southern Nations, Nationalities and People’s Region, Ethiopia. BMC Health Services Research. 2019; 19:175. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-019-3991-7. Accessed 09 Mar 2020.
Aqil A, Lippeveld T, Hozumi D. PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan. 2009 May; 24(3):217–28.
MEASURE Evaluation. Performance of Routine Information System Management (PRISM) Toolkit: PRISM Tools. Chapel Hill, NC, USA: MEASURE Evaluation, University of North Carolina. 2019. https://www.measureevaluation.org/resources/publications/tl-18-12/at_download/document. Accessed 10 Oct 2021.
Yin RK. Validity and generalization in future case study evaluations. Evaluation. 2013; 19(3):321–332. https://journals.sagepub.com/doi/abs/10.1177/1356389013497081.
Yazan B. Three Approaches to Case Study Methods in Education: Yin, Merriam, and Stake. The Qualitative Report. 2015; 20,134–152. https://nsuworks.nova.edu/tqr/vol20/iss2/12/. Accessed 14 Nov 2021.
Patton MQ. Qualitative Research and Evaluation Methods: Integrating theory and practice. 4th ed. Thousand Oaks, CA: SAGE; 2015.
Tsang S, Royse CF, Terkawi AS. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi Journal of Anaesthesia. 2017; 11 Suppl 1:S80–S89. https://doi.org/10.4103/sja.SJA_203_17.
Streubert HJ, Carpenter DR. Qualitative Research in Nursing Advancing the Humanistic Imperative. Wolters Kluwer, Philadelphia. 2011.
Nyumba T, Wilson K, Derrick CJ, Mukherjee N. The use of focus group discussion methodology: Insights from two decades of application in conservation. Methods in Ecology and Evolution. 2018;9(1):20–32. https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.12860.
Lincoln YS, Guba EG. Naturalistic inquiry. Beverly Hills: Sage; 1985.
Gimbel S, Mwanza M, Nisingizwe MP, Michel C, Hirschhorn L, the AHI PHIT Partnership Collaborative. Improving data quality across 3 sub-Saharan African countries using the Consolidated Framework for Implementation Research (CFIR): Results from the African Health Initiative. BMC Health Ser Res. 2017; 17:828. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-017-2660-y. Accessed 14 Aug 2018.
Mohamadali NA, Aziz NFA. The technology factors as barriers for sustainable Health Information Systems (HIS) – A Review. Procedia Computer Science. 2017;124:370–378. https://www.sciencedirect.com/science/article/pii/S1877050917329356. Accessed 15 Oct 2018.
Cheburet SK, Odhiambo-Otieno GW. Process factors influencing data quality of routine health management information system: Case of Uasin Gishu County Referral Hospital, Kenya. International Research Journal of Public and Environmental Health. 2016;3(6)132–139. http://journalissues.org/wp-content/uploads/2016/06/Cheburet-and-Odhiambo-Otieno.pdf. Accessed 30 Nov 2019.
MEASURE Evaluation. a. A Case Study to Measure National HIV Monitoring and Evaluation System Strengthening Côte d’Ivoire. 2014. https://www.measureevaluation.org/resources/publications/sr-14-102. Accessed 18 Oct 2018.
Hochgesang M, Zamudio-Haas S, Moran L, Nhampossa L, Packel L, Leslie H, Richards J, Shade SB. Scaling-up health information systems to improve HIV treatment: An assessment of initial patient monitoring systems in Mozambique. International Journal of Medical Informatics. 2016; 97:322–330. https://www.sciencedirect.com/science/article/abs/pii/S1386505616302519. Accessed 22 Jul 2019.
Umezuruike C, Nwankwo W, Kareyo M. Implementation challenges of health management information systems in Uganda: A Review. Journal of Multidisciplinary Engineering Science and Technology. 2017; 4(7):7726–7731. http://www.jmest.org/wp-content/uploads/JMESTN42352312.pdf. Accessed 12 Mar 2020.
Kumar M, Mostafa J, Ramaswamy R. Federated health information architecture: Enabling healthcare providers and policymakers to use data for decision-making. Health Information Management. 2017; 47(2):85–93. https://journals.sagepub.com/doi/10.1177/1833358317709704.
Kebede M, Adeba E, Chego M. Evaluation of quality and use of health management information system in primary health care units of east Wollega zone, Oromia regional state, Ethiopia. BMC Med Inform Decis Mak. 2020; 20(1):1–12. doi: 10.1186/s12911-020-01148-4.
Braa J, Hanseth O, Heywood A, Mohammed W, Shaw V. Developing Health Information Systems in Developing Countries: The Flexible Standards Strategy. MIS Quarterly. 2007; 31(2):381–402. https://www.jstor.org/stable/25148796?seq=1&cid=pdf-reference. Accessed 19 Oct 2018.
Samal J, Dehury RK. Perspectives and challenges of HMIS officials in the implementation of Health Management Information System (HMIS) with reference to maternal health services in Assam. Journal of Clinical and Diagnostic Research. 2016; 10(6):7–10. https://www.jcdr.net/article_fulltext.asp?id=7955. Accessed 14 Mar 2019.
Moore M, Anthony CR, Lim Y-W, Jones SS, Overton A, Yoong JK. The future of health care in the Kurdistan Region — Iraq: Toward an effective, high-quality system with an emphasis on primary care. Rand Health Quarterly. 2014; 4(2):1. https://www.jstor.org/stable/10.7249/j.ctt6wq8bz.15. Accessed 19 Oct 2018.
Nicol E, Bradshaw D, Uwimana-Nicol J, Dudley L. Perceptions about data-informed decisions: An assessment of information-use in high HIV-prevalence settings in South Africa. BMC Health Services Research. 2017; 17 (2). https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-017-2641-1.
Nutley T, Reynolds HW. Improving the use of health data for health system strengthening. Global Health Action 6. 2013. https://www.semanticscholar.org/paper/Improving-the-use-of-health-data-for-health-system-Nutley-Reynolds/82aeb38960ec49479fdd0ca0e11628310fbe4822. Accessed 18 Nov 2021.
Lippeveld T, Hagan O. Routine Health Facility and Community Information Systems: Creating an information use culture. Global Health: Science and Practice. 2017; 5(3):338–340.
Hosseinpoor AR, Nambiar D, Tawilah J, Schlotheuber A, Briot B, Bateman M, Davey T, Kusumawardani N et al. Capacity building for health inequality monitoring in Indonesia: enhancing the equity orientation of country health information systems. Global Health Action. 2018; 11(22):7–12. https://www.tandfonline.com/doi/full/10.1080/16549716.2017.1419739.
Silas NK. Factors influencing the performance of routine health information system: the case of Garissa subcounty, Kenya. Thesis, Master of Arts in Project Planning and Management, University of Nairobi, Kenya. 2017.
Alshamari M. Usability Factors Assessment in Health Information System. Intelligent Information Management. 2016;8(6):170–180. https://www.scirp.org/journal/paperinformation.aspx?paperid=72201. Accessed 23 Jun 2020.
Makinde OA, Mami MI, Oweghoro BM, Oyediran KA, Mullen SM. Investing in health information management: The right people, in the right place, at the right time. Health Information Management Journal. 2016; 45(2):90–96. https://journals.sagepub.com/doi:10.1177/1833358316639447.
Lloyd S, Collie J, McInnes A, King K, Lollback A, Garland A. Smart use of data, information and communication: The INFORM-ed Best Local Practice Project – Grafton Base Hospital. Health Information Management. 2011; 40(3):26–30. https://journals.sagepub.com/doi/10.1177/183335831104000304?icid=int.sj-abstract.similar-articles.3.
Harikumar S. Evaluation of Health Management Information Systems - A study of HMIS in Kerala. MPH dissertation, Achutha Menon Centre for Health Science Studies, Sree Chitra Tirunal Institute for Medical Sciences and Technology, Thiruvananthapuram, Kerala. 2012. https://www.semanticscholar.org/paper/Evaluation-of-Health-Management-Information-Systems-Harikumar/6c29bbd2fba15668daa0617aa957fd7d161f08c1. Accessed 16 Nov 2020.
Wude H, Woldie M, Melese D, Lolaso T, Balcha B. Utilization of routine health information and associated factors among health workers in Hadiya Zone, Southern Ethiopia. PLoS ONE. 2020; 15(5):1–11. doi: 10.1371/journal.pone.0233092.
Ojo AI, Popoola SO. Some correlates of electronic health information management system success in Nigerian teaching hospitals. Biomedical Informatics Insights. 2015; 7:1–9. https://journals.sagepub.com/doi/full/10.4137/BII.S20229.

No competing interests reported.

Download PDF

Journal Publication

published 26 Apr, 2024

Read the published version in South African Journal of Information Management →

Version 1

posted

You are reading this latest preprint version

The Health Information Management System and HIV/Aids Monitoring: Insights From Ethiopia

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Result

Conclusion

Background

Methods

Study setting

Design

Data Collection Instruments/tools

Data Collection Procedure

Data analysis

Ethical Consideration

Trustworthiness

Results

Biographical Data Of Participants

Theme 1: Organization And Structure Of Hmis In Ethiopia

Use of the HMIS at the facility level

Electronic and paper-based data reporting

Data-capturing system

Challenges with reporting

Management of updates

Usability of the system

It Infrastructure And System

Data Quality Assurance

Theme 2: Use of routine health information for HIV/AIDS monitoring

Ministry-level Data-driven Decisions

Case management

Specific Indicators For The Hiv/aids Programme

Performance monitoring

Specialized Skills

Culture Of Information Use

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1