Challenges and lessons learned
The main challenges faced by the implementation teams centered on access to accurate and timely data, as well as region-specific challenges. It was not possible to systematically obtain daily hospital data that presented cases by severity to compare against model predictions. This pilot demonstrated the importance of having such data disaggregated by mild, moderate, and severe patients, which is now being collected by COVID-19 teams. Due to the COVID-19, emergency staff at hospitals were not consistently available to give the in-country team updates. It was also challenging to obtain the number of staff assigned specifically to COVID-19 patients as the situation on the ground was fluid. Therefore, estimates needed to be made based on the total number of beds allocated and use a ratio of staff to estimate the number of health workers by cadre assigned to COVID-19 patients. On several days each week the team adapted the total number of cases nationally and the ratio of patients by region to estimate the new cases per day until actual data from the regional teams at each of the hospitals could be obtained. In Kenya’s devolved health system, it was necessary to work with the county officials equally with the central MOH, since each level of government is independent. Conversely, in Mali, the central MOH led the information sharing at the regional level, which was later expanded to all regions and overseen by the national team.
Data collection challenges
Challenges with data collection included obtaining accurate staffing data, case data at the city and hospital level, and data on case distribution across COVID-19 severity categories and updating data as the situation changed. These challenges occurred for several reasons. First, facilities were responding to the pandemic, which left little free time to devote to data collection. Second, the process of obtaining national approval to gather local data took several days or weeks, which caused delays in mounting a rapid response. Third, some regions also relied on traditional paper charting, which made it difficult to translate data to an online database. Fourth, since the implementation was done remotely, the staff for our in-country team in Nairobi had to rely on very busy county and hospital staff to give daily updates on the number of cases and changes in mitigating factors. This experience was similar in Mali.
In future phased implementation plans, it is recommended that countries investigate what types of clinical, bed, and HRH data are available before initiating the activity so that they can be prepared with a solution for gathering the required data.
Staffing data
In both countries, it was possible to obtain the total number of health workers by cadre per hospital from the HRIS, which use the iHRIS software supported by IntraHealth. In Mali the number of staff dedicated to COVID-19 per facility by cadre was also available, but it was not possible to obtain the distribution of staff by severity of COVID cases. In Kenya, the number of staff assigned to COVID-19 cases was unavailable at the beginning of the study. Initially a simple ratio was applied to split the staff between COVID-19 and non-COVID-19 assignments, and further assumptions were made to divide the staff by severity level using data from the MOH detailing the percent of mild, moderate, severe, and critical cases.
However, in both countries it was challenging to collect data regarding how many staff were available by level of COVID-19 severity since moderate and severe patients were admitted on the same wards and cared for by the same health workers. This meant that the activity standards for the same professional activities required different amounts of time—for example, in Mali a severe patient required three hours of nursing care in 24 hours whereas a moderate patient required .45 hours in 24 hours.
Challenges with staffing data were further complicated by the confidential nature of health data, particularly involving health workers who had been infected with COVID-19.
Case data
Challenges in collecting case data were most evident when trying to obtain local data per facility. Case data were available at the national level through daily situation reports; however, local facilities either were unable to prioritize case reporting by severity level, or the information was unavailable due to privacy concerns.
A further issue in obtaining case data related to the challenges presented by undertesting the population. The teams in Mali and Kenya both described that many people in the general population associated testing for COVID-19 with stigma that led many people to stay home rather than present for testing. In Kenya, all travelers into the country and suspected local cases were initially required to pay for mandatory quarantine in designated COVID-19 isolation facilities. Treatment for COVID-19 was free in public facilities while at a cost in private facilities. These approaches inadvertently incentivized many people to hide their symptoms or stay home rather than present for testing. The team in Mali also indicated that many people in the general public would only present for testing as a last resort if they were ill and if their symptoms were not improved by other treatments at home.
Challenges with updating data
A challenge noted throughout the process was the need to continuously update the tools as the situation on the ground changed rapidly throughout the progression of the pandemic. It was necessary to review assumptions made on a regular basis and revise the tools as needed to reflect the current reality. This challenge was mitigated through frequent communication between both implementing teams to ensure that the modeled data were realistic. For example, a key driver of any infectious disease is how many susceptible contacts each infected individual has an opportunity to spread the infection to. As this was unknown at the start of the pandemic, the Mali team estimated that each individual in densely populated Bamako may be in contact with 20 persons in one day. That parameter was then updated using the average number of contacts traced for each positive case, which was included in the government’s daily situation report, to a more realistic 7 and later to 3.5 when new mitigation and suppression measures were enacted. This parameter change was validated by comparing the actual cases reported with the predicted cases from the Adaptt model. In Kenya, closer collaboration with WHO at country level was key in ensuring entry to the national COVID-19 emergency command center to get daily reports/updates.
Region-specific challenges
Other challenges noted during the phased implementation plans are attributable to regional variations and contexts.
Kenya
The structure of Kenya’s devolved health sector can create delays in mobilizing a national response. At the beginning of the phased implementation plan, the MOH had several competing priorities and was unable to be as involved. By the later phase of our activity, WHO/AFRO had developed its own African version of the Adaptt model to use with the existing HWFE tool, resulting in ongoing discussion between the MOH and WHO/AFRO on the best tool to use going forward.
Mali
A challenge noted in the Mali context related to language differences. The Adaptt tool has a language feature where graphics can be presented in several languages; however, the HWFE is available only in English. The language difference presented a barrier when translating the term for a mechanical ventilator. This initially led to a misunderstanding where the in-country team was aggregating data for invasive mechanical ventilation, positive pressure ventilation, and all types of oxygen delivery devices, while the remote team was under the assumption that the reported data were for invasive mechanical ventilation only. This misunderstanding was easily resolved by both implementing teams clearly defining their terms, and a lesson learned is to be mindful of language differences when implementing the tools across languages and regions.
Successes and strengths
A key success of this activity is that the Adaptt and HWFE tools, originally created for European countries, could be repurposed to fit an African context. The tools were well received by the MOH and WHO representatives in Kenya and Mali, and generated realistic data that were used to inform health policy decisions at the regional and national levels. The Kenya and Mali implementing teams focused on adjusting the tools for local resources and context for what was rational and reasonable in their countries, including successfully translating workforce needs for health worker categories and professional activities available in-country.
A strength in the approach used in both countries was being able to use proxy data when the actual daily data requirements were not available for number of new COVID-19 cases. Using estimates from the national COVID-19 response teams for the percentages of patients by level of severity as well as the percentage of cases by region permitted ongoing analysis of the epidemic until actual data from the facilities and regions were available. Similarly, knowing the actual denominator of staff or number of beds and the percentage of staff working with COVID-19 patients or percentage of beds allocated to COVID-19 patients permitted calculation of staff and beds allocated for direct COVID-19 care. It is important for the local team populating these two tools to be in close contact with the national COVID-19 response team to have access to daily data and policy changes being made to the COVID-19 program.
A strength to implementing the tools in Kenya and Mali was access to iHRIS data for the denominator of health workers, which made human resources data collection for COVID-19 easier to obtain. Kenya also benefitted from having completed a WISN assessment in 2013, which facilitated quicker access to workforce activity data for specific cadres and the local understanding of professional activities and activity standards. Even though the MOH in Mali had not completed a WISN, the implementing team was still able to understand the concepts of professional activities and activity standards using the European professional activities and times to obtain the necessary data and incorporate it with assistance from a remote clinical expert.
Another key component in the success of the phased implementation plan was the mirrored roles of the remote and in-country implementing teams, which provided an organized structure to the process and facilitated the appropriate expertise and sharing from each team member. The in-country teams are now equipped to carry the results forward and train others on the use of the tools and interpretation of the results.
It was also noted that obtaining early support from the MOH and involving the country-level WHO and USAID offices and other key stakeholders at the onset of the pilot led to quicker implementation, organized collaboration, and a streamlined process for scaling the results to the rest of the country.