The literature on evidence-based humanitarian intervention argues that humanitarian response and programming decisions should take into account data and information derived from reliable assessments of needs, and evaluations of intervention effectiveness [14,15]. However, there is also pragmatic recognition that experience-based response and expert judgment is ubiquitous in humanitarian settings, given the challenges of timely and reliable data collection and analysis in crisis contexts to generate adequate evidence [16]. Combining these approaches, Darcy et al. “understand a well-informed decision to be one that takes due account of data and information most relevant to the crisis context, and combines this with experience-based knowledge to determine what intervention is the most appropriate in that context” [17]. Yet, despite recognising that humanitarian response should be evidence-based, it is widely lamented that many interventions are not [15,18–20].
Key factors for implementing evidence-based response
Our study suggests that determining whether humanitarian interventions and adaptations are evidence-based, experienced-based, or grounded in a mix of information and intuition, may be more complex than previously assumed. Our analysis of humanitarian organisation’s field experiences when responding and adapting to COVID-19 identified four factors relevant to the use of evidence-based guidance, and the nature of that usage: availability and access; coherence and coordination; contextual relevance; trust and credibility. We argue these factors may have an important influence on the extent to which humanitarian response is or can be based on evidence, particularly in dynamic and uncertain humanitarian and fragile settings affected by the COVID-19 pandemic.
Availability and access
As depicted in Table 3, global guidance documents were available on a wide range of framework areas, and our interviews only identified a few specific gaps in terms of topic coverage. For example, one organisation lacked guidance for programming with mobile populations that was specific enough to the complexities of migration journeys in COVID-19 affected settings. Another found that guidance on the modalities and timing for resuming activities and transport post-lockdown was missing. Initially, our guidance document review found some framework areas to be under-documented, such as ‘sexual and reproductive health’, ‘human resources’ and ‘coordination’. However, on closer analysis, these topics were often included within documents categorised under other framework areas. For instance, guidance on adapting existing food security and livelihoods interventions to COVID-19 often includes a section on the safety of staff during food distribution activities, and a section on how to coordinate with other stakeholders. Guidance on maternal, newborn and child health often includes a section on sexual and reproductive health. Thus, guidance on most framework areas was available, even if some topics were more difficult to access by being bound up among broader topics. Clearly, availability of evidence-based guidance is a key first step in order for humanitarian organisations to base their implementations on it. Alone, however, availability is insufficient: access is also crucial.
We identified the importance of access from our analysis of available guidance by organisation. Although we found guidance was available from a wide range of organisations (see Table 1), we identified some limitations in the way large INGOs disseminate guidance, which may create access challenges for less resourced organisations. During the initial review period (March to May 2020), we could not identify guidance documents from major INGOs including MSF, Save the Children, and BRAC, as they did not publish them openly on their organisational websites nor on general humanitarian resource websites (such as Reliefweb or humanitarian cluster websites). We later collected guidance documents from those organisations directly after interviews with their staff or following a more detailed investigation and personal follow-up. We learned that although these organisations had developed various COVID-19 guidance, they had initially kept these documents internal.
We find this practice problematic: by making their guidance more readily available, large INGOs could facilitate more evidence-based programming for other organisations. INGOs specialised in humanitarian response in particular are well placed to disseminate the types of guidance recommended for humanitarian and fragile settings, such as context-specific frameworks [21] and resource-stratified guidelines [22]. They also play an important role in disseminating global standards to local levels, as was evident in the way that many larger INGOs conducted remote in-house trainings based on guidance derived from global WHO standards and adapted to context. Extending such trainings to a wider range of local partners could substantially improve availability and access to evidence-based guidance. For sensitive or private operations, approaches that allow for guidance to be segmented while allowing the main content to be disseminated publicly should be explored. Mechanisms to systematically disseminate new and revised guidance to ground level should also be improved to allow for more fluid and effective adaptation and response.
Coherence and coordination
While making additional guidance available and accessible can be fruitful, striking a coherent balance of quantity and quality, breadth and specificity, and the format of dissemination, is also important. On the one hand, our interviews revealed cases where the guidance available was too general or designed for a broad public audience, rather than tailored to the health and humanitarian professionals who actually implement humanitarian programmes. On the other hand, some organisations found FAQs, information sheets, key messages and other communications materials (as opposed to detailed technical guidance documents) helpful when creating localised response plans and procedures.
These contrasting experiences suggest the utility of technical and non-technical information varies by actor and context. Indeed, our results showed many cases of organisations collating global guidance with local, national, regional materials, as well as mixing guidance of various formats together. Such practices illustrate the importance of a having wide range of guidance messages and formats available, from both global and local levels, to ensure the uptake of evidence in humanitarian and health responses. However, they also place an onus on guidance setters to better distinguish between public information campaigns, and technical guidance for health and other humanitarian programmes, to ensure coherence between these different types of documents and formats. For example, public information tailored to combat the infodemic may not be optimal for sensitising and educating humanitarian and health specialists, who require detailed information tailored to their purposes. Conversely, highly technical procedures may not be fit for purpose in all fragile operational settings, as Hussain & Sen found in their analysis of overly bureaucratic guidance issued by the European Union for managing COVID-19 in Syria [23].
In addition, multiple interview respondents mentioned challenging discrepancies between different COVID-19 guidance documents, for example, between those disseminated by global (UN) organisations, other humanitarian organisations, and national (MoH) institutions. This variation made it difficult for humanitarian organisations to decide which guidance to follow. In CAR, MSF described efforts to align guidelines with local partner organisations as a fragmented, decentralised process. In other cases, the rapidly emerging and evolving evidence base meant that guidance was also rapidly changing, which posed challenges not only in the design and implementation of interventions, but also in gaining community trust and maintaining credibility. This finding suggests continued need for improved information management and coordination, at all levels of humanitarian response, which has long been recognised [24]. The flurry of information service efforts that emerged as large workforces rapidly found themselves desk-bound, while well intentioned, also generated duplication and confusion. Our own COVID-19 Humanitarian platform project experienced this, when we found an individual website had created similar online compilation of guidance documents, prompting us to reach out and coordinate efforts. Thus, in addition to coherence, there is an important role for coordination to ensure guidance is updated, aligned, contextually grounded and responsive to the evolving situation.
Contextual relevance
Our study also found it is important for guidance to be appropriate and sensitive to context, in terms of language, information requirements, and technological realities. Previous recommendations have stressed that guidance on how to respond to humanitarian crises should consider data demonstrating priority needs, efficacy and cost-effectiveness in each context [18]. The fact that most humanitarian organisations we interviewed had to adapt guidance to context suggests humanitarian guidance setters are yet to meet these recommendations.
Multiple organisations explained how global guidance needed to be translated and adapted to their local language, which took time and often required expert consultation. Country-level coordination clusters, such as the WASH cluster, developed national guidance notes that are publicly available. However, the scarcity of global guidance directly written in a variety of languages and local dialects has been found to limit access to information in COVID-19 and other crises [25–28]. The need for translation applies not only to new COVID-19 specific documents, but also to evidence reviews from past epidemics such as Cholera and Ebola, which remain published mainly in English. As many countries currently facing humanitarian crises are French, Spanish or Arabic-speaking countries, the lack of evidence in these languages is particularly problematic. The Africa CDC systematically translates its guidance into French, Arabic, and Portuguese, which is good practice more organisations and institutions could emulate. However, even with major languages covered, the need for translation to the hundreds of languages used at the operational level requires more innovative solutions, including capitalising on automated translation and transcription technologies and natural language processing tools.
We also found that the local data and information landscape has an important relationship with the appropriateness of guidance. REACH encountered multiple challenges to design a severity index that was sensitive to local complexities and flexible enough to handle dynamic and uncertain data generation processes. For instance, they had to refine the general list of indicators for severity to a more limited set when they examined the (lack of) locally available data at the level of the ‘zone de santé’, or health zone. Moreover, they had to retain enough room in the model to add indicators later as more data would become available, and weight for differences in the quality of data across indicators and zones. They also had to take into consideration the secondary impacts of COVID-19, such as on food markets and employment, which were as relevant to severity as the direct impacts on health systems and outcomes. Thus, while methods to quantify and compare severity at the global level continue to advance, it is important that they remain cognisant that many globally conceived models may not generalise to complex local humanitarian settings.
Matching recommended COVID-19 responses as outlined in guidance documents with contextually appropriate technology and resources was another area where organisations faced challenges, and innovated appropriate solutions. For example, guidance on social media messaging for risk communication that assumes internet connectivity or personal device access may have little relevance for contexts where blanket radio or megaphone broadcasting remain prominent. In multiple contexts studied, including Uganda and South Sudan, megaphone broadcasting was found to be an effective risk communication modality. In Nepal, a key enabling factor for a local organisation who adapted their education programmes to an online format was the access of one village elder to a smart phone and data package, which students then borrowed to watch educational videos. In Libya, one organisation placed QR codes in the market with links to a smartphone survey, so they could monitor prices without having enumerators physically present. These examples point to wide variation in technology levels by context, which have implications for guidance setting and dissemination, and correspondingly the uptake of this evidence for response.
Trust and credibility
Finally, trust played an important role in relation to guidance use, particularly in terms of effectively implementing evidence-based response strategies. Previous studies have found successful implementation of guidance protocols requires securing the trust and confidence from both the target community and the regulatory authorities. A rapid evidence review of community engagement for infection prevention and control during previous epidemics, for example, identified trust as a key success factor [29].
Trust in humanitarian organisations in the COVID-19 context has been further complicated in two directions. Widespread rumours and misinformation have engendered competing narratives and alternative solutions to COVID-19, which erode community trust in humanitarian organisations trying to implement guidance-based responses. In addition, humanitarian organisations also faced mistrust from host governments and local authorities, who often needed to be convinced before guidance could be disseminated and programmes implemented. In refugee camps on Lesbos, Greece, MSF described that guidance to isolate vulnerable persons from the broader camp population was not an approach supported by local authorities, hence alternative approaches had to be discussed before evidence-based programmes could be implemented.
Humanitarian organisations reported using various strategies to gain the trust needed to implement their responses and adaptations. For example, communication with local authorities including the police force was helpful in many contexts. Partnerships, in particular, collaboration with the MoH, UN organisations, other INGOs and NGOs, were also integral for organisations to build space and confidence to introduce new or adapt existing interventions. Engagement with the national MoH was particularly important for a wide range of tasks: gaining permission to access sensitive areas, using national hotlines, surveillance and reporting, securing additional supplies, receiving weekly updates, building strategies and action plans, or deploying staff to/from the government response. At times, organisations reporting good relationships with the local health authorities also experienced success in gaining trust of the local population, though this may be contingent on the community’s own relationship with authorities. In general, whether trust of the community, government, or both were required in a given context, successful implementation of guidance protocols was heavily dependent on securing the requisite trust.
Implications for evidence-based humanitarian response
Guidance, when considered broadly played an important role in the design and implementation of responses and adaptations to COVID-19. The wide range of terms organisations use to refer to guidance show that they are concerned with applying various standards and protocols, or at least comparing their innovated responses with existing benchmarks and documents. This suggests that evidence-based humanitarian response may be more prevalent than previously expected.
At the same time, taking a wide view of guidance identifies a diverse set of challenges associated with designing, disseminating and implementing evidence-based humanitarian response. Humanitarian organisations’ tendency to compile and adapt various guidance to steer their COVID-19 responses in a way that was suitable for their contexts indicates that global guidance is not entirely relevant to address programmatic challenges at local level. Unfortunately, organisations at the local level often lack the time or resources to develop their own guidance de novo. Moreover, the exact mix of guidance and innovation driven response varies widely depending on the organisation and context in question. Thus, to be relevant and applied, guidance should take into consideration the ground-level context, resources and language, and be responsive to actual patterns in the use of already available guidance. Considering availability of and access to guidance, coherence and coordination, contextual relevance, and trust and credibility may help foster the implementation of more evidence-based response.
As both the global pandemic and humanitarian contexts are highly dynamic, it is also important that these factors are considered in a dynamic and responsive way. The COVID-19 Humanitarian platform provides a potential model for a more circular, dynamic and responsive evidence-based guidance and implementation process. By combining guidance documents with detailed experiences and anecdotes from the field, documented via qualitative case studies, surveys, webinars, and discussion boards, the platform offers users the chance to consult what is recommended alongside what is actually happening on the ground. In particular, the precise description of the modalities and rational for innovating, adjusting and implementing programmes can be extracted in real-time to provide a resource for global standard setters to consider when developing guidance, particularly in emergency humanitarian response and fast-paced epidemics.
Such a feedback loop could complement the prevailing guidance development model, which relies on updating existing guidance from previous epidemics, and which may not necessarily be relevant to the specific contexts and challenges of contemporary crises. It may also help to counter information asymmetries, such as those experienced in the early phases of COVID-19: transmission patterns from East to West/North before reaching the South led to a proliferation guidance designed for advanced economies, with limited relevance for low-income and humanitarian crisis settings. Incorporating documented qualitative experiences from these settings could therefore help to generate genuinely global guidance. Other studies have found feedback loops are important for circular guidance use, generation, and adaptation [30,31], while academic-humanitarian partnerships [32] have also been useful in such endeavours. While the COVID-19 Humanitarian Platform has room to improve, it could serve as basis for more dynamic and responsive evidence implementation in future.
Limitations
To our knowledge, this is the first study investigating the use of COVID-19 guidance in humanitarian settings. It covers a wide range of humanitarian settings and organisations, and the in-depth interviews reveal details on multiple aspects of response and adaptation to COVID-19, including guidance and evidence use. However, few interviews focused explicitly or solely on the use of guidance. Rather, in explaining the detailed modalities and rationales behind their interventions and adaptations, organisations made natural references to various global and national level guidance documents, as well as internal guidelines, protocols or procedures. A strength of this approach is that we avoided priming respondents to mention guidance which would otherwise have been omitted. A weakness is that we may have missed further different ways in which organisations used and referred to guidance.
Another limitation of the study design is the risk of selection bias among the organisations interviewed. Although we attempted to ensure diversity of interview participants by tracking location and type of organisation, humanitarian organisations may have self-selected into the study, for example by completing the online form to promote their interventions or because they had sufficient time or resources to complete the form. In addition, the use of a convenience approach to sampling and snowballing participants from our own connections meant the coverage of contexts likely favours locations and contexts where the research team had previously worked or conducted research, and may omit other important humanitarian contexts. Nonetheless, we remain confident our sample reflects an informatively diverse range of organisations and humanitarian contexts for our qualitative approach.