The process evaluation used a linguistic ethnographic,[31, 32] methodology, which combines strengths of linguistics and ethnography to systematically investigate human behaviour in context. Linguistic ethnography provides theoretical and methodological tools for analysing how the meaning of talk, text and objects shift over time and space. We have previously adapted this approach [33] to facilitate detailed investigation of complex healthcare interventions across macro-, meso- and micro-contextual levels, drawing on Bronfenbrenner’s socio-ecological model of behaviour[34].
Mixed methods were used including quantitative and qualitative data collection and analytic approaches, with all methods described in our overview paper. In this paper, we provide a detailed report of findings from the qualitative analysis of our observations of non-clinical areas, observed and audio-recorded consultations, documents and interviews with primary healthcare (PHC) facility managers, senior paediatric managers and policymakers.
Research Setting
The setting for this pilot and process evaluation was 10 public- sector PHC facilities serving impoverished urban and rural communities in the Western Cape province, South Africa. Child health services within PHC facilities are provided for children aged 0-13 years. Phase One took place in a single facility, Phase Two in an additional three facilities and Phase Three in a further six facilities. The facilities were purposively selected to provide maximum variation of primary care delivery in partnership with the Western Cape Health Department’s People Development Centre, which oversees training and upskilling of public sector healthcare workers in the Western Cape. See Curran et al. [28] for full details of facility characteristics and sampling strategy.
Data collection
To understand the macro-contextual features shaping delivery of the PACK Child intervention, interviews were conducted with managers at each PHC facility, and a stakeholder focus group with senior paediatric managers, policymakers and clinicians. Facility managers were asked about staff resource allocation to paediatric care, relevant policies, patient flow and perceptions of the PACK Child intervention for supporting the care of children. Senior paediatric managers and policymakers were asked about challenges of the current healthcare system and how they viewed the role of PACK Child in helping to address those challenges. We also conducted a documentary analysis of the structure and content of 1. The PACK Child guide, 2. The IMCI guide and checklist, 3. Integrated Clinical Stationery and 4. The Road to Health Booklet (old version) [35] to understand how the broader principles underpinning these different texts are operationalised to deliver paediatric primary care (see Table 1 and Additional files 1-4).
To understand the meso-contextual features shaping delivery of PACK Child, we drew on the PHC facility manager interviews, in conjunction with observations of waiting room and reception areas to understand the flow of patients through the facility. Using a qualitative observational framework, (see Additional file 5), the researcher recorded field notes of their observations of how children accessed care within facilities, from reception to different clinicians/providers.
To understand how clinician’s use of PACK Child articulated with micro-contextual features of paediatric primary care we conducted observations and audio-recordings of clinical consultations with children and caregivers in each of the pilot facilities. Consultations were conducted in the language or languages the caregiver, child and clinician were most comfortable communicating in. Recordings of consultations conducted in Afrikaans and isiXhosa were translated and transcribed in English. A researcher (RC or JM) was present in the consultation room at the time of recording in order to observe and document how clinicians used PACK Child and other documentation during the consultation, as well as other relevant non-verbal behaviour which contributed to understanding the consultation.
The PACK Child Intervention
The PACK Child guide, which is aligned with recognised standards for guideline development,[36, 37] is an evidence–informed, policy-aligned integrated clinical decision support tool, including algorithms that facilitate identification of likely diagnoses. The guide is designed to be adapted to LMICs globally, covering 63 common symptoms, including IMCI components such as diarrhoea and pneumonia, but importantly, it extends the scope of IMCI by focusing on children 0-13 years. It is also designed to address 16 long-term health conditions most commonly seen in primary care, as well as including a comprehensive approach to screening the well child. Routine care of the well child (see Additional file 1) includes measuring and interpreting growth, screening developmental milestones, checking immunisations, deworming, vitamin A, TB and HIV screening, as well as asking about the mental health of the child or problems in school. It also encompasses an assessment of the carer’s health including screening for psychosocial risk factors such as depression, violence in the home or financial difficulties. Routine care is intended to be sequenced after establishing the need for urgent care for the presenting symptom, but before definitive care for non-urgent symptoms. Clarity around clinician scope is provided with medications colour-coded according to prescriber. Designed to promote the continuum of care required to break the acute episodic care cycle, the guide prompts routine care into every consultation. Its content reinforces the messaging of existing initiatives like the Road to Health Booklet Side-By-Side messaging, the First 1000 Days initiative and the Nurturing Care framework.
Drawing on the successful PACK Adult training methodology, the PACK Child training programme used an onsite in-service cascade model (see Additional file 6) to be delivered in three phases for the pilot[19]. The first phase included one facility trained by a KTU trainer, the second phase included three facilities trained by two KTU trainers and the third phase conducted at six facilities was rolled out two by PACK Child Facility Trainers - government employees trained into the role by KTU during a five-day off-site workshop. The training included eight onsite training sessions delivered weekly in the PHC facilities; this was expanded to nine during phase two of the pilot to include a “health systems session” focusing on patient flow and distribution of tasks among cadres in contact with children. The training was designed to target all cadres of clinicians at facilities, mainly nurses and doctors and emphasises the alignment of the PACK Child content to IMCI, integration of care for the child’s caregiver using PACK Adult, and upskilling of all clinical staff to encourage a multi-disciplinary approach to paediatric primary care.
During the course of the pilot, bi-weekly meetings were scheduled to feedback on the content of the guide and issues with implementation in practice. This provided a regular opportunity to capture further refinements and clarifications in the PACK Child guide and for the training development. One of the content developers attended the training sessions in the first phase to ensure the usability of the guide and identify challenges within the primary care setting.
Eligibility and Sampling
To be eligible for inclusion in the study, nurses and doctors needed to receive PACK Child training, and caregivers and children aged birth to 13 years needed to be receiving paediatric services at the selected facilities. Policymakers needed to be responsible for delivery of primary care in public sector PHC facilities.
Sampling of children for consultation observations was dependent on which children presented at the facility on the day of data collection, with nurses identifying and approaching suitable participants in the waiting room areas. However, the limited number of children in Phase One who had a chronic condition or were older than five years informed identification and inclusion of these children Phases Two and Three. Similarly, the inclusion of only nurses in Phase One informed a proactive attempt to include doctors in Phases 2 and 3.
Ethics
Ethics approval was obtained from University of Cape Town Human Research Ethics Committee, City of Cape Town Research Ethics Committee and the Western Cape Provincial Health Research Committee. Full details of our sampling strategy and ethical considerations are provided in Curran et al [28].
Data Analysis
To understand how PHCs were organised to provide child care, and the interaction between contextual features and intervention delivery, we firstly analysed manager and policymaker interview data, and field notes of our observations of waiting rooms and reception areas. All interviews were transcribed verbatim and thematically analysed. Themes and field notes from observations of waiting room areas were compared to identify and describe similarities and differences in the organisation and flow of patients across facilities. Secondly, we analysed the audio-recordings, transcriptions and researcher field notes of consultations to understand how macro- and meso-contextual features shaped, and were shaped by nurse’s interactions with caregivers and children. A key focus was to identify instances of how use of PACK Child aligned with routine practice, providing “telling cases”,[38] of the wider social forces structuring intervention delivery at the point of delivery.
Audio recordings of consultations were transcribed verbatim. A sub-sample was transcribed using conversation analytic conventions,[39, 40] to provide detailed evidence of how clinician’s use of the PACK Child guide was negotiated within interactions with caregivers and children. We then inductively coded each transcript by activity, for example “eliciting the child’s presenting problem”, “physical examination”, or “advice giving”. We cross-referenced these against the field notes of the researcher’s observations to determine what documentation, if any, was used during each activity. This enabled us to obtain a broad picture of the structure of consultations within and between clinicians and facilities. We then coded clinician’s questions according to their function as part of the clinical assessment process (e.g. asking about presenting complaint, wider information gathering) and the structural form of the question (e.g. polarised, content or alternative question). This enabled us to understand patterns of questioning within each activity and the role of PACK Child and other documentation in shaping clinicians’ questioning. Using data collected during Phase One, one researcher (RC) completed all the coding of activities and questions and a second (JM) independently coded a sample 10% of the data. A Kappa score was calculated in a first round of question coding (0.72-0.83). Disagreements in coding and coding categories were discussed, refined and then second round of coding for a further 10% of questions conducted, revealing a high level of agreement (0.92-0.94). Finally we interrogated each transcript to understand the consequences of the consultation structure and question-response sequences for the ongoing interaction, how the clinician’s use of the PACK Child guide influenced the direction of the consultation, and how this use interacted with the use of other documentation.
Data synthesis
The analysis of qualitative data was iterative, moving between data collection and analysis to test emerging theories, comparing how managers’ views related to actual implementation of primary care and use of PACK Child. For example, managers reported particular facility processes or protocols that we then compared with our observations of waiting room areas and clinical consultations. Instances of how PACK Child aligned with routine practice within consultations provided insight into the tensions between different contextual features which we could then investigate further in subsequent observations and triangulate with data obtained from manager interviews.
The synthesised data were then used to map macro-, meso- and micro-contextual features with a consideration of how national policy at a macro level impacted on the organisation and skill mix of staff at a meso level, and then ultimately how care was delivered to children at a micro level within consultations. By focusing on (mis)alignments to implementation and setting the PACK Child intervention within a contextual framework, we were able to make the transition from the identification of patterns of PACK Child use in specific facilities, to theoretical explanations of how different structural relations and mechanisms organise moments of delivery, facilitating generalisable inferences and predictions on how to optimize PACK Child for future implementation.