The process evaluation used a linguistic ethnographic [30, 31] methodology, which combines strengths of linguistics and ethnography to systematically investigate human behaviour in context. Linguistic ethnography provides theoretical and methodological tools for analysing how the meaning of talk, text and objects shift over time and space. We have previously adapted this approach [32] to facilitate detailed investigation of complex healthcare interventions across macro, meso- and micro-contextual levels, drawing on Bronfenbrenner’s socio-ecological model of behaviour, which conceptualises individual action as a response to socially structured processes and characteristics, organised across a layered system of relationships [33].
Mixed methods were used including quantitative and qualitative data collection and analytic approaches. Qualitative data included observations of training sessions; semi-structured interviews with caregivers; clinician, policymaker and paediatric manager focus groups; documentation used in child consultations; and ethnographic observations of consultations and non-clinical areas in each facility. Quantitative methods included auditing of training attendance logs and clinician’ questionnaires completed six months after finishing the PACK Child training programme. In this paper, we provide a detailed report of findings from the qualitative analysis of our observations of non-clinical areas, observed and audio-recorded consultations, documents and interviews and focus groups with primary healthcare (PHC) facility managers, senior paediatric managers and policymakers.
Research Setting
The setting for this pilot and process evaluation was 10 public- sector PHC facilities serving impoverished urban and rural communities in the Western Cape province, South Africa. Child health services within PHC facilities are provided for children aged 0-13 years. Phase One took place in a single facility, Phase Two in an additional three facilities and Phase Three in a further six facilities. The facilities were purposively selected to provide maximum variation of primary care delivery in partnership with the Western Cape Health Department’s People Development Centre, which oversees training and upskilling of public sector healthcare workers in the Western Cape – see Table 1. Factors considered important for observing variation included whether clinics were Ideal Clinic sites, (an initiative to improve quality of primary healthcare) (34), number of IMCI-trained nurses; differing levels of PACK Adult training coverage; and use of Integrated Clinical Stationery (an initiative to standardise documentation and facilitate continuity of care of children up to six years).
Table 1: Characteristics of PACK Child Pilot Facilities
Phase
|
Facility
|
Urban/Rural
|
Jurisdiction
|
No IMCI trained
|
No PACK Adult trained
|
Ideal Clinic Site
|
ICS pilot
|
Total Number of Staff
|
Number seeing children
|
Brief description of facility
|
Number completed PACK Child training
|
Average Facility Attendance
(04/2016-04/2017)
|
Under 5
|
5-9 years
|
1
|
1
|
Urban
|
Municipal
|
11
|
26
|
No
|
No
|
26
|
4
|
1 triage area (ENA)
1 EN Immunizations
2 PN for sick child
|
15
|
1000
|
Not Available
|
2
|
2
|
Urban
|
Provincial
|
9
|
36
|
No
|
Yes
|
38
|
4
|
1 triage area (EN);
1 PN immunizations,
2 PN Sick child
|
9
|
1153
|
295
|
3
|
Urban
|
Municipal
|
9
|
20
|
No
|
No
|
20
|
6
|
1 EN/PN Triage
1 EN Immunizations
1 EN PMTCT
2 PN Sick child
|
9
|
1000
|
Not available
|
4
|
Rural
|
Provincial
|
5
|
15
|
Yes
|
Yes
|
16
|
8
|
1 EN Immunization/triage;
5 CNPs Sick child
|
6
|
700
|
Not available
|
3
|
5
|
Rural
|
Provincial
|
10
|
9
|
No
|
No
|
18
|
9
|
All staff see well child and sick child
|
13
|
1153
|
28
|
6
|
Urban
|
Provincial
|
Not Available
|
Not Available
|
Yes
|
Yes
|
84
|
3
|
Dermatology and Asthma Clinic, Trauma, Recent well child visits (1 PN)
|
9
|
1944
|
199
|
7
|
Urban
|
Provincial
|
1
|
40
|
No
|
No
|
64
|
2
|
1 PN Immunizations
1 PN Sick child
|
17
|
2061
|
13
|
8
|
Urban
|
Provincial
|
3
|
20
|
Yes
|
Yes
|
20
|
2
|
1 EN Immunizations
1 CNP Sick Child
|
5
|
535
|
24
|
9
|
Urban
|
Provincial
|
9
|
37
|
Yes
|
Yes
|
37
|
1
|
Currently mainly see children in trauma; but introducing well/sick childcare
|
8
|
144
|
227
|
10
|
Urban
|
Municipal
|
7
|
12
|
No
|
No
|
12
|
6
|
1 EN Immunizations;
1 PN Sick children
|
8
|
977
|
18
|
Data collection
To understand the macro-contextual features shaping delivery of the PACK Child intervention, interviews were conducted with managers at each PHC facility, and a stakeholder focus group with senior paediatric managers, policymakers and clinicians. Facility managers were asked about staff resource allocation to paediatric care, relevant policies, patient flow and perceptions of the PACK Child intervention for supporting the care of children. Senior paediatric managers and policymakers were asked about challenges of the current healthcare system and how they viewed the role of PACK Child in helping to address those challenges. We also conducted a documentary analysis of the structure and content of 1. The PACK Child guide, 2. The IMCI guide and checklist [9], 3. Integrated Clinical Stationery and 4. The Road to Health Booklet (old version) [35] to understand how the broader principles underpinning these different texts are operationalised to deliver paediatric primary care (see Table 2 and Additional files 1-4).
To understand the meso-contextual features shaping delivery of PACK Child, we drew on the PHC facility manager interviews, in conjunction with observations of waiting room and reception areas to understand the flow of patients through the facility. Using a qualitative observational framework, (see Additional file 5), the researcher recorded field notes of their observations of how children accessed care within facilities, from reception to different clinicians/providers.
To understand how clinician’s use of PACK Child articulated with micro-contextual features of paediatric primary care we conducted observations and audio-recordings of clinical consultations with children and caregivers in each of the pilot facilities. Consultations were conducted in the language or languages the caregiver, child and clinician were most comfortable communicating in. Recordings of consultations conducted in Afrikaans and isiXhosa were translated and transcribed in English. A researcher (RC or JM) was present in the consultation room at the time of recording in order to observe and document how clinicians used PACK Child and other documentation during the consultation, as well as other relevant non-verbal behaviour which contributed to understanding the consultation.
The PACK Child Intervention
The PACK Child guide, which is aligned with recognised standards for guideline development [36, 37] is an evidence–informed, policy-aligned integrated clinical decision support tool, including algorithms that facilitate identification of likely diagnoses. The guide is designed to be adapted to LMICs globally, covering 63 common symptoms, including IMCI components such as diarrhoea and pneumonia, but importantly, it extends the scope of IMCI by focusing on children 0-13 years. It is also designed to address 16 long-term health conditions most commonly seen in primary care, as well as including a comprehensive approach to screening the well child. Routine care of the well child (see Additional file 1) includes measuring and interpreting growth, screening developmental milestones, checking immunisations, deworming, vitamin A, TB and HIV screening, as well as asking about the mental health of the child or problems in school. It also encompasses an assessment of the carer’s health including screening for psychosocial risk factors such as depression, violence in the home or financial difficulties. Routine care is intended to be sequenced after establishing the need for urgent care for the presenting symptom, but before definitive care for non-urgent symptoms. Clarity around prescribing scope is provided by colour-coding each medication according to prescriber level. Designed to promote the continuum of care required to break the acute episodic care cycle, the guide prompts routine care into every consultation. Its content reinforces the messaging of existing initiatives like the Road to Health Booklet Side-By-Side messaging [35], the First 1000 Days initiative [38] and the Nurturing Care framework [39] (see Table 2).
Drawing on the successful PACK Adult training methodology [18], the PACK Child training programme used an onsite in-service cascade model (see Additional file 6) to be delivered in three phases for the pilot [19]. The first phase included one facility trained by a KTU trainer, the second phase included three facilities trained by two KTU trainers and the third phase conducted at six facilities was rolled out two by PACK Child Facility Trainers - government employees trained into the role by KTU during a five-day off-site workshop. The training included eight onsite training sessions delivered weekly in the PHC facilities; this was expanded to nine during phase two of the pilot to include a “health systems session” focusing on patient flow and distribution of tasks among cadres in contact with children. The training was designed to target all cadres of clinicians at facilities, mainly nurses and doctors and emphasises the alignment of the PACK Child content to IMCI, integration of care for the child’s caregiver using PACK Adult, and to develop the skills of all clinical staff to encourage a multi-disciplinary approach to paediatric primary care.
During the course of the pilot, bi-weekly meetings were scheduled to feedback on the content of the guide and issues with implementation in practice. This provided a regular opportunity to capture further refinements and clarifications in the PACK Child guide and for the training development. One of the content developers attended the training sessions in the first phase to ensure the usability of the guide and identify challenges within the primary care setting.
Eligibility and Sampling
To be eligible for inclusion in the study, nurses and doctors needed to receive PACK Child training, and caregivers and children aged birth to 13 years needed to be receiving paediatric services at the selected facilities. Policymakers needed to be responsible for delivery of primary care in public sector PHC facilities.
Data collection for the process evaluation occurred concurrently with the three phases of the pilot, enabling analysis of Phase One data to inform the sampling strategy in Phases Two and Three. All facility managers were invited to be interviewed. On a typical day, 2-3 clinicians consulted children and all were invited to participate in consultation observations. Purposive sampling was planned in Phase One to select and recruit caregivers and children and was intended to be informed by diversity of child conditions, level of deprivation and the age of the child. However, consultation observations were dependent on which children presented at the facility on the day of data collection, and on nurses identifying and approaching eligible participants in the waiting room areas. In Phase One, nurses approached all eligible participants unless they decided it would not be appropriate to do so (e.g. child needed urgent attention and the mother was distressed). However, the limited number of children in Phase One who had a chronic condition or were older than five years informed identification and inclusion of these children in Phases Two and Three. To do so, we asked facilities to prioritise approaching caregivers of children who met these criteria. Similarly, the inclusion of only nurses in Phase One informed a proactive attempt to include doctors in Phases 2 and 3. We asked doctors in each facility if and when they consulted with children and then asked them to approach the caregiver and child about participation in the research. Senior paediatric managers, facility managers, nurses and doctors involved in the pilot; and policymakers from the City of Cape Town and Western Cape departments of health were invited to participate in stakeholder focus groups to review findings and facilitate discussions on the implications of PACK Child for wider implementation.
Ethics
Ethics approval was obtained from University of Cape Town Human Research Ethics Committee, City of Cape Town Research Ethics Committee and the Western Cape Provincial Health Research Committee. Written consent for interviews and observations was obtained from all facility managers, clinicians and caregivers. Children over seven years old were asked to give assent to their participation. Caregivers and children were asked to consent to interview and observation on the day they attended the clinic. Facility managers provided consent for observations of non-clinical areas. All participants were provided with written information about the research, informed that their participation was voluntary and that they could withdraw from participation at any time.
Data Analysis
To understand how PHCs were organised to provide child care, and the interaction between contextual features and intervention delivery, we firstly analysed manager and policymaker interview data, and field notes of our observations of waiting rooms and reception areas. All interviews were transcribed verbatim and thematically analysed. Themes and field notes from observations of waiting room areas were compared to identify and describe similarities and differences in the organisation and flow of patients across facilities. Secondly, we analysed the audio-recordings, transcriptions and researcher field notes of consultations to understand how macro- and meso-contextual features shaped, and were shaped by nurse’s interactions with caregivers and children. A key focus was to identify instances of how use of PACK Child aligned with routine practice, providing “telling cases” [40] of the wider social forces structuring intervention delivery at the point of delivery.
Audio recordings of consultations were transcribed verbatim. A sub-sample was transcribed using conversation analytic conventions [41, 42] to provide detailed evidence of how clinician’s use of the PACK Child guide was negotiated within interactions with caregivers and children. We then inductively coded each transcript by activity, for example “eliciting the child’s presenting problem”, “physical examination”, or “advice giving”. We cross-referenced these against the field notes of the researcher’s observations to determine what documentation, if any, was used during each activity. This enabled us to obtain a broad picture of the structure of consultations within and between clinicians and facilities. We then coded clinician’s questions according to their function as part of the clinical assessment process (e.g. asking about presenting complaint, wider information gathering) and the structural form of the question (e.g. polarised, content or alternative question). This enabled us to understand patterns of questioning within each activity and the role of PACK Child and other documentation in shaping clinicians’ questioning. Using data collected during Phase One, one researcher (RC) completed all the coding of activities and questions and a second (JM) independently coded a sample 10% of the data. A Kappa score was calculated in a first round of question coding (0.72-0.83). Disagreements in coding and coding categories were discussed, refined and then second round of coding for a further 10% of questions conducted, revealing a high level of agreement (0.92-0.94). Finally we interrogated each transcript to understand the consequences of the consultation structure and question-response sequences for the ongoing interaction, how the clinician’s use of the PACK Child guide influenced the direction of the consultation, and how this use interacted with the use of other documentation.
Data synthesis
The analysis of qualitative data was iterative, moving between data collection and analysis to test emerging theories, comparing how managers’ views related to actual implementation of primary care and use of PACK Child. For example, managers reported particular facility processes or protocols that we then compared with our observations of waiting room areas and clinical consultations. Instances of how PACK Child aligned with routine practice within consultations provided insight into the tensions between different contextual features which we could then investigate further in subsequent observations and triangulate with data obtained from manager interviews.
The synthesised data were then used to map macro-, meso- and micro-contextual features with a consideration of how national policy at a macro level impacted on the organisation and skill mix of staff at a meso level, and then ultimately how care was delivered to children at a micro level within consultations. By focusing on (mis)alignments to implementation and setting the PACK Child intervention within a contextual framework, we were able to make the transition from the identification of patterns of PACK Child use in specific facilities, to theoretical explanations of how different structural relations and mechanisms organise moments of delivery, facilitating generalisable inferences and predictions on how to optimize PACK Child for future implementation.