Study setting
The research was conducted in Italy, within the SONAR project (Observational Study in Neonatology: Assistance and CaRe). In 1999, the Italian Neonatal Network (INN) set up the 5-year SONAR project. INN includes 93 NICUs with the goals of improving safety and quality of care in neonatal care by using research, training and quality improvement projects. Since 2004, INN has been part of the Vermont-Oxford Network, the most important neonatal network in the world, which includes more than 1,300 hospitals across 30 countries working to continuously improve neonatal care. 63 NICUs of the INN participated voluntarily in the SONAR project. SONAR’s aim was to identify the characteristics of NICUs, monitor outcomes and promote best practices [31].
Data collection
Our study includes 51 of the 63 NICUs that took part in the SONAR project. Operational units (OUs) that did not present complete data in managerial variables or outcomes were excluded from our sample. SONAR covered all healthcare professionals (doctors and nurses) who work in NICUs, preterm babies, and their parents. Questionnaires on the activities and managerial features of the OU were administered in written form to nurses, doctors and parents. The questionnaire was developed by the Steering Committee of SONAR on the basis of a review of international literature and a meeting with health professionals. The questionnaire was validated by the Advisory Board and the Steering Committee of SONAR and pilot testing of questionnaires was conducted in several OU that took part in the project.
For this study we used only responses relating to medical and nursing staff.
The questionnaires were only administered to doctors and nurses who had been working in the NICU for at least three months. Each NICU had a manager (Local SONAR manager), belonging to the medical staff of the unit, who provided data about the activities and outcomes of its OU and was responsible for coordinating the project at local level.
Ethical approval
The study complies with the Italian regulations on observational clinical studies (Ministerial Circular n. 6 of 2/9/2002). The Ethics Committees of each centre involved in the SONAR project approved this study.
All the respondents to the questionnaire received an information sheet, containing a clear and comprehensive definition of the characteristics and objectives of the study. All the respondents provided their written consent to the processing of personal data.
Managerial features
For each of the areas of the framework developed by Øvretveit for WHO [8], one variable was identified to describe the managerial features (MF) of the NICU. These features can be used to describe different managerial models.
MF1. Complexity of case
Local SONAR managers detected the acuity score for each neonate hospitalised with their own NICU. The acuity score is a tool developed by the Vermont-Oxford Network to define the type of patients admitted to the NICU, and includes five categories of increasing severity and care commitment [31, 32]. The lowest level of complexity is associated with the infant who requires minimal care, while level 5 is associated with the unstable infant who requires complex intensive care. Each NICU was thus assigned an acuity score given by the average of the acuity scores of all infants treated within it. NICUs that show a higher acuity score, nearer to 5, handle more complex cases on average and have a greater absorption of resources than NICUs with a lower acuity score nearer to 1.
In this study, we used the acuity score as an indicator available to managers for the evaluation of the complexity of NICU activities (MF1). It thus applies to the first area of the managerial model: “Performance and results review”. Previous studies have used case complexity as a performance indicator [20, 33]. The United Kingdom Neonatal Staffing Study Group also suggests that case complexity, together with random chance and care quality, is one of the main elements leading to variability in outcomes in neonatal care [34].
MF2. Performance measurement
Doctors in their questionnaires were asked to indicate the system used in their NICU to evaluate qualitative performance by choosing one of the five following statements : 1) Quality measurement is sporadic; 2) The individual evaluates his or her own performance using information made available by the hospital; 3) Regular meetings between colleagues to discuss some clinical cases; 4) We jointly evaluate all cases not following defined procedures; 5) Periodical evaluation on quality indicators by external entities. The first and fifth statements express two extreme methods of evaluating performance. The first embodies a managerial approach poorly oriented towards quality assessment, which is left to the will of individual professionals. The fifth statement, on the other hand, presents a clear orientation of management to robust measurement systems. For each NICU, the score for the performance measurement system was defined through the evaluation expressed by its doctors. NICUs that show values nearer to 5 use more robust measurement systems than those that have a score that tends to 1.
The use of performance measurement systems is the basis for activating benchmarking processes [35]. Managers can compare data collected from their measurement system with data from other hospitals, or with their data collected in the past to develop adequate improvement processes. This study uses the type of performance measurement system (MF2) adopted to evaluate the “Benchmarking” area in the NICU management model.
MF3. Participation and support
The Practice Environment Scale of the Nursing Work Index (PES-NWI) is included in the questionnaire for nursing staff. PES-NWI is a tool developed by Lake for measuring the working environment and has a high degree of reliability and validity [36]. PES-NWI is made up of 31 items evaluated on a 4-point Likert-type scale, from 1 (maximum disagreement) to 4 (complete agreement). Items are grouped by Lake into five areas, two of which were used for this study. “Nurse participation in hospital affairs” and “Nurse manager ability, leadership, and support of nurses” areas measure whether or not the hospital uses participatory leadership in managing activities. For each NICU, the average score was calculated in the two areas of the PES-NWI. An average tends towards 4 shows that an NICU is oriented towards participation, support and involvement of nurses. This element is used to identify the style of “Leadership” (MF3) in the NICU managerial model.
F4. Foundations for quality of care in nursing
Nurses are the professionals most involved in neonatal intensive care and are characterised by a very high level of skills. In PES-NWI, two items are useful for measuring NICU propensity to involve nurses more actively in the care processes: 1) Use of nursing diagnoses; 2) Nursing care is based on a nursing, rather than a medical, model. These two items of the PES-NWI show to what extent the quality of care in the NICU is based on guidelines and procedures developed by nurses (F4). Together these two items can be used to measure the “clinical guidelines, protocols and procedure” area of the managerial model [37, 38]. As for the previous variable, as the value of the variable increases towards 4, the NICU propensity to use guidelines, protocols and procedures increases.
MF5. Motivation of medical and nursing staff
Doctors were asked to evaluate the level of satisfaction within their NICU on a four-point Likert-type scale (from 1: very low, to 4: high). The average level of its staff was calculated for each NICU. Motivation levels nearer to 4 express the managers’ ability to create a favourable working climate within the OU [39]. Managerial practices that promote job satisfaction and emotional support are also important for preventing burnout [40]. The motivation of medical and staff nursing (MF5) is thus closely linked to the last area of the managerial model: staff satisfaction.
Neonatal Outcomes
Neonatal outcomes used for this study were defined as for the Vermont-Oxford Network. Nine neonatal outcomes are analysed. Below is a brief description of each.
Mortality refers to intra-hospital mortality. Nosocomial infection indicates whether the infant has late bacterial infection (including Coagulase Negative Staphilococcus) after day three of life. Severe Intraventricular Haemorrhage (severe IVH) indicates whether the infant has a grade 3 or 4 periventricular-intraventricular haemorrhage. Cystic periventricular leukomalacia (PVL) refers to multiple small periventricular cysts identified on a cranial ultrasound, computerised tomography scan, or magnetic resonance imaging scan. Severe Retinopathy of Prematurity (ROP) indicates whether the infant has a stage 3, 4 or 5 ROP. Necrotizing Enterocolitis (NEC) was diagnosed at surgery, at post mortem examination, or clinically (bilious gastric aspirate or emesis, abdominal distension, occult or gross blood in stool) and radiographically (pneumatosis intestinalis, hepato-biliary gas, pneumoperitoneum). Pulmonary Bronchodysplasia (BPD) refers to oxygen at 36 weeks postmestrual age. Newborns discharged before 36 weeks do not have BPD. Morbidity indicates whether the infant survived with one of the following key morbidities: severe IVH, BPD, NEC, pneumothorax, nosocomial infection, or PVL. Human milk indicates whether the infant was discharged receiving human milk (including human milk fortifier and/or formula milk).
All the outcomes were adjusted with a model including gestational age, gestational age squared, multiple gestation, outborn status, Apgar score, gender, Caesarean section delivery, and presence of a congenital anomaly.
Data on outcomes are collected by the Local SONAR manager for each NICU. Low values associated with outcomes means better outcomes, except for human milk where higher values are preferable.
Data analysis
We conducted a cluster analysis to identify managerial models of NICUs. All statistical analyses were carried out using SPSS Statistics Version 25. Variables were first normalised in order to identify the outliers, which can in fact significantly influence the results of the cluster analysis [41]. Patterns of respondents’ answers were then examined by applying the agglomerative hierarchical cluster analysis using the Ward method. The hierarchical cluster analysis with its dendrogram and statistics measuring the cluster fit (pseudo F-statistic) are useful to identify the number of the clusters. In a cluster analysis, in fact, the number of clusters is subjective. In our case, dendrogram and pseudo F-statistic suggested three clusters as the best option. Final clusters were thus formed using k-centered clustering with three groups. The method of Euclidean-distance was used in order to calculate differences between the variables. ANOVA was used to test whether the clusters significantly differ from each other in terms of the five managerial features used in the cluster analysis.
After the formation of the three managerial models, we conducted post-hoc tests (Bonferrroni) to assess differences between clusters on the neonatal outcomes.
Finally, in order to better describe the managerial models of NICUs we also examined the association between other key elements of the NICU and cluster membership. These elements can be very useful for measuring the resources available to the NICU. The structural aspects were: size (number of beds), and human resource endowment (number of doctors and nurses per bed and per 1,000 day of length of stay). To investigate differences between clusters on structural aspects we used one-way ANOVA and Bonferroni’s test.
All statistical analyses were conducted using a 5% significance level (p-value: α < 0.05).