The results section includes first a presentation of the empirical findings linked to each of the seven domains in the NASSS framework, followed by an analysis of how the complexity inherent in each domain and the interaction between the domains influenced the QI programme. Categories identified within each domain are presented in Table 3.
Table 3
Description of sub-categories linked to each domain of the NASSS framework
Domain | Category |
Condition (D1) | Broad patient population including both high and low risk patients with diverse background. |
Large birth clinic also accepting patients from other regions. |
Technology (D2) | The platform enabled staff to easily understand data and to gain new knowledge; prior understanding of statistics was helpful. |
The tool was easily accessible. |
Timely feedback of data made it more relevant for QI than historical data previously used. |
Case-mix adjustment made the data more relevant for QI. |
Highly detailed data was important for its use in QI. |
Lack of detail in certain data in the tool resulted in the need of additional data collection from other sources. |
Adaptation of the platform to meet future needs was deemed possible. |
Data was generally considered reliable and credible. However, use of data led to increased awareness of differences in measurement for certain variables which led to discussion about data quality and coding. |
Collaboration and a mutual learning process with the platform supplier enabled the continous continuous adaptation of the platform tool to the clinic's specific needs. |
Value proposition (D3) | The cross-regional Sveus report created an awareness of around differences in performance and the possibility of using data for QI purposes. |
Previously used data was not suitable for QI due to large time lag and poor accessibility. |
Adopter system (D4) | The clinic had an established culture of working interprofessionally. |
Interprofessional teams were thought to create unity around improvements and lead to better care for patients. |
Increased use of measurable data could create excessive focus on risk groups over "normal" patient groups, groups over the individual patient, and measurable aspects over unmeasurable aspects of care. |
The project did not require any significant changes in staff roles,but led to the need of development of some new skills in the clinical work. |
There was an ambition of evolving staff’s role in using data independently, but lack of hindered this development. |
Data visualized the need for change and improvements in results and triggered a dialogue which motivated staff to further improve. |
Data united staff around a common understanding of the current situation and emphasized the need for change. |
Better understanding of data could improve information to patients. |
Domain | Category |
Organization (D5) | Data was not available for staff who were not involved in QI. |
Staff not involved in QI had little insight into the innovation and the work done within QI teams. |
Staff not involved in QI were involved only as recipients of directives decided within QI teams. |
Even though the ambition was that anyone should access the data, in practice the managers and staff involved in QI were the ones who used it and then presented it to the staff. |
An already established digital way of working facilitated the use of the tool. |
A clear mandate was perceived to be needed to conduct QI. |
A long learning process was required for managers to understand the tool before beginning its implementation in the clinic. |
Use of the tool was promoted by managers through creating a curiosity and demand for the information, which was intended to secure longevity of the initiative |
Engaged manager enabled the implementation of the project. |
Recruitment to QI teams was based on expressed interest or decided by managers. |
Lack of time and difficulty scheduling limited staff's ability to work with QI and use the data. |
Use of data led to improvements in data registration in order to improve data quality. |
Wider System (D6) | Data improved communication around patients with other sections of the women’s clinic, the neonatal clinic and external actors. |
Medial discussion about the quality of birth care caused concern amongst patients |
The Region had a goal to reduce infections, but this was not important for the initiation of the QI programme. |
A hospital-wide programme on VBHC supported the implementation of the tool to some extent. |
Condition: Pregnancy spans from simple to complex (D1)
Representatives from all focus groups described that the unit treated a broad patient population e.g. both emergency and elective care took place at the unit and that some patients were low risk in normal labour while others were high-risk with complex conditions such as premature delivery and maternal co-morbidity.
Technology: A practical but not trivial analytics tool (D2)
Representatives of the managers and staff involved in QI described that the feature of case mix adjustment made data more relevant compared to unadjusted data and counteracted the practice of justifying poor performance outcomes with misconceptions about patient complexity.
(One informant) – And the case mix adjustment has made a difference. Before we blamed a lot on the fact that our patients are so special. (Another informant) – Yes, absolutely, [we said] “We have so difficult patients” and “It’s a little bit special here”. [Managers]
Staff involved in QI teams expressed that the level of data detail was generally high which was considered important for its usage, although in some cases it was too coarse. Moreover, both managers and staff involved in QI perceived the timelines of data feedback to be relevant and useful.
(One informant) - Now it is more easily accessible. (Another informant) - Quickly look at recent data that are divided into different focus areas so that you can quickly get an overview as you say. “The last month something has happened, it is suddenly 30% caesarean sections, what should we do?” [Staff involved in QI teams]
The managers said the dashboards were easily available on any devices such as computers, smart phones, and tablets. The graphic presentation of the data via the web interface was perceived as understandable, user-friendly, and clear by managers and staff involved in QI-teams. The staff who were involved in QI teams reflected on their different preferences concerning the visual presentation of data and they suggested that further improvements in the interface would increase data accessibility even more. The two staff groups said more guidance and support was needed for identifying, selecting, extracting, and understanding relevant data. It was also described by involved staff that prior understanding of statistics was helpful to be able to fully grasp the presented data.
Representatives from the manager group and staff involved in QI said that the relationship with the supplier was such that it was possible to customize the analytics tool to the local needs and conditions. For example, in addition to indicators established through the cross-regional benchmarking, the supplier facilitated measurement of indicators specifically demanded by the unit, incorporated local data available in the department’s EMR, and adapted to the hospital’s system for data transfer. Involved staff described how they contributed with their clinical knowledge to identify what data was needed to guide QI. The supplier was in turn able to translate these needs into data demands to the hospital-IT department.
Value proposition: timely and reliable data (D3)
According to informants from all focus groups, the analytics tool was needed because of the existing QI challenges. The managers described that a previous cross-regional report, from the Sveus project, suggested there was room for improvement in a number of areas. This report was said to motivate the unit to require more data to better understand performance and to initiate improvement activities. The managers emphasized that the cross-regional report which presented case mix adjusted variations was a motivating factor. It became clear that there were areas of underperformance in the unit, and that the patient mix was not the cause of the variations. The managers described that even before the introduction of the technology-supported QI programme, data were seen as essential to QI in the unit. However, historically data were often difficult to access and were often out-of-date.
(One informant)- Statistics has been our weak spot. We’ve been able to measure but it has been difficult. And really difficult sometimes when we wondered: ”How many women with diabetes do we have?” or ”How many complications do we have?”. So it was very difficult to get that data (Another informant) – It was basic, with a pencil and put into Excel-files. [Managers]
Adoption system: managers and QI teams use the tool for improvement purposes but need complementary data to capture the patient perspective more comprehensively (D4)
The managers and the staff involved in QI teams were the main adopters of the analytics tool. They described how they used the tool to identify performance deviation and improvement needs that were grounded in valid information. They also used the tool to evaluate the effect of changes to the unit’s protocols.
Despite the clear use of the tool for improvement purposes, the adopters reflected on the need to integrate other sources such as surveys, focus groups, and individual interviews, to complement the data provided by the tool with data on patient perspective. Medical records were also reviewed. Taken as a whole, all these data sources were used to make changes to the unit’s protocols. In some cases, new protocols were adopted that required the staff to acquire new skills and to develop new competences.
Some informants reflected on the potential risks of using performance data in QI. The managers worried that the improvement efforts would shift focus to certain metrics and to aggregated patient groups over the needs of the individual patient. Staff were concerned that the improvement efforts would focus on measurable areas to the exclusion of areas that were not easily quantified.
According to the managers, one goal of the QI programme was that all managers and staff should be able to easily access and use the analytics tool. In practice, staff not involved in QI had limited or no knowledge of, or experience with, the tool; yet they expressed an interest in learning about the improvement efforts as they talked about the tool.
(One informant) - I haven’t even seen it. (Another informant) - Nor have I. (Yet another informant)- I think we should see it more. (The first informant)- You could do it very shortly in a group like this. One afternoon, just bring up that: ”This is what it looked like three months ago and today it looks like this. Look how good it is”. [Staff not involved in QI teams]
This was further corroborated by the managers who said it would be beneficial if staff could start to use the tool independently; however, this would require time and knowledge that was not available.
One manager said she could use the data to communicate with patients about the medical risks associated with their conditions and staff members said that data helped them to feel more confident when they explained the reasons behind their decisions to patients.
”And then be able to present to the patients also that ”If we induce labour in this way then we have a high proportion who deliver vaginally, where everything works well, and these are the risks if you start to induce labour very early”. So that you also have a solid fact base for your own sake. [Staff involved in QI teams]
Organization: The tool supports multidisciplinary QI-efforts and emphasizes the need for change (D5)
Several organizational factors were described to have supported the adoption of the analytics tool such as the multidisciplinary approach to QI, formal education seminars and workshops, and an implementation approach that focus on demonstrating the benefit of the tools.
Managers saw the unit as a pioneer in QI as they had run several QI initiatives, such as lea. The unit had used data in QI efforts before and had a practice of working in multidisciplinary teams. Staff involved in QI described how the improvement work done in multidisciplinary teams did not use a specific and standard approach to QI. Instead, the teams self-determined how to organize their work. Participants in QI teams were included on a voluntary basis or via appointment by managers. However, representatives from all three groups expressed that an impediment to engaging in the QI teams was insufficient time and that meeting times conflicted with clinical engagements.
The managers described that the long experience of working in multi-disciplinary teams supported the implementation of new clinical routines owing to the diversity of knowledge of, and experience with, current clinical practices and created unity around changes.
The managers also described that formal education seminars and workshops also promoted acceptance of the QI programme. However, the staff who were not involved in the QI programme described themselves as “passive” recipients of the new practices. It was not clear to them how the new routines were developed or by whom. The staff who were engaged in the QI programme acknowledged the need for better communication and interactions between the QI teams and other employees.
Staff involved in QI teams described that the head of unit played an important role in the adoption of the analytics tool since she had great interest in development of care and increased patient safety. The managers described that her approach to implementing the tool was to demonstrate the possibilities the tool offered. The managers argued that this was a strategy to ensure the sustainable adoption of the tool, and the technology-supported QI was seen as a long-term ambition as much as an immediate goal.
Both involved staff and managers described that the improved access to data unified staff and managers around their interpretation of current performance and the goals of the QI programme. The managers described how they used the data to motivate staff by calling attention to the variations between observed and expected levels of performance indicators. They also used the data to generate greater interest in performance measurement and to improve the dialogue with other hospital departments.
But it is also the fact that it becomes easier with the communication to the staff. Because data has been old previously and what kind of feedback is that? People could say…” Yes, that’s the way it was then, when they worked here, and not me”. Now, we are looking at two-week old data… I believe the discussion is much more here and now which also makes it easier to motivate. [Managers]
Both manager and involved staff mentioned that the increased focus on performance measurement and benchmarking led to an increased and shared understanding of the importance of data reliability. This gradually had led to efforts made to improve routines for data recording. For example, changes were made to standardize the data entries in the EMR system.
Wider system: The political and societal debate reinforced the need for improvement in obstetrics care (D6)
Staff involved in QI mentioned how the ongoing societal and political debate emphasized the need to improve birth care. For example, media reports on the quality of birth care had created concern among patients which triggered the need to act upon this situation.
Some objectives of the QI programme, such as the reduction in the number of infections, were identified by involved staff as goals set at the county level. These externally set goals only indirectly affected the units efforts to improve birth care.
The managers reported that the QI initiative at the obstetric unit linked well to VBHC, which provided additional support for the QI programme.
Embedding: Close collaboration with the supplier enabled adaptation over time (D7)
The research mainly focused on the early adoption of the QI programme. Thus, only limited findings were identified concerning D7. A factor that contributed to adaptation over time was the close collaboration between the supplier of the tool, the managers, and the staff to adapt the tool to local needs and conditions. They learned that data reliability and validity were essential and could not be taken for granted.
Analysis of complexity
Patients treated at the unit ranged from simple, i.e. low risk, to complex, i.e. high-risk patients and with varying comorbidities (D1). In this setting, the new technology provided case mix adjusted performance indicators that enabled staff and managers to better understand the complexity that characterized their patients and to trust the performance measurement (D2).
Even with the case mix adjusted data, the data provided in the tool were not always sufficient to fully grasp quality. Therefore, multiple data sources were used to complement the tool. These factors complicate the adoption since multiple components and agents need to interact to get a broad perspective on performance (D4).
The analytics tool required significant adaptation from both the supplier and its adopters. It was necessary to customize the performance indicators and to integrate the technology with existing data systems (D2). Thus, the supplier and the adopters modified and co-developed over time specific features of the tool. This complex adaptation process ultimately resulted in a simpler and more practical technology that produced timely and reliable data (D2). The high desirability of the technology (D3) also contributed to make the technology a good fit for the adopters, i.e. the possibility the technology offered to support QI based on reliable data.
The relative simplicity of the adopter system, as limited changes were needed to staff roles and routines (D4), combined with several organizational factors, reduced the uncertainty associated with the new technology. The QI programme was limited to the obstetric unit where the multi-disciplinary QI teams were already in place and were led by a motivated leader (D5). All this contributed to a successful initial adoption. However, an observation was made that staff not involved in QI were not fully aware of the technology.
As far as the wider system (D6) limited insights were gained from the interviews. The hospital-wide VBHC initiative, however, seemed beneficial since it linked to the QI programme. Our previous knowledge of this area suggests that other important factors in the wider system may have facilitated the local adoption process. The national, cross-regional benchmarking initiative, Sveus, which dealt with a number of challenges related to data collection, informatics, methodology, and legal issues, had smoothed the technology adoption path for the QI programme. Moreover, previous work by Sveus had helped to legitimize the selection and definition of relevant birth care indicators. This can be argued to have reduced the complexity of the adoption.