Aim
The aim of this simulation study is to determine the maximum number of women specified clinic configurations, combination of specified clinic resources, can feasibly serve within clinic opening times.
The cost effectiveness of the configurations is outside the scope of this paper as we do not have access to patient outcome data. More nuanced appointment scheduling, to minimise the women’s waiting time and total LoS will be considered in future analysis. Due to the fluctuating DNA rate of women who miss appointments, as a result of giving birth, and the majority of women arrive before the clinic opens, the maximum number of women each configuration is considered the main model outcome.
Study Design and Setting
A computer simulation modelling study was conducted.
The simulation model represents the operation of the post-term pregnancy outpatient clinic at AHUS, a large Norwegian hospital. Prior to the catchment area change in 2011 AHUS served ~300,000 people afterwards it served ~480,000 people, of which ~61,000 and ~96,000 were females aged 15-44 respectively [54-56].
Characteristics of participants
The participants are hypothetical computer simulated pregnant women in GW 41, 7 – 9 days after term, who have been scheduled to attend the post-term pregnancy outpatient clinic.
Model Specification
The clinic operates between 08:00 and 10:30 four days a week. One day a week the clinic finishes at 09:30 as the doctors have to perform other tasks. The clinic operates on weekdays. In the current system appointments are given in 5 minute intervals which begin at 08:00 and run up until 09:00, resulting in 12 possible appointments. However, as previously mentioned a challenge of operating this clinic is the high risk of DNAs due to women giving birth before appointments, and also the uncertain demand, so the range of appointments actual attended is 0-15. Additionally, although women are given appointment times, those that arrive on the day of their appointment tend to arrive when the clinic opens. The model is constructed to evaluate the maximum number of appointments that can be satisfied within clinic opening times on a typical day, 08:00 to 10:30. The scheduling and outpatient configuration literature discussed previously generally assumes higher attendance rates and less uncertain demand.
Following Norwegian guidelines a post-term pregnant woman, with a singleton pregnancy without complications, is given a set of appointments to attend the clinic for checks. Women with multiple pregnancies or known complications are monitored by alternative mechanisms. If the woman attends her appointment, the standard process is that she will check in at reception. She will be called in to the midwife consultation which consists of various tasks including a CTG scan once resources, rooms and equipment are available. Following the midwife consultation if the doctor is free the woman will be called into the doctor’s office, for the doctor consultation. At the end of the consultation, the doctor will either recommend the woman for an induced birth or a follow-up appointment a few days later. The woman will check out at reception, return home or be scheduled for an induction.
An overview of the key components of the model is provided here. The hybrid Agent Based Discrete Event Simulation model is constructed in Any Logic Professional 8.3.3 [57]. DES is widely used to model systems in which entities in this case pregnant women pass through sequential activities that may be constrained by resources, e.g. space, staff, equipment. Due to resource constraints, variability in arrivals and activity service times, queues can form in these systems resulting in delays and bottlenecks. The clinic DES model captures the processes depicted in Figure 2.
Figure 2 An illustration of the pathway through the overdue clinic
The numbers illustrate the sequence of activities. The current pathway is indicated with “a”. A proposed alternative pathway is indicated with “b”. The letters in brackets indicate which staff group delivers the activity, Doc = Doctors, Mid = Midwives, and Rec = Reception staff.
Each post-term pregnant woman in the model is represented as an agent from the Agent Based Modelling/Simulation (ABM/ABS) paradigm. A woman agent (WA) will be referred to as woman (or women where appropriate) in this paper to differentiate between computer simulated and actual women. In addition to post-term women, the model also simulates women belonging to other clinics, which for the present analysis only affect the flow of women by creating queues at check in and check out and competing for waiting resource, in the waiting area and corridors. Overall, the model is more detailed than indicated by Figure 2, and includes e.g. a representation of the physical layout of the clinic, and even chairs in the waiting areas. For a more detailed model description see [45] and the Appendix.
Each woman contains variables relating to its demographic and clinical characteristics, in additional to variable relating to its foetus. It was anticipated that these variables could be used to make more informed decisions, but data access and quality prevents the use of these variables in this paper. Each woman records information about its interaction with the DES clinic in relation to the activities depicted in Figure 2, including the waiting time for and the duration of each activity. A woman’s length of stay (LoS) is the difference between arrival at and departure from the clinic which is equivalent to the sum of the woman’s recorded time intervals. The DES clinic model collects utilisation results for resources described in Figure 2: reception staff, midwives, doctors, CTGs and waiting capacities. The model results collected at the women and the DES clinic model levels are collated at the end of each run of the simulation model and exported as csv files for further analysis.
Configuration
A model configuration relates to the six parts of the model that can be changed to assess clinic performance, with respect to a specific demand. The input values for each configuration are provided in the inputs section but this section introduces them briefly. A configuration consists of the type of arrival pattern the model uses to determine when the specified demand. As mentioned the clinic does use an appointment system but the women often arrive when the clinic opens. Two simplified appointment systems are considered in this paper. Changes to the pathway discussed in the model specification and are illustrated in Figure 2. The pathways considered in this paper are those illustrated in Figure 2, but alternative pathways could also be analysed with minimal effort. The other components of a configuration are the number of clinical staff, midwives and doctors and the number of CTG machines.
Data Sources
To represent the clinic accurately in a simulation model, data relating to clinic processes and activities are required, specifically the pathways currently in operation, feasible alternative pathways, and probability distributions that represent the service times of the different processes in the model.
The electronic patient registry and the hospital information system do not collect time stamped data; we could use to derive duration times at the activity level required by the model for outpatient clinics. The hospital information system is geared towards the collection of inpatient data, the only time data we have for outpatient clinics are appointment times which are recorded consistently and as we have alluded to women tend to arrive when or before the clinic opens <=08:00.
We asked staff to provide their best estimates for the durations (service times) for various activities. They were in the best place to provide these as they serve this patient group daily and it was the only feasible way to get service time data for the model. Neither the clinic nor the research team have the resources at the time to conduct a time in motion study, whereby we would follow patients around the clinic recording their process times or devise a method to collect this information.
We therefore developed an Excel VBA program based on the method described by Leal et al.[58], to elicit estimates from employees. The tool was not used to record actual service times. In Figure 2 the bracketed abbreviations indicate which estimates were provided by reception staff (Rec), midwives (Mid) and doctors (Doc). Five obstetricians, six midwives and five reception staff took part in the estimation exercise. Service time probability distributions created for: Check in, Connect CTG, CTG scan, Disconnect CTG, Consultation, and Check out, based on the input from the staff, these staff derived service time distributions were combined and validated through discussion with clinic management and the clinical authors of this paper. Descriptive statistics of the service time distributions used in the model and the number of CTG checks are provided in table 1.
Table 1 Summary statistics of service time distributions (minutes) and number of CTG checks
A more detailed description of how the service time distributions were derived is provided in the Appendix. No waiting time estimates were elicited from clinic staff, as these are the results of queues in the system, and estimated by the simulation model.
Assumptions
The model assumptions listed below, are made to reduce the scope of the system to its key factors.
- The clinic’s midwives and doctors did not attend to other non-clinic women during clinic hours.
- The reception staff and waiting area resources (waiting area chairs and corridor space) are shared with women attending all clinics.
- The midwives are able to attend to multiple women.
- A midwife will stay with a woman who requires 5 or more CTG checks, (mean number checks is 2.120). A summary of the CTG distribution is provided in Table 1 and the distribution in the Appendix.
- If a woman sees a doctor before a midwife, the woman does not have to see a doctor again after the midwife consultation. It was assumed that the doctor has sufficient information to complete the consultation.
- The clinic will stay open if women still need to use it. Results are collected within and outside clinic hours. In reality staff would have other duties to attend to. The simulation model runs until midnight to treat as many women as possible. Configurations that fail to satisfy the demand in clinic hours are rejected for that level of demand.
- All clinic appointments are attended. This places greater stress on each configuration, as in reality women may miss their appointment as they have given birth prior to it. Therefore all of the results presented are based on the maximum number of women each configuration can serve within clinic opening hours.
- The model does not differentiate between new and returning patients. New patients are those referred to the clinic for their first appointment for the current pregnancy. Returning patients are those attending their second, third etc. appointment for current pregnancy. The service time estimates provided by staff are for all clinic appointments.
- Two simplistic appointment systems are assessed in this paper. That either all of the demand arrives when the clinic opens at 08:00 or half of the demand arrives at 08:00 and half at 08:30. More nuanced appointment systems can be evaluated, but in general if the women arrive they tend to arrive before the clinic opens.
- The queuing discipline currently employed in the model is first in first out (FIFO).
- The model of the clinic operates a single queue multiple server system. Women are not assigned to particular midwives, CTG machines or doctors.
- The model represents the clinic in isolation, and does not take into consideration the occupancy of the maternity ward or operating theatres with respect to induction decision. This is outside the scope of this model.
Validation
Since detailed time stamp data is not collected for the clinic, we could not validate the model against historical data. As a substitute for this, we validated the model through visual demonstrations of the model during the construction process, and the clinic staff concluded that the model had a high degree of face validity. We performed extreme value tests, including: 1) very large/small arrival numbers at the clinic, checking queues formed in the model where appropriate, 2) excessive/insufficient amounts of resources (e.g. doctors etc.) and checked if the model outcomes were as expected. Additionally, we varied the activity services distributions, between (un)favourable alternatives and verified this had the expected effects on a woman’s visit. Finally, we validated the women’s average LoS against the LoS estimate provided by the midwives. For the baseline configuration (configuration 0a, Table 1) the model output matched the estimates well. This suggests that the sum of the waiting times (which are based on the simulated queues in the system, rather than pre-specified times) match the real waiting times well.
Experimental Design
Time Horizon
The model can run for any time horizon. In this paper it runs, in minutes, for a single clinic day, from 08:00 to 23:59. Utilisation results for resources (e.g. midwives etc.) are recorded during clinic hours 08:00 to 10:30. It runs until 23:59 to allow all women who attend the clinic to be seen, the clinic may not be able to satisfy the demand if it is poorly configured or if demand is too great. The simulation starts with an empty clinic, which reflects the reality, since there are no overnight stays. A warm-up period for the model was also not considered necessary
Multiple Runs
Due to the randomness in a single model run, we collected statistics from 10,000 runs (model trial) for each model configuration. Randomness in this model relates to the service times. It is necessary to run the model multiple times for each configuration to avoid basing decisions on a potentially unrepresentative single run of the model.
Inputs
Through discussion with clinic staff, six inputs (decision variables) were defined. A clinic configuration is the values these inputs take. The inputs and the values they can take are described in Table 2.
Table 2 Model inputs
A full factorial experimental design for the inputs described above results in 608 configurations. Each of these configurations was evaluated by a model trial.
Key Performance Indicators
The configurations are evaluated using seven model KPIs. The average KPIs derived from a model trial per configuration are stated in Table 3. Waiting time KPIs are collected to indicate where potential bottlenecks may exist.
Table 3 Model KPIs – Average results from 10,000 runs of the model
Optimal Treatment Capacity
The six model decision variables (X0, X1, X2, X3, X4, and X5) described generated 608 configurations. The arrivals input (X0) ranges from 2 to 20 women (19 arrival patterns). Optimisation was performed on the configurations where X1, X2, X3, X4 and X5 were set and X0 incremented. This results in 32 configurations (608/19). This is illustrated in Table 4.
Table 4 Optimal treatment capacity
The goal of the optimisation was to maximise the number of women (X0/Y1) that could be seen subject to the average time the last woman checks out (Y2) ≤10:30 and minimising the average clinic overrun time (Y4). This provided the optimal number of women each of the 32 configurations could serve. The ability to vary the model demand enables the effects of increases that may arise due to changes in the catchment areas or changes to guidelines to be evaluated.
The simulation model output is recorded in a number or text files for each model trial for each configuration. This large number of files is then analysed post hoc using R to produce outputs and enable the team to examine individual patient responses in a particular run in a particular trial if necessary. More information about how AnyLogic interacts with R is provided in the Appendix.