Our analysis showed that the NAS overestimates the needed nursing time for patients in the Dutch ICU setting. Times of most NAS items were overestimated by the NAS, except for four activities (support or care for patient for about 1 hours, administrative tasks for less than 2 hours, administrative tasks for about 2 hours, and specific interventions outside the ICU), where the NAS gives an underestimation of the observed time. This study showed that 35% of nursing time is explained by the NAS model (R2 = 0.35). The converted NAS time per patient (202.6 minutes per shift) in our study was comparable with the converted NAS times per patient in other studies. Bernet et al. (2005)16 found 150 to 156 minutes per shift and Deberg et al. (2007)17 found 180 to 228 minutes per shift. The different articles on the NAS give variable NAS times per shift. A full shift of work equals 480 minutes of nursing time.
The low correlation of Pearon’s R and R2 (0.59 and 0.35) implicates that the NAS is not accurate enough to estimate the nursing time at patient level. However, it is currently still the best nusing workload model for quantifying nursing workload in ICUs 5. There is no clear cut-off point from which the model can be identified as ‘good enough’ based on the R2. However, since the NAS is used for capacity planning, a R2 closer to 1 would be more desirable.
Since in almost each shift ICU nurses also spend time on non-nursing duties, e.g. coaching a student or participating in an emergency team within the hospital, we performed a sensitivity analysis to determine whether these non-nursing duties were affecting the correlation. According to several studies nurses spend approximately 3 to 6% of their shift on non-nursing duties 18,19,20,21,22. We therefore took the average of 4.5% and substracted this from the 80% of productive nursing time, which we used in this study to calculate the converted time per NAS point. Using this approximation, the converted time would have changed from 3.84 to 3.62 minutes per NAS point. This change does not affect the results and we therefore conclude that non-nursing duties are not significantly influencing the performance of the NAS.
A strength of our study is that we validated the NAS with time-and-motion measurements which are considered to be the best technique for measuring nursing workload 12. To our knowledge this has not been performed before in the context of NAS validation. Measurements for nursing activities by using time-and-motion measurements, are more accurate compared to the work-sampling approach as used for the development of the NAS 23. Furthermore, since measurements took place in all types of ICUs, we believe that results of this study are generalizable to all Dutch ICUs.
One of the limitations in our study are the excluded NAS activities due to their non- or limited occurrence of less than ten times. Two of these activities are mostly scored in other categories of activities: the activity ‘intravenous replacement of large fluid losses’ is mostly scored under NAS item 1 ‘bedside’. The activity ‘treatment of complicated metabolic acidosis/alkalosis’ is mostly scored in NAS item 3 ‘medication’. Since these activities could be scored in other categories, these activities can be excluded from the NAS. Three NAS activities (respectively left atrium monitoring, cardiopulmonary resuscitation after arrest, and specific interventions in the ICU) and six subcategories 1b, 1c, 4b, 4c, 7b, and 8c (the nurse activities that required dedication from the nurse for more than 2, 3 or 4 hours) did not happen often enough (>=10 times) during the measurements which makes the validation of the NAS incomplete. Given the fact that the median time of nursing care per patient is 2.4 hours (144.3 minutes), dedication of a nurse for more than 2, 3 or 4 hours to one activity is extremely high. As these nursing activities rarely occur in daily ICU practice it is not likely that our results have been affected by this situation.
Furthermore, the observed patients seem to have been more severely ill and consequently had a longer length of stay compared to all Dutch patients in the same time period, which is likely caused by our selection mechanism. In order to measure as many nursing activities as possible we probably choose more often nurses who took care of patients that were expected to stay the whole shift and these patients were probably more severely ill. This may have biased our results since our aim was to validate the NAS and check for under- or overestimations compared to time-and-motion measurements and it is possible that observed times in sicker patients differ from those in less sick patients. However, according to Armstrong et al. (2015) NAS scores in intermediate care patients did not differ from those in ICU patients 24.
Based on our results we believe there is room for improvement in the measurement of nursing workload. The NAS could be improved by adjusting the NAS points given to the different items. The developers of the NAS did not report the Pearsons R or R2, but stated that the NAS is reflecting 81% of total nursing time. About 11% of the nurses’ time is spent on personal activities. The remaining 8% comes from nursing activities derived from medical interventions, related exclusively to the severity of illness of the patient not measured by the NAS 7. The TISS is taking these medical interventions into account, such as induced hypothermia, cardiac assist device, pacemaker, or ECG monitoring. For this reason, we suggest additional research towards the merging of the TISS-28 and the NAS. The models could be partly combined which could possibly improve the estimation of nursing workload. Our results on observed time per patient and per nursing activity could be taken into consideration when assigning weights to the activities in this new model. Moreover, we think that expressing nursing activities in minutes or hours would be more informative compared to points, since it is more straight forward for ICU managers to work with.