Immersive narratives consume more attentional resources
Experiment 1 explored the relationship between dual-task reaction time and immersion. Dual-task reaction times are a classic and extremely well established measure of attention within psychology26. In this paradigm, participants complete a primary task (watching a film) alongside a simple secondary task (e.g. responding to an auditory tone). Reaction time to the secondary task is taken to indicate the available cognitive resources for that task. Given a finite amount of cognitive resources, any reduction in cognitive resources to the secondary task suggest that more resources are being allocated to the primary task 27 (see 28 for a review). As such the reaction time gives a measure moment-by-moment of how much attention is being allocated to the film.
All data were analysed using R 4.1.129 and are made available at https://osf.io/. Mean reaction times were M = 999 ms, SD = 427 ms with M = 92.7%, SD = 26.9% correct responses (see Supplementary Fig. 2 for an overall distribution of correct and incorrect responses). In subsequent analyses, responses over 3000 ms and incorrect responses were excluded, as in 9. While reaction times were not normally distributed (Supplementary Fig. 2), we did not transform the data as analyses were conducted on mean data for each clip, which following central limit theorem will conform to a normal distribution. Mean narrative engagement scores were M = 4.27, SD = 0.94. For each dimension, this is: attentional focus (M = 4.37, SD = 1.75), emotional engagement (M = 3.58, SD = 1.61), narrative presence (M = 3.90, SD = 0.73), narrative understanding (M = 5.21, SD = 1.44). Mean single-item immersion rating was M = 4.41, SD = 1.83.
Figure 1 shows the overall correlation between reaction time and narrative engagement. Reaction times were strongly positively correlated with narrative engagement: that is, more engaged participants are slower to response to the secondary task (r(5) = .88, p = .01). The Narrative Engagement Scale is a 12-item scale assessing 4 dimensions of engagement: attentional focus, emotional engagement, narrative presence, and narrative understanding. Participants rated each item on a 7-point Likert scale. When referring to narrative engagement, we describe the mean of all 12 items. When referring to a dimension of narrative engagement, we describe the mean of the subscale items which assess that dimension. In addition to the Narrative Engagement Scale, we asked a further single-item question to compare to the full scale (‘During the program, I was very immersed’), which was rated on the same 7-point Likert scale as the Narrative Engagement Scale. This allows us to investigate whether it is appropriate to reduce self-reported engagement to a single dimension. We refer to this single-item question as immersion. Immersion was also significantly correlated with reaction times: r(5) = .83, p = .02.
To assess whether this correlation was robust within participants, we computed the correlation between reaction time and narrative engagement for each participant and compared this distribution against zero. This approach accounts for individual differences in preference, as the test makes no assumptions about which content participants may rate as most engaging. Using a one sample, two-tailed t-test, we found that this overall distribution was significantly greater than zero: mean r = .218, t(163) = 6.82, p = 1.69 x 10-10. Similarly, participants’ individual correlation between reaction times and self-reported immersion was significantly greater than zero: mean r = .185, t(163) = 5.85, p = 2.58 x 10-8.
To assess which dimensions of the Narrative Engagement Scale were influencing reaction time, we fitted a linear mixed model to our data using the ‘lme4’ package in R30. We included fixed effects for each dimension of narrative engagement (attentional focus, emotional engagement, narrative presence, narrative understanding). We also included a fixed effects for single-item immersion, clip order (given the tendency for reaction times to increase over time9), and for familiarity (whether participants had seen the clip before). Participant was set as a random intercept to account for participant level differences in average reaction time.
As shown in Figure 2, emotional engagement led to a significant increase in reaction time: b = 15.06, 95% CI[5.89, 24.23]. Clip order also significantly increased reaction time: b = 12.79, 95% CI[8.09, 17.49]. Narrative presence (b = 7.23, 95% CI[-11.85, 26.31]), narrative understanding (b = -4.20, 95% CI[-12.42, 4.02]) and familiarity (b = 10.24, 95% CI[-5.10, 25.59]) did not significantly affect reaction time. Interestingly, despite the dual-task paradigm being a measure of attention, attentional focus also did not affect reaction times (b = -3.95, 95% CI[-15.08, 7.18]). We can conclude therefore that increases in reaction time are predominantly driven by emotional engagement in the story.
Synchronous viewer behaviour has been presented as a major consequence of media viewing, and can be seen in neural32, behavioural24, and physiological data33. Here, we investigate whether this synchronous behaviour may be driven by engagement. The are several ways to assess viewer synchrony – for example, inter-subject correlation34,35, cross recurrence quantification analysis36, and circular shuffle statistics37. Here, we rely on the most widely used method, inter-subject correlation, to assess whether synchrony in reaction time (ISCRT) was related to narrative engagement. For each clip, a correlation matrix between all pairs of participants is produced. We then take an average of each row of that matrix (each participant), which provides a score for each participant of how synchronous they are with all other participants. We find that ISCRT was not significantly related to narrative engagement (r(5) = .554, p = .197) or immersion (r(5) = .490, p = .264). However, individual participant’s ISCRT was significantly related to narrative engagement (mean r = .216, t(162) = 6.73, p = 2.84 x 10-10) and immersion (mean r = .182, t(162) = 5.75, p = 4.26 x 10-08).
Immersive narratives synchronise heart rate
Experiment 2 explored the relationship between heart rate, skin conductance, and narrative engagement in 10 video clips. For each participant, physiological measures were averaged over 1 s intervals and we calculated difference scores by subtracting the grand mean for each participant from their scores in each clip. This standardises our responses across participants. In Experiment 2, mean heart rate was M = 75.17, SD = 11.43. Mean narrative engagement scores were (M = 4.63, SD = 1.01), and broken into constituent dimensions this was: attentional focus (M = 4.80, SD = 1.74), emotional engagement (M = 4.17, SD = 1.75), narrative presence (M = 4.17, SD = 0.73), narrative understanding (M = 5.39, SD = 1.61). For single-item immersion, mean scores were M = 4.92, SD = 1.64.
First, we investigated the relationship between heart rate, heart rate variability, and narrative engagement. Heart rate is a way to index parasympathetic and sympathetic nervous system activity38. Heart rate is known to vary in response to cognitive processing demands, and as such has been applied within media psychology to index immersion18. Due to dual-innervation of heart rate by sympathetic and parasympathetic activity, heart rate does not always vary predictably in response to stimuli, and as such heart rate variability is sometimes used as a measure of cognitive processing28. Neither heart rate (r(8) = -.501, p = .140) nor heart rate variability (r(8) = -.524, p = .120) were significantly correlated with narrative engagement. Individual participant’s heart rate and engagement correlations were significantly lower than zero for engagement (mean r = -.123, t(46) = -2.61, p = .012) but not immersion (mean r = -.110, t(46) = -1.95, p = .057), offering some indication that higher self-reported engagement may be associated with lower heart rate. Individual correlations between heart rate variability and engagement (mean r = -.093, t(46) = -1.86, p = .069) or immersion (mean r = -.070, t(46) = -1.33, p = .192) were non-significant.
We find strong evidence that synchronicity in heart rate is a predictor of engagement. Figure 3 (top) plots the significant relationship between heart rate inter-subject correlation (ISCHR) and narrative engagement: r(8) = .847, p = .002. A similar relationship was found between ISCHR and immersion: r(8) = .717, p = .020. ISCHR is defined in the same way as ISCRT above.
This relationship between heart rate synchrony and engagement was also robust across participants. Again, we computed correlations between ISCHR and engagement for each participant (Fig. 3: Bottom left). Individual participants show significant non-zero relationships between ISCHR and narrative engagement (mean r = .129, t(45) = 7.38, p = 2.79 x 10-9) as well as between ISCHR and immersion (mean r = .110, t(45) = 6.54, p = 4.95 x 10-8).
As with reaction times above, we then built a linear mixed model, including fixed effects for each dimension of narrative engagement, single-item immersion, clip order, familiarity, and a random intercept for participant. Figure 4 plots these regression estimates.
As shown, attentional focus (b = .009, 95% CI[.005, .013]), emotional engagement (b = .010, 95% CI[.008, .014]), clip order (b = .005, 95% CI[.003, .006]), and familiarity (b = .010, 95% CI[.004, .012]) significantly increase HRISC, while narrative presence (b = -.007, 95% CI [-.014, .000]), narrative understanding (b = .001, 95% CI[-.000, .004]) and immersion (b = -.002, 95% CI [-.006, .002]) do not significantly affect ISCHR. Higher narrative engagement drives synchrony between participants heart rate then, and this is predominantly related to attentional and emotional engagement with the narrative. Supplementary Figure 5 provides an example of how this inter-subject correlation technique could be used dynamically within a piece of content and related to narrative moments.
Skin conductance is not related to immersion
Skin conductance is one of the few physiological measurements singly innervated by the sympathetic nervous system, and is considered a measure of arousal39. Skin conductance levels are known to vary with the emotional content of a stimulus, and have been used as a proxy for engagement in several media studies19,40,41. Mean skin conductance level for Experiment 2 was M = 10.60, SD = 6.08. We did not find a significant relationship between skin conductance level and narrative engagement (r(8) = .020, p = .957), or immersion (r(8) = -.143, p = .693). Individual participant’s correlations between skin conductance and narrative engagement (mean r = -.029, t(46) = -0.58, p = .57) or immersion (mean r = .005, t(46) = 0.11, p = .912) were also non-significant. Additionally, we did not find evidence of a relationship between inter-subject correlation of skin conductance (ISCSC) and narrative engagement (r(8) = -.140, p = .699) or immersion (r(8) = -.295, p = .408). This relationship was also not significant at an individual participant level for engagement (mean r = -.010, t(46) = -0.50, p = .617) or immersion (mean r = -.010, t(46) = -0.69, p = .492).
Familiarity
Supplementary figure 4 presents a breakdown of participant’s familiarity scores in each experiment. Familiarity with the content affected narrative engagement, but not reaction time, heart rate, or skin conductance. To assess if familiarity affected our measures, we used Welch’s two sample, two-tailed t-tests to account for the unequal sample size and variance between familiar and unfamiliar groups. In experiment 1, participants who had seen any of the series before were more engaged (M unfamiliar = 4.04, M familiar = 4.63, t(1006) = -11.06, p < .001) but did not show differences in reaction times (M unfamiliar = 985, M familiar = 989, t(939) = -0.22, p = .82). Similarly, participants who had seen the specific clip before were more engaged (M unfamiliar = 4.14, M familiar = 4.9, t(264) = -11.62, p < .001) but did not show differences in reaction times (M unfamiliar = 982, M familiar = 1019, t(226) = -1.45, p = .15).
In experiment 2, familiarity was related to higher engagement both when participants had seen any of the series before (M unfamiliar = 4.52, M familiar = 5.12, t(101) = -5.49, p = 2.99 x 10-7) and when they had seen the specific clip before (M unfamiliar = 4.52, M familiar = 5.25, t(74) = -6.02, p = 6.23 x 10-8). Heart rate was not affected by familiarity with the series before (M unfamiliar = 75.23, M familiar = 75.58, t(86) = -0.27, p = .791) or the specific clip (M unfamiliar = 75.19, M familiar = 75.88, t(63) = -0.44, p = .664). Similarly, skin conductance was not affected by participants seeing any of the series before (M unfamiliar = 10.57, M familiar = 10.46, t(88) = 0.15, p = .880) or the specific clip before (M unfamiliar = 10.62, M familiar = 10.17, t(74) = 0.65, p = .516). From this we can conclude that while participants found familiar content more engaging, this did not affect reaction times or physiological responses.
Single-item question of immersion indexes the full narrative engagement scale
Finally, we looked to assess whether our single-item question of immersion was related to overall narrative engagement. In experiment 1, there was a significant correlation between immersion and the Narrative Engagement Scale: r(1146) = .797, p = 2.2 x 10-16. To assess which dimensions immersion was related to, we fit a linear regression predicting immersion from each dimension of narrative engagement. Immersion was predicted by attentional focus (b = .61, p = 2 x 10-16), emotional engagement (b = .28, p = 2 x 10-16), and narrative presence (b = .50, p = 2 x 10-16), but not narrative understanding (b = .01, p = .77). The same pattern of results was present in experiment 2, immersion was significantly correlated with engagement: r(478) = .755, p = 2.2 x 10-16. Immersion was predicted by attentional focus (b = .579, p = 2 x 10-16), emotional engagement (b = .150, p = 5.88 x 10-6), narrative presence (b = .366, p = 4.96 x 10-7), but not narrative understanding (b = .014, p = .608). This offers the promising indication that most dimensions of engagement (excluding understanding) could be indexed reasonably accurately by a single-item questionnaire.