It has been argued that the most challenging roadblocks towards widespread acceptance of self-driving vehicles could be more social than technical in nature [1–4]. Perception of usefulness [5, 6], trust [7, 8], safety [9, 10] have all been identified as important factors that influence acceptance. Investigating the perceptual factors of technology acceptance, however, has its own challenges. Perception is rather an umbrella term, as it may refer to the opinion of the user, feelings, ideas, attitudes colored by personality traits, previous experiences, desires etc. [11, 12]. These factors to this day are commonly measured using questionnaires or interviews [13, 14]. However, from a more biological or ecological perspective perception refers to awareness of factors of the environment that have significance with respect to our goals, actions, and wellbeing. Most importantly, perceptual processes involve the visual and auditory system, the central nervous system (CNS) and they are closely coupled to cardiovascular, respiratory, and motor systems. This strongly suggests that an effective research method investigating factors of perception and acceptance of AV technologies must include physiological measurements.
In a previous pilot study, electroencephalography (EEG) and eye-movement data were collected in a 3-minute-long real-world driving situation [15]. In the analyses, both measurement types signaled a difference between traditional and self-driving: participants’ biological signals such as frontal alpha asymmetry and multifractal spectrum suggested higher preference for human driver in contrast to the autonomous driving mode. However, even for a road trip this short, the experience is not homogeneous. During a road trip passengers experienced regular or expected events such as acceleration, deceleration, changing lanes, following the curvature of the pavement. Irregular or less expected events may also occur when sudden deceleration or acceleration or quick path alterations are necessary to avoid collisions. A passenger, paying attention to the movement of the vehicle has expectations about the dynamics of these events. The extent to which these expectations are being met could be a factor in the perception of safety and usefulness of any given mode of transportation.
Our aim in the current investigation is to test human driven and self-driving scenarios with the addition of unexpected events to passenger experience. Life size dummies made of plastic and other relatively collision-safe materials were placed on the side of the road prompting quick evasion maneuvers in both human driven and self-driving conditions. We expected these events to either override the effect of driving conditions or interact with them in ways they allow us to draw conclusions about the strength of our manipulations on the physiology of the passengers. If measured differences between self-driving and human driving conditions gets moderated or eliminated by the unexpected events, it may signal an important message to developers and users alike: firsthand experiences of safety and the ability to handle critical situations on the road may be an important factor in technology acceptance. In the following, we enumerate the main characteristics of the measurements we utilized during the data collection as well as the most important results from the literature, and articulate our hypotheses.
EEG
In the past decade, a rapid growing is observable in the availability of portable sensors making it possible to design ecologically valid field studies to measure biological signals such as eye movements or the electric activity of the brain [16]. EEG signal consists of different frequency ranges, and their ratio allows one to draw conclusions about changes in one’s mental and emotional state over the recording [17].
Numerous studies demonstrated that higher frequency oscillations reflect more alerted and aroused states when the power ratio of higher (beta: 13–30 Hz, gamma: 30–45 Hz) and lower frequencies (alpha: 8–12 Hz, theta: 4–8 Hz) are compared [18, 19]. In contrast, lower frequencies dominate when participant is relaxed. For example, concentrated attention was correlated with higher relative gamma activity [19], and enhanced stress level was reflected by increased beta/alpha [19, 20], decreased alpha/beta and theta/beta [21] or increased relative gamma ratio [22].
In addition to arousal, a further well-studied area of EEG band signatures is to follow-up emotional and motivational states. Based on the hemispherical (left-minus-right) differences in the alpha power at frontal electrodes (frontal alpha asymmetry: [23, 24], higher values indicate more positive or approaching attitude while lower values indicate more negative or withdrawal attitudes [24–26]. Frontal alpha asymmetry was found not only to be a good primer of depression [24] but also indicative tool in applied sciences [27].
Both arousal and frontal alpha asymmetry were used in studies investigating drivers’ or passengers’ reactions to unexpected road events, mostly in simulated tasks. For example, when the driver detected a hazard cue, the power of the alpha band immediately decreased, and the beta power increased approximately 300 ms after the cue appeared [28]. Similarly, when a driver could not avoid collision, higher frequencies dominated in comparison to successful collision avoidance [29]. Specifically focusing on self-driving vehicles, in a case study [30], participants were presented with positive (smooth highway driving) and negative (erratic driving and violating common rules of the road) driving situations. The beta-to-alpha power ratio increased in the negative scenario, suggesting elevated stress level when being exposed to hazardous driving. Similarly, the most effective warning signals in an autonomous vehicle simulator were predominantly accompanied with the presence of higher frequencies [19].
Regarding emotional valence [31], when malfunctions appeared during a simulated drive in an autonomous vehicle, frontal alpha power reduction was present in the right but not in the left hemisphere during the fully automated condition in comparison when participants were able to control the vehicle. The authors interpreted this effect as an enhanced motivation of the driver towards controlling the vehicle which was in line with participants explicitly verbalized preferences [31]. In a further study utilizing narrow vehicles, higher arousal and higher frontal alpha asymmetry values were measured when the vehicle was able to tilt in curved sections of the road which in line with subjective evaluations reflecting user satisfaction [32]. Comparable results were found when instead of a traffic situation, wheelchair users were sitting in an autonomous wheelchair driving across a narrow/constrained or a wide/open path. Narrow paths resulted in frontal alpha asymmetry pattern related to avoidance during these hazardous situations [33].
Eye movements and head movements
Recently more focused analyses targeted eye and head movements in the context of developing driver assistance systems and self-driving navigation technologies. Drivers and passengers are surrounded by a highly dynamic, rapidly changing environment in which the visual information that is relevant for navigation must be sampled in specific ways often including a wide variety of eye and head movements [34]. It has been pointed out that despite being linked to distinct neuromuscular systems, eye and head movements are highly correlated [35] and are a source of a wide range of information about the driver or the passenger. In a wider context, constant fluctuations in movements of the eyes, head or hands could be harnessed as an information “substrate” spreading across the perceptual/motor system linked to coordination and cognition [36]. In this line of research, multifractal analysis of continuous displacement data has been repeatedly shown to signal cognitive transitions and processes including problem solving [37], magnitude perception [38], perceptual intent [39], visual recognition [40], comprehension [41], and memory [42]. Focusing on fluctuations and displacement data also facilitates non-invasive, in vivo research methods that allow for experimentation in close to real life circumstances. Besides complex movement patterns, relatively simple changes in spontaneous blink frequency (SBF) have been reported to show correlation with anxiety and novel stimulation [43, 44].
Hypotheses
The goal of the present study was to investigate the fluctuation changes in biological signals of a passenger when unexpected road events appear, and the potential differences between situations when a human person drives the car (Human condition) versus when the vehicle requires no manual input from a human driver to complete the track (Self-driving condition). For the EEG measurement, we hypothesized lower frontal alpha asymmetry values in the self-driving condition. We also hypothesized higher arousal and lower frontal alpha asymmetry values to unexpected road events. For the eye-, and headtracking data, we expected to see the effect of novelty and anxiety manifest in narrower multifractal spectrum for the self-driving conditions countered by the contribution of unexpected events where heightened need for visual information may result in wider spectra. For the same reason, blinking frequencies were expected to show lower values for the novel and unexpected stimulation.