Reliability and validity of a virtual reality-based measurement of simple reaction time: a cross-sectional study

doi:10.21203/rs.3.rs-2981817/v1

Background

Reaction time (RT) is an important dependent variable to assess components of cognitive function. Thus, it has been a valuable parameter for research and clinical evaluation. A head-mounted display for virtual reality (HMD-VR) provides a standardized external visual condition and could be a promising tool for measuring RT. The purpose of this study is to establish the feasibility, reliability, and validity of HMD-VR-based software in evaluating simple RT (SRT).

Methods

Thirty healthy participants volunteered for the study. A SRT test was created by VIVE ProEye (HTC, Inc.). The participants responded with a keyboard when a square target was used in random intervals for 100 trials. To determine the optimal test repetition, the difference between the SRTS calculated with different trial numbers was analyzed. The one-week reliability of the median SRT was evaluated with the intraclass correlation coefficient (ICC). Finally, the convergent validity was tested by computing the correlation coefficient with a personal computer-based (PC-based) software, RehaCom^Ò(HASOMED, Inc.) with a similar task design.

Results

The median SRTs of the virtual reality-based (VR-based) and computer-based systems were 326.0 and 319.5 ms, respectively. Significantly longer RT obtained by the VR-based method was observed in the last 25-trial block for the non-dominant hand and bilateral hands according to Friedman’s test. The ICC was 0.71 (p<0.001), indicating good test-retest reliability. There was a high correlation (r=0.85~0.89) and good agreement between the VR-based and PC-based tests, with the VR-based SRT being 9-10 ms longer than the PC-based SRT according to Bland–Altman plots.

Conclusions

Our results supported the good reliability and high convergent validity of this HMD-VR-based RT testing. A test length of 50 trials was suggested to avoid possible decremental performance while maintaining good reliability. The program can be applied in future studies when spatial-specific RT is the main interest to provide a standardized external environment.

Attention

reliability

simple reaction time

virtual reality

Response time or reaction time (RT), widely used in mental chronometry, is the amount of time that occurs between perceiving and responding [1, 2]. It reflects the ability to detect, process, and respond to a stimulus and has a significant correlation with general mental ability in the field of cognitive neuroscience [2, 3]. Therefore, the measure can be used to identify functional deficits at various stages of recognition, processing, and execution, which may involve multiple systems, such as the auditory-cognitive, attentional, working-memory, visuocognitive, visuomotor, and executive systems [4]. It can serve as a simple yet quantitative tool for clinical evaluation and research in various populations, including healthy children, adults, elderly individuals, and individuals with cognitive impairment [5–9]. For example, SRT has been used for evaluating cognitive processes and executive function in children [9]. SRT is related to cognitive function in both healthy subjects and hospitalized patients and inversely related to the physical component summary of quality of life [10]. In older people without dementia, greater SRT variability is associated with poorer executive function and greater double support time variability, corresponding to memory decline and falls [11].

Despite the important role of RT in neuropsychologic evaluation, the measurements are highly variable. RT can be measured in several forms, such as simple reaction time (SRT) or choice RT, or various measurement methods, such as computer-based software or the ruler-drop test [12]. Computer-based measurements have been a mainstay of measurement since the 1970s [13], but the measurement results are influenced by many methodological factors [14–16]. The tests are highly dependent on the exact visual assessment conditions, not only the size, duration, intensity, and color of the stimulus but also the room lighting specific monitor, head position, and target distance relative to the tested individuals [6, 15, 17–22]. In some studies, precise control of target positions in the visual field is needed, and researchers have to design a hemispheric dome with an external light source or add an external apparatus to preclude extraneous light [23, 24]. Standardized and controlled testing conditions with these delicate setups are essential for evaluating RT and further analysis, for example, interhemispheric transfer time. In the meantime, the computer video display units may not be able to provide a well-controlled testing environment, such as a symmetric luminance computer screen [25].

As the technology evolves, virtual reality-based (VR-based) visuospatial measurements have been proposed for evaluating the assessment of variables of visuospatial cognitive function, such as visual processing, visuospatial neglect, attention, and response time [26–31]. Head-mounted displays for virtual reality (HMD-VR) has the potential to be a reliable evaluation tool for visual processing since they provide standardized external visual assessment conditions, for example, controlled room lighting and the target position within a wide field-of-view 3-dimensional space [26, 32]. In the meantime, previous studies also raise concerns about the psychomotor response in virtual environments created by HMD-VR, despite the high fidelity and immersion [33–35]. Visual perception in the real world and virtual environments has a fundamental difference because of the technological design of VR. The current technology of HMD-VR is capable of simulating depth that resembles the spatial properties of the real world but does not exactly replicate how humans see and perceive depth under natural viewing conditions [35]. For example, kinematic evaluation of the upper limbs shows slower movements with longer deceleration times in virtual environments than in the physical world [33]. Provocative visual stimulation, namely, immersion into virtual reality, may induce cybersickness, with the presentation of pronounced nausea and elevated heart rate, and prolong the SRT [36]. Therefore, the VR-based measurement is expected to be a promising tool but experiments are needed prior to possible application, such as RT.

The evaluation of psychometric properties (reliability, validity, and sensitivity) is an important phase of the lifecycle of computer-based cognitive assessment tests [37]. Despite a growing number of studies using VR for the assessment of RT and attention [38–41], there have been very limited published studies on the psychometric properties of these VR-based measurements.

We designed VR-based software to test SRTs in a process similar to Deary-Liewald tasks [42]. The goals of our study were to evaluate the test-retest reliability and the convergent validity of HMD-VR-based software in evaluating SRT with commercial personal computer-based (PC-based) software. Moreover, we would like to determine the optimal trial numbers regarding the reliability and potential fatigability.

Study design and participants

This was a cross-sectional study. Thirty healthy adults 20 years of age or older were enrolled through convenience sampling from the study institute (Table 1). The exclusion criteria were a history of neurological conditions, vestibular dysfunction, vertigo, psychiatric conditions, significant visual problems (such as visual loss, cataract, glaucoma, or other major eye conditions), upper limb musculoskeletal disorders causing difficulty using a keyboard or controller, and an inability to tolerate HMD-VR. They were asked not to consume alcohol, nicotine, or hypnotics on the day of testing. All participants agreed to undergo retesting. This study was approved by the ethical committee of the National Taiwan University Hospital (approval number: 202104030RINC, date of approval: 05/13/2021), and written informed consent was obtained prior to participation. All procedures were carried out in accordance with the ethical rules and the principles of the Declaration of Helsinki.

VR- and PC-based SRT evaluation

The concurrent validity was tested against the PC-based software Alertness (ALET) module in RehaCom software (version 6.9.0.0, HASOMED GmbH, Magdeburg, Germany), which is used for training and evaluation in clinical settings [43]. The software was run on a desktop computer with Windows 10, Intel i5-8500 CPU, 8 GB RAM, and an Intel UHD Graphics 630 graphics card. The display was a 21-inch screen. The stimulus was a 50 x 50 mm red square and appeared in a fixed position on the center of the display on a light-green colored background. The interstimulus time intervals were random within 3 seconds, and no audible warning before the stimuli was provided. The participants responded to the stimuli by pressing a button on a special custom keyboard.

The VR-based SRT was tested using the Vive ProEYE system (HTC, Inc. Taiwan), which includes an HMD with eye tracking and two infrared laser emitter units (lighthouse 2.0). The HMD was connected to the measurement computer through a cable and Vive link box, which provided an HMDI connection, a USB connection, and a power supply. Two lighthouses were positioned diagonally and mounted at a height of 2.1 m, angled downward at an angle of 25–30°, and connected by a synchronization cable. A custom C# script and the SteamVR (Valve Corp, Washington, USA) plugin for Unity3D were used to provide integration with the HTC Vive to design the software for testing the SRT. The VR software operated on a laptop computer, Asus ProArt StudioBook Pro H500GV (Asus, Taiwan), equipped with Windows 10, Intel i7-8700, 16 GB DDR4 RAM, and an Nvidia GeForce RTX 2060 graphics card. We set SRT testing environments similar to those for the PC-based testing by building a virtual board as the screen of a Vive tracker, which was adjusted to a position of 65 cm in front of the user. The stimulus was also a 50 x 50 mm red square block presented in a fixed position on the virtual board, with random time intervals between 1000–3000 ms. The participants responded to the stimuli by pressing the “space” key on the keyboard of the laptop computer, and the values for all RTs to relevant stimuli were recorded in ms (Fig. 1).

Procedure

The procedure of our study design was shown in Fig. 2. The demographic data, including age, sex, and handedness, were obtained via a questionnaire. The participants were randomized to two groups in the order of the PC- and VR-based tests. This is, some participants underwent the PC-based tests prior to the VR-based one, and others encountered in an opposite order. There was a 10-minute break between the 2 testing modalities. For both PC- and VR-based tests, the participants were seated in a comfortable chair in a quiet room. To control the effect of the weight of the HMD, the participants wore the HMD on top of their heads without blocking their vision during the PC-based SRT test. The participants were asked to complete two tests, one responding with the right index finger and another with the left one, in each modality. A practice of 20 trials was provided prior to each formal test, with the instruction of responding to the stimuli as fast as possible without error and anticipation. Then, the participants completed two 100-trial tests with either the right or left hand, respectively, in a randomized order, similar to the protocols used in previous studies [14, 44]. There was a 2-minute break between each testing condition. They were allowed to stop the test if there was any discomfort. The total time to complete the VR-based test was computed from the recorded raw data. They were verbally asked about any symptoms of cybersickness, such as dizziness, nausea, eye strain, or headache, after the VR-based test. The participants repeated the VR-based SRT test within 1 week. The schedule of the retest was arranged in the morning or afternoon, as in the first test.

Statistical analysis

The PC-based SRT was automatically generated in ms by the RehaCom software as a median value. For the VR-based SRT, we excluded the data with RTs below 100 ms or above 1000 ms. We further divided the data of the 100-trials into 4 blocks, with each containing 25-trials, and computed the median values for the 100-trial and each block of 25-trial. The descriptive statistics of the median SRTs were checked for normality of the data distribution using the Shapiro–Wilk test. Parametric or nonparametric methods for statistical analysis were chosen according to the distribution.

The differences in the median SRTs of each 25-trial block of the VR-based SRTs were examined with Friedman’s test or repeated measures analysis of variance, and the p values for the post hoc examination among multiple comparisons of tasks were examined by the Wilcoxon signed rank test or paired t-test with Bonferroni correction. The test-retest reliability of VR-based SRT was assessed with the intra-class correlation coefficient (ICC) based on a single rating, absolute-agreement, 2-way mixed-effects model. An ICC higher than 0.75 was considered excellent, an ICC between 0.6 and 0.74 was considered good, an ICC between 0.4 and 0.59 was considered fair, and an ICC below 0.4 was considered poor [45]. The ICC was calculated for 25-trial, 50-trial, 75-trial, and 100-trial SRTs to assess the reliability of various trial numbers. The convergent validity was tested against the PC-based SRT obtained by RehaCom with Pearson’s correlation coefficient and Bland–Altman plot [46]. The size of the correlation coefficient was interpreted as being very high (0.90), high (0.7 to 0.9), moderate (0.5 to 0.7) or low (0.3 to 0.5) [47]. Statistical analyses were performed using SPSS 15.0 for Windows (SPSS Inc., Chicago, USA) with a statistical significance of p < 0.05.

Thirty participants (Table 1) completed both the PC- and VR-based tests. No one reported symptoms of cybersickness, such as dizziness, nausea, eye strain, or headache. The time required to complete the VR-based test was on average 334.7 ± 48.1 seconds. Nonparametric methods were used because most of the data violated a normal distribution. The median SRTs obtained from the VR-based test were 325.3 ms for the dominant hand and 325.8 ms for the non-dominant hand and were similar between the two hands (p = 0.47). A Friedman test revealed a significant effect of different trial numbers, analyzed with 25-trial each block, on SRT when using non-dominant hand, χ²(3, n = 120) = 23.4, p < 0.001, and both hands, χ²(3, n = 120) = 21.6, p < 0.001., but not for the dominant hand, χ²(3, n = 120) = 6.48, p = 0.09. The post hoc comparison with Bonferroni correction (p < 0.0083) showed a significantly longer SRT in the forth 25 trial block compared to the first three blocks in using non-dominant hand and bilateral hands (Fig. 3). Test-retest reliability was analyzed with various trial numbers, and the ICCs were all between 0.60 and 0.78 (Table 3). However, the ICC was the lowest (0.60) on 25 trials when using the dominant hand for response.

The median SRTs for the PC-based task were 321.5 ms for the dominant hand and 317.0 ms for the non-dominant hand (Table 2) and thus were similar between the two hands (z= -0.72, p = 0.47 by Wilcoxon signed rank test). Generally, the VR-based SRT was longer than the PC-based SRT for both hands, with a significant difference for the non-dominant hand (z= -2.49, p = 0.01 by Wilcoxon signed rank test). Nonetheless, there was a high correlation (r = 0.85 ~ 0.89) and good agreement between the SRT obtained from VR-based and PC-based methods (Table 2). Bland–Altman plotting showed only one data point of both dominant and non-dominant hands outside the 1.96 standard deviation (SD) range (Fig. 4). The VR-based SRT was on average 9–10 ms longer than the computer-based SRT.

Table 1

Demographic data of the participants
Variable	Results
Age (years), mean ± standard deviation	37.6 (16.5)
Female, N (%)	20 (66.7%)
Right handed, N(%)	25(83.3%)

Table 2

The descriptive data, and the reaction time tested with PC- and VR-based methods.
Hand	Virtual reality		Computer		p value*	Correlation coefficient
Hand	median (IQR)	mean ± SD	median (IQR)	mean ± SD	p value*	Correlation coefficient
Dominant	325.3 (57.2)	333.6 ± 47.3	321.5 (27.0)	328.3 ± 41.1	0.225	0.85 (p < 0.001)
Non-dominant	325.8 (52.4)	336.4 ± 49.4	317.0 (44.3)	324.2 ± 42.7	0.013	0.89 (P < 0.001)
Bilateral	326.0 (54.5)	334.7 ± 48.1	319.5 (46.0)	325.3 ± 42.6	0.006	0.86 (p < 0.001)
Note: *comparison between computer and virtual reality-based results with Mann-Whitney U test; Note: IQR: interquartile range; SD: standard deviation.

Table 3

Intraclass correlation coefficient of SRT for various trial numbers in VR-based tests
	Dominant hand	Non-dominant hand	Bilateral hand
25 trials	0.60 (0.32–0.79, p < 0.001)	0.71 (0.48–0.85, p < 0.001)	0.66 (0.49–0.78, < 0.001)
50 trials	0.64 (0.38–0.81, p < 0.001)	0.77 (0.57–0.88, p < 0.001)	0.71 (0.56–0.82, p < 0.001)
75 trials	0.66 (0.39–0.82, p < 0.001)	0.78 (0.59–0.89, p < 0.001)	0.72 (0.58–0.83, p < 0.001)
100 trials	0.72 (0.49–0.86, p < 0.001)	0.71 (0.47–0.85, p < 0.001)	0.71 (0.56–0.82, p < 0.001)

The study results documented good test-retest reliability and high correlation with commercially available PC-based software. We also examined the change with various trial numbers to determine the clinical feasibility regarding the reliability and potential decrements of performance. The results could be fundamental for future applications and warrant further discussion.

It is widely reported that SRTs can be tested with computer-based measurements, while the data related to VR-based SRT measurements were very limited. The RT latencies measured by CONVIRT, a concussion assessment tool with VR and eye-tracking technology, were 245 to 312 ms [48, 49]. In comparison, our results obtained from the VR-based measurements were slightly longer than the reported ranges. Notably, the mean ages in these studies were mostly younger than those in our study (early twenties vs. 37 years old). Age alone can be associated with a slowing of 0.55ms per year [14]. Besides, the relatively wide range of reported SRTs could be attributed to intraindividual variability, hardware features and software used [14, 22].

We conducted the SRT task with 100 trials as in some previous studies [14, 44]. The total testing time was nearly 4 minutes, which was longer than the studies using CONVIRT (120 s and 35 trials) or Dynavision (1 minute) [39, 41]. We observed a significantly longer SRT in the fourth 25-trial block for the non-dominant hand and bilateral hands, which implied a selective decrement of performance. Previous studies used variable numbers of trials to determine RTs, and there was no standard methodology. The selection of trial numbers should be considered with concerns of practice effect after repeated tests and fatigability from excessive repetition [50]. The decremental performance is likely to be related to attention rather than muscle fatigue [51], since the keypresses are simple and unlikely to be tiring [50]. Therefore, we suggest a trial number of 75 or less to avoid decremental performance on the VR-based SRT measurement, but further studies in terms of practice effect are needed.

Our results showed good test-retest reliability, with ICCs between 0.60 and 0.78 at one-week intervals. The test-retest reliability of RTs for other VR-based RT tests has rarely been reported. The RTs obtained from CONVIRT had good to excellent reliability in both healthy and alcohol-intoxicated groups (ICC = 0.90 and 0.83, respectively) [48, 49]. The visuomotor RT obtained from the Dynavision VR had an ICC of 0.85 to 0.91[39]. The above data were obtained on the same day. The ICC would be reduced as the time intervals increased according to the experiments with PC-based RT tests: 0.84–0.9 for the same day, 0.76 within one week, and 0.51 between sport seasons [52–54]. We also computed the ICC for various trial numbers, and they were mostly between 0.60 and 0.78, with the lowest one with 25 trials in using the dominant hand. This result was close to one review suggesting at least 40 trials for RT measurement [22] but inconsistent with other studies obtaining a reliable SRT with few trials [55]. We suggest a trial number greater than 25. Nonetheless, other options can also be considered, such as discarding the test results of the first few trials [50] or adding the number of practice trials to minimize the practice-related improvement [56].

We chose commercial software to test convergent validity and found a high correlation (r = 0.85 ~ 0.89) and good agreement of the SRTs obtained from both systems. This correlation was higher than that between CONVIRT, a VR system, and the commercial Cogstate battery (Cogstate Research software; Cogstate Limited) (r = 0.59 and 0.58 in 2 separate sessions) [48]. Comparatively, the correlation coefficient between the data obtained from the Oculus Rift and a standard computer assessment was the highest in visual processing speed (r = 0.51 and 0.66) and the lowest in the threshold of conscious perception r = 0.37 and 0.54 in 2 separate sessions) [57]. However, we observed an average bias of 9–10 ms longer SRTs in the VR-based tests than the rehab software according to the Bland–Altman plots. A similar finding indicates that Dynavision with a two-dimensional design had a 3–10% better performance than under VR conditions [39, 58]. There were several possible explanations for the longer RTs with the VR-based measurement. The hardware and software differences between the PC-based and VR-based measurements were substantial, despite our efforts to design a similar visual display in terms of target size, color, and distance. For example, the physiological response in the virtual environments (VEs) had some impact. For example, more time was required for the healthy participants to react to virtual stimuli than for them to react to direct vision stimuli, with mean RTs of recognition of 524.7 ms and 295.6 ms, respectively [59]. Different responses in the real world and VEs have been reported in other studies of balance control and limb motions [33, 34]. Another reason is the different response modalities used in PC-based and VR-based measurement, one with the custom keyboard and another with the space bar. The systematic bias does not always impede clinical usability if reliability is assured, but users should be aware of the bias for cross-comparison.

Using the VR system to measure RTs has some advantages. For example, calibrated head movement or integration with eye tracking provides reliable information on visual-perceptual strategies [60]. The eye tracking system in the VR system can also provide tracking of the gaze of users in three-dimensional space, allowing visual exploration behavior and spatial distribution analysis [61]. As a result, more information regarding visual perceptual strategies and visual attention could theoretically be obtained in combination with the RT.

Study limitation

Several limitations should be considered in our study. First, the study protocol was tested in healthy participants and for SRT only. More studies are required to examine the feasibility of other RT measurements and in clinical-intended groups, such as patients with cerebral diseases, stroke, dementia, and Parkinson’s disease, with special concern for the influence of cybersickness. Second, we tested the method only for standard keyboard input but not the HTC Vive’s motion controllers. Using motion controllers allows the evaluation of visuomotor RTs and tracks the hand movements of the participants simultaneously. Third, the operation of the software relies on game engines and VR runtime software. Any future updates of these third-party tools might cause compatibility problems or result in different measurements. Some researchers have suggested regularly validating the precision and accuracy of time measurements with new software releases [40].

Our results supported the feasibility, reliability, and validity of an HMD-VR-based program to test SRTs. Our next goal would be to design other RT test paradigms, such as choice RT or visual-field-specific RT tests. In addition, further studies are warranted to determine the feasibility, reliability, and validity in diseased groups.

HMD-VR

head-mounted display for virtual reality

ICC

intra-class correlation coefficient

IQR

interquartile range

PC

personal computer

RT

reaction time

SD

standard deviation

SRT

simple reaction time

VE

virtual environment

VR

virtual reality

Ethics approval and consent to participate: This study was approved by the ethical committee of the National Taiwan University Hospital (approval number: 202104030RINC, date of approval: 05/13/2021). Written informed consent was obtained from the participants. All procedures were carried out in accordance with the ethical rules and the principles of the Declaration of Helsinki.

Consent for publication: Not applicable

Availability of data and materials: Raw data were generated at National Taiwan University Hospital. Derived data supporting the findings of this study are available from Huey-Wen Liang on request.

Conflicts of interest: The authors declare no conflicts of interest.

Funding statement: The study was made possible by research funding was from National Taiwan University Hospital (111-N0034).

Authors’ contributions: Y.-C.C. and H.-W.L. did the literature search. Y.-C.C. and H.-W.L. designed the study. Data was collected by Y.-C.C. H.-W.L. carried out the statistical analysis. Y.-C.C. and H.-W.L. created the figures. Y.-C.C. H.-W.L. were involved in the interpretation of results and writing the report. Both authors have read and agreed to the published version of the manuscript.

Acknowledgements: Not applicable

Gautam Y, Bade M. Effect of auditory interference on visual simple reaction time. Kathmandu Univ Med J (KUMJ). 2017;15(60):329–31.
Posner MI, Rueda MR. Mental chronometry in the study of individual and group differences. J Clin Exp Neuropsychol. 2002;24(7):968–76.
Der G, Deary IJ. The relationship between intelligence and reaction time varies with age: Results from three representative narrow-age age cohorts at 30, 50 and 69 years. Intelligence, 2017. 64: p. 89–97.
Mena-Garcia L, et al. Visual processing speed in hemianopia patients secondary to acquired brain injury: a new assessment methodology. J Neuroeng Rehabil. 2020;17(1):1–14.
Kaizer F, et al. Response time of stroke patients to a visual stimulus. Stroke. 1988;19(3):335–9.
Vishteh RA, et al. Evaluation of simple visual reaction time of different colored light stimuli in visually normal students. Clin Optometry. 2019;11:167.
Eckner JT, et al. Reliability and criterion validity of a novel clinical test of simple and complex reaction time in athletes. Percept Mot Skills. 2015;120(3):841–59.
Mercer VS, et al. Reliability and validity of a clinical test of reaction time in older adults. J Geriatr Phys Ther. 2009;32(3):103–10.
Willoughby M, et al. Between-and within-person contributions of simple reaction time to executive function skills in early childhood. J Exp Child Psychol. 2020;192:104779.
Jakobsen LH et al. Validation of reaction time as a measure of cognitive function and quality of life in healthy subjects and patients. 2011. 27(5): p. 561–70.
Jayakody O, et al. The association between simple reaction time variability and gait variability: The Tasmanian Study of Cognition and Gait. Volume 89. Gait & Posture; 2021. pp. 206–10.
Eckner JT, et al. Evaluating a clinical measure of reaction time: an observational study. Percept Mot Skills. 2009;108(3):717–20.
Girardin Y, Montpetit R, Manceau J. Measure of simple reaction time with the aid of digital computers. Int J Sport Psycol, 1970.
Woods DL, et al. Factors influencing the latency of simple reaction time. Front Hum Neurosci. 2015;9:131.
Kosinski RJ. A literature review on reaction time. Clemson Univ. 2008;10(1):337–44.
Brenner E, Smeets JBJ. How Can You Best Measure Reaction Times? J Mot Behav. 2019;51(5):486–95.
Janssen S. The determinants of reaction times: influence of stimulus intensity. University of Waterloo; 2015.
Bamne SN, Fadia AD, Jadhav AV. Effect of colour and gender on human reaction time. Indian J Physiol Pharmacol. 2011;55(4):388–9.
Plewan T, Weidner R, Fink GR. The influence of stimulus duration on visual illusions and simple reaction time. Exp Brain Res. 2012;223(3):367–75.
Habekost T. Clinical TVA-based studies: a general review. Front Psychol. 2015;6:290.
Bundesen C. A theory of visual attention. Psychol Rev. 1990;97(4):523.
Crabtree DA, Antrim LR. Guidelines for measuring reaction time. Percept Mot Skills. 1988;66(2):363–70.
BERLUCCHI G, et al. SIMPLE REACTION TIMES OF IPSILATERAL AND CONTRALATERAL HAND TO LATERALIZED VISUAL STIMULI. Brain. 1971;94(3):419–30.
Braun C, Larocque C, Achim A. Experimental disentangling of spatial-compatibility and interhemispheric-relay effects in simple reaction time (Poffenberger paradigm). Exp Brain Res. 2004;157:442–56.
Ratinckx E, Brysbaert M, Vermeulen E. CRT screens may give rise to biased estimates of interhemispheric transmission time in the Poffenberger paradigm. Exp Brain Res. 2001;136(3):413–6.
Foerster RM, et al. Using the virtual reality device Oculus Rift for neuropsychological assessment of visual processing capabilities. Sci Rep. 2016;6(1):1–10.
Chua SIL et al. Virtual reality for screening of cognitive function in older persons: comparative study. 2019. 21(8): p. e14821.
Debeljak M, et al. Simple and choice reaction times of healthy adults and patients after stroke during simulated driving. Int J Rehabil Res. 2019;42(3):280–4.
Kim T-L et al. FOPR test: a virtual reality-based technique to assess field of perception and field of regard in hemispatial neglect. 2021. 18(1): p. 1–12.
Lim JE et al. A Fully-Immersive and Automated Virtual Reality System to Assess the Six Domains of Cognition: Protocol for a Feasibility Study. 2020. 12.
Pratviel Y et al. Reliability of the Dynavision task in virtual reality to explore visuomotor phenotypes. 2021. 11(1): p. 1–12.
Dvorkin AY, et al. Mapping the neglected space: gradients of detection revealed by virtual reality. Neurorehabil Neural Repair. 2012;26(2):120–31.
Magdalon EC, et al. Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment. Acta Psychol. 2011;138(1):126–34.
Liang HW, et al. The Effects of Visual Backgrounds in the Virtual Environments on the Postural Stability of Standing. IEEE Trans Neural Syst Rehabil Eng. 2021;29:1129–37.
Szpak A, et al. Beyond feeling sick: the visual and cognitive aftereffects of virtual reality. iEEE Access. 2019;7:130883–92.
Nalivaiko E, et al. Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time. Physiol Behav. 2015;151:583–90.
Schlegel RE, Gilliland K. Development and quality assurance of computer-based assessment batteries. Arch Clin Neuropsychol. 2007;22(Suppl 1):S49–61.
Rodríguez C, et al. Comparison between two continuous performance tests for identifying ADHD: Traditional vs. virtual reality. Int J Clin Health Psychol. 2018;18(3):254–63.
Pratviel Y, et al. Reliability of the Dynavision task in virtual reality to explore visuomotor phenotypes. Sci Rep. 2021;11(1):587.
Wiesing M, Fink GR, Weidner R. Accuracy and precision of stimulus timing and reaction times with Unreal Engine and SteamVR. PLoS ONE. 2020;15(4):e0231152.
Horan B, et al. Development of a new virtual reality test of cognition: assessing the test-retest reliability, convergent and ecological validity of CONVIRT. BMC Psychol. 2020;8(1):61.
Deary IJ, Liewald D, Nissan J. A free, easy-to-use, computer-based simple and four-choice reaction time programme: The Deary-Liewald reaction time task. Behav Res Methods. 2011;43(1):258–68.
El-Tamawy M, et al. Primary fatigue contributes to cognitive dysfunction in patients with multiple sclerosis. Egypt J Neurol Psychiatry Neurosurg. 2016;53(2):74–8.
Pellicano A, et al. Interhemispheric vs. stimulus-response spatial compatibility effects in bimanual reaction times to lateralized visual stimuli. Front Psychol. 2013;4:362.
Cicchetti D. Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instrument in Psychology. Psychol Assess. 1994;6:284–90.
Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–60.
Hinkle DE, Wiersma W, Jurs SG. Applied Statistics for the Behavioral Sciences. 5th ed. Boston: Houghton Mifflin; 2003.
Horan B, et al. Development of a new virtual reality test of cognition: assessing the test-retest reliability, convergent and ecological validity of CONVIRT. BMC Psychol. 2020;8(1):1–10.
Amato I, et al. Assessing the utility of a virtual-reality neuropsychological test battery,‘CONVIRT’, in detecting alcohol-induced cognitive impairment. Behav Res Methods. 2021;53(3):1115–23.
Günendi Z, Taskiran OO, Beyazova M. What is the optimal repetition number in electromyographic reaction time studies? Clin Biomech Elsevier Ltd. 2005;20(7):754–8.
Milroy T. Fatigue studied in reaction time experiments. Q J Experimental Physiology: Translation Integr. 1909;2(3):277–82.
Eckner JT, Kutcher JS, Richardson JK. Between-seasons test-retest reliability of clinically measured reaction time in National Collegiate Athletic Association Division I athletes. National Athletic Trainers' Association, Inc.; 2011. pp. 409–14.
Collie A, et al. CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clin J Sport Med. 2003;13(1):28–32.
Woods DL, et al. The effects of repeated testing, simulated malingering, and traumatic brain injury on high-precision measures of simple visual reaction time. Front Hum Neurosci. 2015;9:540.
Hamsher KdS, Benton A. The reliability of reaction time determinations. Cortex. 1977;13(3):306–10.
Del Rossi G, Malaguti A, Del S, Rossi. Practice effects associated with repeated assessment of a clinical test of reaction time. J Athl Train. 2014;49(3):356–9.
Foerster RM, et al. Using the virtual reality device Oculus Rift for neuropsychological assessment of visual processing capabilities. Sci Rep. 2016;6(1):1–10.
Wells AJ, Johnson B-aD. Test–Retest Reliability, Training, and Detraining Effects Associated With the Dynavision D2™ Mode A Visuomotor Reaction Time Test. Journal of Sport Rehabilitation, 2021. 1(aop): p. 1–9.
Sanchez Y, Pinzon D, Zheng B. Reaction time for processing visual stimulus in a computer-assisted rehabilitation environment. Disabil Rehabilitation: Assist Technol. 2017;12(7):725–9.
Pratviel Y, et al. Reliability of the Dynavision task in virtual reality to explore visuomotor phenotypes. Sci Rep. 2021;11(1):1–12.
Clay V, König P, Koenig S. Eye tracking in virtual reality. J eye Mov Res, 2019. 12(1).

No competing interests reported.

Reliability and validity of a virtual reality-based measurement of simple reaction time: a cross-sectional study

Status:

Version 1

Abstract

Figures

Background

Methods

Statistical analysis

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1