Sequential Motor Learning Transfers from Real to Virtual Environment

doi:10.21203/rs.3.rs-94349/v1

Download PDF

Research

Sequential Motor Learning Transfers from Real to Virtual Environment

https://doi.org/10.21203/rs.3.rs-94349/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 30 Jun, 2021

Read the published version in Journal of NeuroEngineering and Rehabilitation →

You are reading this latest preprint version

Background:

Skill acquisition of motor learning between virtual environments (VEs) and real environments (REs) may be related. Although studies have previously examined the transfer of motor learning in VEs and REs through the same tasks, only a small number of studies have focused on studying the transfer of motor learning in VEs and REs by using different tasks. Thus, detailed effects of the transfer of motor skills between VEs and REs remain controversial. Here, we investigated the transfer of sequential motor learning between VEs and REs conditions.

Methods:

Twenty-seven healthy volunteers performed two types of sequential motor learning tasks; a visually cued button press task in RE (RE task) and a virtual reaching task in VE (VE task). Participants were randomly assigned to two groups in the task order; the first group was RE task followed by VE task and the second group was VE task followed by RE task. Subsequently, the response time in RE task and VE task was compared between the two groups respectively.

Results:

The results revealed that sequential motor learning was transferred when motor learning in VEs was performed after motor learning in REs, but not when motor learning in REs was performed after motor learning in VEs.

Conclusions:

These findings suggested that sequential motor learning in VEs can be facilitated by motor learning task consisting of the same sequence in REs even when different task is applied. These results may derive from the fact that motor learning in REs is more implicit than that in VEs.

Cellular & Molecular Neuroscience

Cognitive Neuroscience

sequential motor learning

virtual environment

real environment

transfer of motor learning

Motor learning refers to an improvement in the performance of sensory-guided motor behavior through practice ¹. The acquisition of new skills through motor practice is essential for interaction with the environment and to make adjustments involving integration of multiple elements of movement. The two components of the learning process^2,3 include implicit motor learning that improves the performance of a sequence without the knowledge of the sequence⁴ and explicit motor learning that involves conscious recollection with the knowledge of the sequence. The memory system differs between the implicit and explicit learning processes ⁵. The serial reaction time task (SRTT) is widely used to quantitatively evaluate the acquisition of a new skill ^6–8.

Recently, a study demonstrated that learning effects can be reproduced not only in real environments (REs) but also in virtual environments (VEs) ⁹. VEs can modulate perception and cognition by providing coherent sensory feedback that corresponds to the actions taking place within the VE and can make users a psychological sense of being there ¹⁰. To illustrate, previous studies have reported that if the visual presentation of the virtual (or fake) hand movement matches the subject’s active hand movement, both the agency and ownership of the virtual hand are induced concurrently ^11–14. Virtual hand illusion (VHI) was also used in motor learning owing to its applicability and expandability. Several studies have reported that motor learning is promoted even in VEs ^{9, 15, 16}. In addition, a previous study has suggested that post-training performance in virtual and real training is equivalent and both of which significantly exceeds without training ¹⁷. However, another study demonstrated that training in VE cannot promote better performance than the same task in RE ¹⁸. Additionally, several reports focused on interactions of skill acquisition of motor learning in VEs and REs. For instance, a study in healthy subjects demonstrated that skill acquisition of the sequential motor learning occurs at the same rate in both VE and conventional screen environments, while the transfer of motor skills was not observed from VE to the screen environment ¹⁵. Another study reported that skill acquisition in individuals with the neuromuscular diseases transferred from VEs to REs ¹⁹. In contrast, motor learning and motor performance did not transfer from VEs to REs in older adults and individuals with neuromuscular diseases ^{18, 20, 21}. These results imply the presence of some relationship in skill acquisition of motor learning between VEs and REs. However, whether transfer of skill acquisition of motor skills occur between VEs and REs is still controversial. In addition, the above-mentioned studies have investigated motor learning with the same tasks in different dimensions (VE and RE), whereas few studies have focused on motor learning with the different tasks in different dimensions. Considering the relationship in skill acquisition of motor learning between VEs and REs, we hypothesized that transfer of motor learning occurs between VEs and REs even in different tasks.

Thus, the purpose of the present study was to reveal whether transfer of motor learning can be caused even in different tasks between VEs and REs. The present study was a crossover design, participants performed sequential motor learning tasks in both the conditions. In order to examine the transfer of motor learning, we compared the response time between both motor learning task.

Participants

Thirty healthy volunteers (21.5 ± 3.0 years; 23 women) participated in this study. All participants were right-handed, as determined by the Edinburgh Handedness Inventory Test (Oldfield, 1971). No participant had a history of neurological or psychiatric disease, and all had a normal or corrected-to-normal vision. In accordance with the Declaration of Helsinki, we explained the purpose and possible consequences of this study to all participants and obtained their informed consent before the study commenced.

Experimental design

In order to examine the transfer of motor learning, the participants performed sequential motor learning tasks in both REs and VEs (Fig. 1A). For the motor learning task in a RE, the participants performed a visually cued button press task (RE task; Fig. 1B). For the motor learning task in a VE, the participants performed a virtual reaching task (VE task; Fig. 1C).

The participants were randomly assigned to two groups to keep a counter balance in the task order; the first group was RE task followed by VE task (RE-VE group) and the second group was VE task followed by RE task (VE-RE group). There was a 5-min interval between the tasks. Three participants were excluded from data analysis because mean responses were beyond ± 2SD from the mean for subjects. Thus, the final sample size was 27 participants.

Motor learning task in RE (RE task)

The participants performed a visually cued button press task consisting of a 12-digit motor sequential learning task in which participants were required to react with their four right-hand fingers (Fig. 1A) ⁸. Four horizontal bars were displayed on the screen. When the color of a bar changed from gray to blue, the participants were instructed to press the corresponding button as quickly and accurately as possible. If a participant pressed the correct button, the next stimulation was presented after 1 s. If a participant pressed an incorrect button, the stimulation was unchanged until the participant pressed the correct button. This motor learning task consisted of five sequence blocks (S1-S5) and two random blocks (R1, R2). Random blocks were set before (R1) and after the five sequence blocks (R2). The sequence blocks comprised five repeats of 12 stimuli in the same sequence. Thus, a total of 300 button presses were performed in a sequence block, whereas 120 button presses were performed in a random block.

Motor learning task in VE (VE task)

In the virtual motor learning task, four transparent gray targets with a cylindrical shape and a transparent blue base point with a hemispherical shape were displayed on a virtual desk located in a VE. The participant was fixed with a head-mounted display (HMD) and instructed to sit in a chair at a specific position where they could reach all the targets and the base point easily with minimal arm movement, and to perform a virtual reaching task (Fig. 2A). The base point disappeared when the participants kept their virtual hand in the hemisphere for 1 s, which meant the beginning of an experimental trial. As the color of a target changed from gray to green, the participants were instructed to put the virtual right hand, which was rendered overlapping at the participant’s hand position in the RE, into the designated target as quickly and accurately as possible. The color of target changed from green to red when the participants reached the target. As the base point was displayed again just after the virtual hand successfully reached the target; the participants could start the next trial whenever they liked by touching it. On the other hand, if the virtual hand could not reach the intended target within 2 s, the color of target changed from green to red together with re-emergence of the base point; in this case, the color of target returned to gray when the participant touched the base point.

In order to examine the transfer of sequential motor learning, the orders of button pressing in RE and reaching in VE were completely same sequence in both tasks. Thus, VE task also consisted of five sequence blocks (S1-S5) and two random blocks (R1, R2) and a total of 300 virtual reaching were performed in a sequence block, whereas 120 virtual reaching were performed in a random block.

Apparatuses

In the RE task, visual stimuli were applied by Presentation System (Neurobehavioral Systems, USA) and recording of response times were realized by fiber optic computer response system (PKG-9904, Current Design Inc., USA).

In the VE task, we developed a Virtual reality experimental system by integrating an HMD (Oculus Rift, Oculus) and a hand tracking system (Leap Motion, Ultraleap Ltd.) into a game engine (Unity 2018.4.24, Unity Technologies) (Fig. 2A). Our VE task mainly consisted of four cylindrical targets, a hemispherical base point, a desk, and a virtual right hand. Leap Motion attached on the HMD tracked the participant’s hand position, orientation, and posture, and the information was applied to render the virtual hand in the VE. The four cylindrical targets were symmetrically allocated every 30 degrees centering around the base point (Fig. 2B); the distance between the base point and each target was 25 cm in the scale of RE. The collision detection with the targets or base point was ran based on the distance between their center positions and the palm position of virtual hand. It was considered that the hand reached a target or base point when the distance became smaller than their diameters; in the scale of RE, the diameters of targets and base point were 10 cm, respectively. Rendering of VE was performed in 80 Hz which was the same as the refresh rate of Oculus Rift. The sampling time for the experiment control was set in 20 ms (i.e., 50 Hz sampling rate), within which the collision detection, control of experimental condition, and data acquisition were performed.

Virtual hand illusion questionnaire

We subjectively evaluated the participant’s experience or feeling of the right hand during movements of the virtual right hand. The participants were asked to answer a VHI questionnaire with a seven-point Likert scale (− 3 to + 3). In the seven-point Likert scale, − 3 and + 3 were set as “I strongly disagree with the statement” and “I strongly agree with the statement,” respectively; 0 was considered as a neutral rating allocated for unjudgeable experience. The employed questionnaire items based on the original rubber hand illusion questionnaire ²² were as follow:

Q1: I felt as if the virtual hand was my own hand.

Q2: The movement of virtual hand matched with the movement produced by my hand.

Q3: I felt like my hand was becoming bigger.

Q4: I could not feel my hand.

Q5: I felt as if my hand was turning “virtual”.

Q6: I felt as if my whole hand was moving.

The first two items were designed to correspond to the VHI. The illusion items assessed the embodiment of virtual hand (Q1) and the sense of agency (Q2) during the experiment, respectively. The other items served as control for suggestibility, which were unrelated to the VHI. The suggestibility meant that sometimes the participants rated all the items in the same manner for any reason and was therefore removed from the analysis.

Statistical analysis

Statistical analyses were performed using MATLAB (R2017a) and SPSS (version 25). First, the effects of time (R1 vs. S5) and task order (RE-VE group vs. VE-RE group) on the response times were assessed with a two-way repeated measure analysis of variance (ANOVA). The Greenhouse-Geisser correction was applied to the degrees of freedom when the sphericity assumption was violated. In case of significant effects, post hoc analyses were performed to test interaction effects with Student t-tests. Next, the mean rating for Q1 and Q2 was compared with RE-VE group and VE-RE group using a Mann-Whitney test.

Response time in the RE task

A two-way repeated measure ANOVA with factors time (Blocks) and task order (RE-VE group vs. VE-RE group) showed a significant main effect of time (F_(6,25) = 5.121, p = 0.003, η²p = 0.17) but not task order (F_(1,25) = 0.197, p = 0.661, η²p = 0.008) or time × task order interaction (F_(6,25) = 0.667, p = 0.568, η²p = 0.026) (Fig. 3A).

Response time in the VE task

A two-way repeated measure ANOVA with factors time (Blocks) and task order (RE-VE group vs. VE-RE group) revealed no significant main effect of time (F_(6,25) = 0.984, p= 0.387, η²ｐ = 0.038) and task order (F_(1,25) = 1.184, ｐ= 0.287, η²ｐ = 0.045), and a significant time × task order interaction (F_(6,25)=4.835, ｐ=0.01, η²ｐ=0.162) (Fig. 3B). In order to further investigate the significant time × task order interaction, post hoc analyses were performed for each block with Student t-test. The results showed the significant difference between RE-VE group and VE-RE group only in S5 block (p = 0.017).

Virtual hand illusion questionnaire

A Mann-Whitney U test showed no significant difference in VHI questionnaire between VE-RE group and RE-VE group (Q1 : z = −0.351, p = 0.725; Q2 : z = −0.30, p = 0.764) (Fig.4).

The present study investigated transfer of motor learning between VEs and REs in different tasks. The results showed that sequential motor learning was transferred only when motor learning in VEs was performed after motor learning in REs, suggesting that motor learning in VEs can be facilitated with motor learning in REs even when different task is applied.

VEs enable individuals to have virtual experiences that are similar to those in REs. However, recent studies have shown that the learning mechanism between VEs and REs is different ²³. As for the neurophysiological aspects of visual processes, visual information is processed thorough ventral and dorsal pathways. The ventral path provides perception and identification of visual inputs, and the dorsal path monitors real-time information ^{24, 25}. In the REs, the visual information within peripersonal space was processed using the dorsal path ²⁶, whereas visual processing in VEs induced the ventral path ²⁷. These results indicate that processing of sensory information in the brain between VEs and REs is different. In the present study, motor learning in REs improved motor learning in VEs, whereas motor learning in VEs did not improve motor learning in REs. Some studies have demonstrated that motor learning in VEs requires more brain activity than that in REs for cognitive and motor control ^28,29. In addition, Ranganathan et al. ³⁰ reported that motor learning in different dimensions interferes with each other. If the first task did not share task dimensions with the later task, there was prolonged interference in learning the later task. However, another study reported that motor learning in REs involved more implicit motor learning compared to motor learning in VEs ³¹. This result supports our finding that sequential motor learning in REs facilitates sequential motor learning in VEs beyond dimensional interference. However, additional research is required to assess this possibility because we did not evaluate neurophysiological changes using electroencephalogram and functional magnetic resonance imaging.

In the present study, virtual reaching task in VEs did not promote sequential learning in the VE-RE group. Previous research on VEs employed several tools, such as a two-joint mechanical arm ¹⁶, joysticks ⁹, and pinch force sensors ¹⁵, which showed improvement in the motor learning ability. These experiments were performed using sensory information as clues instead of visual information, like the arm-reaching task used in the present study. Another study reported that providing tactile feedback using sensor gloves in VEs reduces the discrepancy between the virtual and physical environments ³². These results indicate that the arm-reaching task in VEs may not facilitate sequential learning because of the lack of tactile stimulation. Furthermore, several studies have reported that motor learning in VEs is influenced by individual factors ^15,33,34. For instance, experience with video games showed a positive impact on performance in VEs ^15,35. Considering the reports of the aforementioned studies, the lack of motor learning capacity in VEs observed in the present study may be attributed to individual factors.

The present study investigated whether transfer of motor learning can be caused even in different tasks between VEs and REs. In order to examine the transfer of motor learning, we compared the response time between both motor learning task. The results showed that sequential motor learning in REs facilitated sequential motor learning in VEs. These results may derive from the fact that motor learning in REs is more implicit than that in VEs. However, we cannot discuss the neurophysiological aspect for the transfer of motor learning between VEs and REs since we did not evaluate the neurophysiological data. Thus, to demonstrate the detailed neural mechanism of sequential learning from RE to VE, further study is needed.

VEs: virtual environments

REs: real environments

SRTT: serial reaction time task

VHI: virtual hand illusion

HMD: head-mounted display

Ethics approval and consent to participate

The experimental procedure was approved by the Ethical Committee for Oita University School of Medicine. In accordance with the Declaration of Helsinki, we explained the purpose and possible consequences of this study to all participants and obtained their informed consent before the study commenced.

Consent for publication

Written informed consent for the publication was obtained from all the participants.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

The work was supported by a grant from the KAKENHI [18H03132]; the Japan Society

for the Promotion of Science (JSPS).

Authors’ contribution

YT and HS contributed to designing of the experiment, data collection, data analysis, and drafting of the manuscript. MH contributed to the construction of virtual reality system. TI contributed to the construction of serial reaction time task system. YS contributed to the recruitment of participants. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to acknowledge all the volunteers who participated in this study.

Authors’ information

¹ Department of Rehabilitation, Oita University Hospital, Oita, Japan

² Graduate School of Welfare and Health Science, Oita University, Oita, Japan

³ Graduate School of Science and Engineering, Saitama University, Saitama, Japan

⁴ Faculty of Welfare and Health Science, Oita University, Oita, Japan

⁵ Research Center for Child Mental Development, Kanazawa University, Kanazawa, Japan

Conflict of Interest and Funding Sources

None of the authors declare any conflict of interest. This work was supported by a grant for KAKENHI (18H03132) grants funded by the Japan Society for the Promotion of Science (JSPS).

Krakauer JW, Mazzoni P: Human sensorimotor learning: adaptation, skill, and beyond. Curr Opin Neurobiol 2011, 21:636-644.
Willingham DB: Becoming aware of motor skill. Trends Cogn Sci 2001, 5:181-182.
Willingham DB: A neuropsychological theory of motor skill learning. Psychol Rev 1998, 105:558-584.
Robertson EM: From creation to consolidation: a novel framework for memory processing. PLoS Biol 2009, 7:e19.
Kantak SS, Mummidisetty CK, Stinear JW: Primary motor and premotor cortex in implicit sequence learning--evidence for competition between implicit and explicit human motor memory systems. Eur J Neurosci 2012, 36:2710-2715.
Nojima I, Watanabe T, Gyoda T, Sugata H, Ikeda T, Mima T: Transcranial static magnetic stimulation over the primary motor cortex alters sequential implicit motor learning. Neurosci Lett 2019, 696:33-37.
Sugata H, Yagi K, Yazawa S, Nagase Y, Tsuruta K, Ikeda T, Matsushita K, Hara M, Kawakami K, Kawakami K: Modulation of Motor Learning Capacity by Transcranial Alternating Current Stimulation. Neuroscience 2018, 391:131-139.
Mary Jo Nissen, Bullemer P: Attentional requirements of learning: Evidence from performance measures. Cognitive Psychology 1987, 19:1-32.
Sense F, van Rijn H: Probabilistic motor sequence learning in a virtual reality serial reaction time task. PLoS One 2018, 13:e0198759.
Slater M, Sanchez-Vives MV: Enhancing our lives with immersive virtual reality. Frontiers in Robotics and AI 2016, 3: 74.
Ma K, Hommel B: The virtual-hand illusion: effects of impact and threat on perceived ownership and affective resonance. Front Psychol 2013, 4:604.
Llobera J, Sanchez-Vives MV, Slater M: The relationship between virtual body ownership and temperature sensitivity. J R Soc Interface 2013, 10(85):20130300.
Sanchez-Vives MV, Spanlang B, Frisoli A, Bergamasco M, Slater M: Virtual hand illusion induced by visuomotor correlations. PLoS One 2010, 5:e10381.
Slater M, Perez-Marcos D, Ehrsson HH, Sanchez-Vives MV: Inducing illusory ownership of a virtual body. Front Neurosci 2009, 3:214-220.
Juliano JM, Liew SL: Transfer of motor skill between virtual reality viewed using a head-mounted display and conventional screen environments. J Neuroeng Rehabil 2020, 17:48.
Shibuya S, Unenaka S, Ohki Y: The Relationship Between the Virtual Hand Illusion and Motor Performance. Front Psychol 2018, 9:2242.
Rose FD, Attree EA, Brooks BM, Parslow DM, Penn PR, Ambihaipahan N: Training in virtual environments: transfer to real world tasks and equivalence to real task training. Ergonomics 2000, 43:494-511.
Bezerra Í MP, Crocetta TB, Massetti T, Silva TDD, Guarnieri R, Meira CM, Jr., Arab C, Abreu LC, Araujo LV, Monteiro CBM: Functional performance comparison between real and virtual tasks in older adults: A cross-sectional study. Medicine (Baltimore) 2018, 97:e9612.
Quadrado VH, Silva TDD, Favero FM, Tonks J, Massetti T, Monteiro CBM: Motor learning from virtual reality to natural environments in individuals with Duchenne muscular dystrophy. Disabil Rehabil Assist Technol 2019, 14:12-20.
Massetti T, Fávero FM, Menezes LDC, Alvarez MPB, Crocetta TB, Guarnieri R, Nunes FLS, Monteiro CBM, Silva TDD: Achievement of Virtual and Real Objects Using a Short-Term Motor Learning Protocol in People with Duchenne Muscular Dystrophy: A Crossover Randomized Controlled Trial. Games Health J 2018, 7:107-115.
de Mello Monteiro CB, Massetti T, da Silva TD, van der Kamp J, de Abreu LC, Leone C, Savelsbergh GJ: Transfer of motor learning from virtual to natural environments in individuals with cerebral palsy. Res Dev Disabil 2014, 35:2430-2437.
Botvinick M, Cohen J: Rubber hands 'feel' touch that eyes see. Nature 1998, 391(6669):756.
Chu VW, Sanger TD: Two different motor learning mechanisms contribute to learning reaching movements in a rotated visual environment. F1000Res 2014, 3:72.
Goodale MA, Milner AD: Two visual pathways - Where have they taken us and where will they lead in future? Cortex 2018, 98:283-292.
Milner AD, Goodale MA: Visual pathways to perception and action. Prog Brain Res 1993, 95:317-337.
Weiss PH, Marshall JC, Zilles K, Fink GR: Are action and perception in near and far space additive or interactive factors? Neuroimage 2003, 18:837-846.
Beck L, Wolter M, Mungard NF, Vohn R, Staedtgen M, Kuhlen T, Sturm W: Evaluation of spatial processing in virtual reality using functional magnetic resonance imaging (FMRI). Cyberpsychol Behav Soc Netw 2010, 13:211-215.
Rohde M, Narioka K, Steil JJ, Klein LK, Ernst MO: Goal-related feedback guides motor exploration and redundancy resolution in human motor skill acquisition. PLoS Comput Biol 2019, 15(3):e1006676.
Slobounov SM, Ray W, Johnson B, Slobounov E, Newell KM: Modulation of cortical activity in 2D versus 3D virtual reality environments: an EEG study. Int J Psychophysiol 2015, 95:254-260.
Ranganathan R, Wieser J, Mosier KM, Mussa-Ivaldi FA, Scheidt RA: Learning redundant motor tasks with and without overlapping dimensions: facilitation and interference effects. J Neurosci 2014, 34:8289-8299.
Anglin JM, Sugiyama T, Liew SL: Visuomotor adaptation in head-mounted virtual reality versus conventional training. Sci Rep 2017, 7:45469.
Perez-Marcos D: Virtual reality experiences, embodiment, videogames and their dimensions in neurorehabilitation. J Neuroeng Rehabil 2018, 15:113.
Grassini S, Laumann K, Rasmussen Skogstad M: The Use of Virtual Reality Alone Does Not Promote Training Performance (but Sense of Presence Does). Front Psychol 2020, 11:1743.
Piber D, Nowacki J, Mueller SC, Wingenfeld K, Otte C: Sex effects on spatial learning but not on spatial memory retrieval in healthy young adults. Behav Brain Res 2018, 336:44-50.
de Castell S, Larios H, Jenson J: Gender, videogames and navigation in virtual space. Acta Psychol (Amst) 2019, 199:102895.

Download PDF

Journal Publication

published 30 Jun, 2021

Read the published version in Journal of NeuroEngineering and Rehabilitation →

Review #2 received at journal
17 Jan, 2021
Editorial decision: Major revision
17 Jan, 2021
Review #1 received at journal
08 Dec, 2020
Reviewer #2 agreed at journal
28 Nov, 2020
Reviewers invited by journal
18 Nov, 2020
Reviewer #1 agreed at journal
18 Nov, 2020
Editor assigned by journal
16 Oct, 2020
Submission checks completed at journal
15 Oct, 2020
Editor invited by journal
15 Oct, 2020
First submitted to journal
14 Oct, 2020

You are reading this latest preprint version

Sequential Motor Learning Transfers from Real to Virtual Environment

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Participants

Experimental design

Motor learning task in RE (RE task)

Motor learning task in VE (VE task)

Apparatuses

Virtual hand illusion questionnaire

Statistical analysis

Results

Response time in the RE task

Response time in the VE task

Virtual hand illusion questionnaire

Discussion

Conclusion

Abbreviations

Declarations

Ethics approval and consent to participate

Consent for publication

Availability of data and materials

Competing interests

Funding

Authors’ contribution

Acknowledgements

Authors’ information

Conflict of Interest and Funding Sources

References

Status:

Journal Publication

Version 1