The flowchart for this systematic review is illustrated in Figure 1. Sixteen studies were included; fourteen articles (188 participants) and two literature reviews. The Key data extracted from the included studies are presented in Table 2. Results were grouped into AVT of hemianopia in adults and in children, and then further grouped according to the AV task-type: tasks measuring the training effects by comparing visual stimulation training (VT) to (AVT), localization abilities in HH, and AV integration in patients with HH.
Quality assessment:
The risk of bias was assessed for each of the included articles (Supplementary Tables 1, 2 and 3) [see Additional files 1,2,3]. Overall, no article scored 100% for quality assessment in this section. The articles included 12 observational studies, two randomized control studies and two reviews. Twelve of the 16 articles scored between 76 and 87% on qualitative assessment and were deemed to have good quality. Four studies scored between 52 and 73% on the relevant quality checklists. All articles were included in this review.
Adult populations
1- AVT outcome measures on hemianopia patients:
Two reviews discussed visual rehabilitation using multisensory stimulation to compensate for the visual loss after stroke (24, 25). Seven original research articles recruited HH patients and trained them on AVT in order to study the effects of multisensory training on oculomotor scanning behaviour in comparison to the VT. A total of 71 patients (and 36 controls) were recruited for the AVT and tested on the same apparatus before and after the training period, either on unimodal visual detection task or on both uni- and multimodal visual detection tasks. These included the following study types; two randomised control studies (n=32) (18, 26), two cohort studies (n=40) (27, 28), one case control study (n=24) (12), and two uncontrolled longitudinal studies (n=11) (15, 29). The training duration varied between 2 weeks and 2 months, except in one study where one practice and one experimental session were undertaken, and the evaluation was performed during the experiment (29).
During the training, visual and auditory stimuli were presented in four different ways:
-
Spatially and temporally congruent only, by presenting the same duration of acoustic and visual stimuli (100 ms) at the same time and in the same spatial position: in two studies (18, 27).
-
Spatially congruent and disparate with temporally congruent only, in which the acoustic and visual stimuli were presented either at the same time and location or in the same time but different locations (16 and 32° of disparity in either side): in two studies (15, 29).
-
Spatially and temporally congruent and disparate, the spatial disparity between the acoustic and visual stimuli were systematically varied (0, 16, and 32° of disparity) as well as the temporal interval between the acoustic stimulus and the visual target was gradually reduced from (500-300) to 0 ms: in two studies (12, 26).
-
Passive auditory stimulation which depends on the hypothesis that sensory input from an intact modality (auditory) may improve processing of information by spared structures of a damaged sensory system (visual) through synchronous neural activity by repetitive sensory stimulation without requiring any active task from the patient, this protocol is referred to as coactivation or unattended activation-based learning : in one study (28).
All studies reported an improvement in ocular exploration after AVT, which allowed patients to efficiently compensate for the loss of vision with a clear advantage of the AVT in comparison to the visual-only exploration training. It has been found that a sound, spatially and temporally coincident to a visual stimulus, can improve visual perception in the blind hemifield of HH patients (24, 26).
Imaging studies in humans have confirmed the involvement of the SC and posterior cortical areas, including the temporo-parietal and posterior parietal cortices, in mediating AV multisensory integration (Dundon et al., 2015a). The studies suggested that, because most of the patients with damage to the visual cortex have an intact SC, it might be possible to train the use of retinotectal functions by AV stimulation in both chronic and acute phases after the stroke (12, 18, 24). Nevertheless, Ten Brink, Nijboer (29) indicated that saccade accuracy was affected only by visual stimuli in the intact, but not in the blind visual field for all HH patients participating in their study and only in one participant with a more limited quadrantanopia, was an enhancement in the oculomotor eye movement after the spatially coincident visual stimulus was observed. They concluded that multisensory integration is infrequent in the blind field of patients with hemianopia.
Conversely, in Jay and Sparks (30) study, trained monkeys made saccadic eye movements to auditory or visual targets while monitoring the activity of visual-motor (VM) cells and saccade-related burst (SRB) cells. The authors stated that “the SC is a site where sensory signals (either auditory or visual signals), originally encoded in different coordinates, converge and are translated into a common motor command: a command to correct for saccadic motor error”(30). This was based on the largely accepted basic hypothesis that sensory input from an intact modality (audition) can enhance processing of information by spared structures of a damaged one (vision). Lewald et al., (2012) assumed that AV bimodal neurons not only react to stimulus combinations and integrate information from different sensory inputs, but also can respond to unimodal stimuli, and provide a substrate for signalling in two separate modalities. It has been shown that one-time passive auditory stimulation on the side of the blind, but not of the intact, hemifield of patients with hemianopia induced an improvement in visual detections by almost 100% within 30 minutes after stimulation (28). The authors assumed that an activation of the surviving parts of the primary pathway and/or the colliculo-pulvinar-extrastriate pathway in HH may lead to an improvement of the related residual visual abilities in the blind field, either by more effective sensory processing of unimodal visual information within the residual pathway or by an increase of spatial attentional functions.
Head fixation and eye movement were monitored in all the studies. Five studies used an optic eye tracker (Eye-Track ASL-6000) and/or infrared video camera where the position of the subject's eye in the visual scene was monitored on-line by the experimenter (12, 15, 18, 27, 29). Two studies analysed the eye movement using electro-oculography (EOG) (18, 28). Fixation was monitored visually by the experimenter standing behind the apparatus in one study (26). Improvement in oculomotor exploration characterized by fewer fixations and refixations, quicker and larger saccades, reduction in scan path length and the mean exploration time was reported in five studies (12, 15, 18, 27, 28).
All studies demonstrated that the improvement found can be ascribed to compensatory behaviour as there was no significant difference observed in perceptual sensitivity when patients were not allowed to move the eyes (fixed-eye condition), emphasising that the treatment did not improve the scotoma in the visual field. This means that AVT is not a restorative treatment in nature.
Three articles evaluated the ADL for HH (total n=30 [8, 10, 12]) in the chronic stage, more than 3 months after stroke (12, 26, 27). A self-evaluation questionnaire containing 10-item, on a 5-point Likert scale was used including: 1, seeing obstacles; 2, bumping into objects/obstacles; 3, losing the way; 4, finding objects on the table; 5, finding objects in the room; 6, finding objects in the supermarket; 7, walking in a crowd; 8, reading; 9, to go upstairs/downstairs on the staircase; 10, crossing the streets. Evaluation of ADL for stroke survivors with HH in the subacute stage, between 3 and 24 weeks after stroke, have been reported in one study (n=20) (18). The questionnaire comprised only items that can be observed in an inpatient rehabilitation setting including: finding objects on the table, avoiding bumping into objects/persons, eye contact, seeing obstacles, and reading.
AVT promoted a significant reduction in subjective perceived disability according to the analysis of ADL for patients in both chronic and acute stages after brain damage, confirming a transfer of training effects to ecological environments. It has been indicated that while there was a significant improvement in the ADL results after the AVT, no difference was observed after a control VT, consisting of systematic visual stimulation of the visual field on the same apparatus as the AVT. The control VT was performed for 2 weeks before starting the AVT and for a similar amount of training time (4h/day) (12).
Visual scanning or visual search tests consisted mainly of three subtests: 1) The ‘E–F test’, where patients search for the letter F embedded among distractors, the letter E, 2) the ‘triangle test’ where patients reported the number of triangles embedded within square distractors with the same size and colour, and, 3) the ‘number test’ containing 15 numbers (from 1 to 15) randomly distributed over a black background from which the patient is asked to point to the numbers in an ascending order.
Findings provided by four articles (n=50, 32 males and 18 females) reported a significant improvement in visual search performance both in terms of accuracy and search times. Patients visual scanning behaviour became more efficient and faster, by comparing pre- and post- AVT tests, giving evidence that stimulating the SC, may induce a more organized pattern of visual exploration due to an implementation of efficient oculomotor strategies (12, 18, 26, 27). Passamonti, Bertini (12) observed significantly fewer fixations, saccadic duration was reduced, and mean saccadic amplitude was significantly increased in all comparisons before and after AVT. Additionally, in a study conducted by (18) the detection rate of target stimuli improved by about 46% in patients of the AVT group, whereas in patients of the VT group it only improved by 16%. This may suggest that the amelioration in visual perception induced by training is mostly mediated by the oculomotor system where patients can actively scan via eye movement (26), supporting the hypothesis suggested by (30) that auditory and visual signals have been translated into common coordinates at the level of the SC and share a motor circuit involved in the generation of saccadic eye movements.
Assessment of hemianopic dyslexia has been undertaken in four articles before and after the AVT (n=50, 32 males and 18 females). In one study the reading task was for single words only, which were presented in upper-case Italian letters (26). The other three studies examined the reading time and accuracy by using longer texts (8 to 20 lines) (12, 18, 27). Comparisons between VT and AVT revealed statistically significant differences in favour of bimodal training. The reading time for patients in the AVT group reduced from 177s before training to 75s after training. However, a slight reduction in the reading time was shown for patients in the VT group (from 195s initially to 175s post training) (18). In addition, Grasso, Ladavas (27) revealed a significant effect of the AVT on the reading speed of HH patients. Generally, according to a review on visual rehabilitation comparing multisensory stimulation and visual scanning, the reading performance improved in all patients after the AVT treatment period, reducing the ocular reading parameters for both progressive and regressive saccades (24). Lateralization effect on reading impairment was observed in regards to the affected hemifield in HH patients, by measuring five variables; number of saccades in the reading direction, number of regressions (backward saccades), number of saccades during the return sweep (additional to the one necessary to start a new line), mean duration of fixation, and amplitude of reading saccades, in only one study (12). For right HH, the saccadic amplitude increased and the fixation duration reduced during reading, with fewer errors, fewer progressive saccades and fewer regressions; while only the number of saccades during return sweep decreased in left HH (12). Left HH patients obtained an almost complete normalization of defective ocular responses, however, right HH patients still showed an impairment of the ocular responses, despite the clear benefit gained (12).
A similar study paradigm was used in two studies for the EEG assessment, in which patients performed a simple visual detection task. EEG data were recorded in both studies (n=18, 15 males and 3 females) (27, 31) at three time points; baseline (B1), 2 weeks after B1 and immediately before the AVT (control baseline B2), and after the AVT treatment (p). In addition, Grasso, Ladavas (27) included a follow-up session (f) 8 months after the treatment ceased (Figure 2).
The data was recorded from 27 electrode sites and the right mastoid. The left mastoid was used as reference, while the ground electrode was positioned on the right cheek (27, 31). P3 components were measured as the mean amplitude in a time window between 200 and 600ms after the presentation of the stimulus. In the chosen time window, scalp topography at B1 indicated a maximal positive inflection over electrodes CP1, P3 and Pz (27, 31). Therefore, data from these electrodes were used for statistical analysis. Dundon, Ladavas (31) computed the P3 amplitudes separately for the left and the right HH groups. The average value of electrodes that fell within each group’s zone of maximal P3 amplitude from the individual group topographies was used; i.e., Pz, P3 and CP1 for the right lesion group, P4, CP2, C4 and CP6 for the left. A reduction in P3 amplitude in response to stimuli presented in the intact field was reported in both studies, indicating reallocation of spatial attention resources after AVT (figure 3). The EEG results obtained by Grasso, Ladavas (27) and Dundon, Ladavas (31) showed that the mean P3 amplitude at sessions P (7.38µV) was significantly lower compared to the mean P3 amplitude at B1 (9.62 µV; p ˂ 0.05) and at B2 (9.435 µV; p ˂ 0.05). No significant difference, however, in P3 amplitude was recorded between B1 and B2 (figure 3). In the follow up session, Grasso, Ladavas (27) reported that the mean P3 amplitude at session F (7.99 µV) was also significantly lower than the mean P3 amplitudes at B1 and B2. A reduction in the intensity of cortical processing in the contralesional hemisphere manifests an improvement in the dynamic visual performance, specifically in the hemianopic field, which indicates attenuation of the allocation of attention towards the intact hemifield. Dundon, Ladavas (31) concluded that multisensory stimulation may significantly reduce the ipsilesional attentional bias in HH patients.
To test possible different contributions of the left and right hemisphere to the resulted P3 reduction in Dundon et al., (2015b) study, only the more lateralized electrodes CP1 and P3 were considered, with group as a between subject factor (left-lesion patients vs. right-lesion patients) and with electrode (CP1, P3), session (B1, B2, P) and position (upper, middle, lower) as within subject factors. A significant effect of session (p = 0.029) were found with no significant effect of group or any significant interaction between group and the other factors. These results suggested that no considerable difference of the P3 amplitude reduction was found between left and right HH patients in post AVT training. So that, the observed reduction in attention towards the intact hemifield, which might co-occur with a shift of spatial attention towards the blind field, happens similarly in both hemispheres (31).
Assessment of the intervention effects over a prolonged period of time is of importance to consider treatment effectiveness (25). Four articles (n=45, 28 males and 17 females) incorporated a follow-up test post AVT training in their design at a period between one month and one year (12, 15, 26, 27). There was a transfer of AVT treatment gains to functional measures assessing visual field exploration and to daily-life activities which were found to be stable at follow-up control sessions in all the studies, indicating a long-term persistence of treatment effects on the oculomotor system. These long-lasting effects according to Grasso et al., (2016) are most probably subserved by the activation of the spared retino-colliculo-dorsal pathway, which boosts orienting responses towards the blind field, increasing the ability to both compensate for the visual field loss and concurrently attenuate visual attention towards the intact field.
Nevertheless, in the study conducted by (28), ten patients with pure HH received one hour of passive auditory stimulation by application of repetitive trains of sound pulses. Immediately before and after the auditory stimulation as well as after a recovery period of 2 hours, they completed a simple visual task (see visual training vs audio-visual training section for more details). While the visual detection improved immediately post auditory activation, after the recovery period the enhancement in performance had returned to baseline, showing that the improvement in performance is not long lasting when passive auditory stimuli was used.
2- Audio-visual localization and space perception in hemianopia:
Four observational studies (n=43, 29 males and 14 females) investigated AV localization (32, 33) and the geometry of the visual space in HH using multisensory stimuli (34, 35). The former studies predicted that visual stimuli in the intact visual field would bias the auditory localization, so that sounds would be mislocated toward their visual source. On the contrary, they expected that in the blind field, where the occipital cortex damage had disrupted its’ underlaying neural circuit, this effect of bias would not occur (32). Both studies examined the cross-modal condition in which the auditory stimuli were presented with visual stimuli in either spatially coincident or spatially disparate. The results in the intact visual field were in line with the phenomenon known as the ventriloquism effect. In this effect, a presentation of auditory and visual stimuli that are temporally coincident and spatially disparate, might leads to mislocation of sounds toward their visual source (36). In the hemianopic field, however, no visual bias occurred when the two stimuli were spatially separated, which supports the key role of visual cortex for such an effect, so that, when the visual cortex has been damaged no visual bias was observed (32).
This is because the enhancement of auditory localization is expected via SC neurons, depending on the multisensory activation. It has been shown that visual stimuli affected auditory localization only when stimuli were spatially and temporally coincident, meaning that covert visual processes remain active in hemianopia (32). The authors explained the difference between the enhancement in multisensory stimulation and the visual bias as these two results are dependent on different neural pathways. The multisensory stimulation is dependent on circuits that involve the SC which facilitate orientation and localization of cues from multiple senses; and the visual bias is dependent on geniculo-striate-circuits that provide analysis of the visual scene (32). A similar result was obtained by (33), by comparing patients with HH and patients with neglect (n= 9, n=6 respectively). A consistent shift in sound localization toward the visual attractor was still evident in neglect patients but not in patients with HH, supporting the role of the geniculo-striate-circuits, which is damaged in HH, in such an effect.
The latter studies investigated the concept of how unilateral brain damage in HH can affect the perception of body orientation in space, leading to an attentional bias towards the contralesional field. Lewald, Peters (35) indicated that auditory spatial orientation in HH, without spatial neglect, was almost normal compared with healthy subjects. Thus, it was suggested that in multimodal space, visual brain areas, as are damaged in HH, are not directly involved in relating body position to the external space (35). Additionally, subjects were asked to match the location of a single visual target with an auditory marker or vice versa to estimate the potential distortions in the representation of visual space accompanied by HH (34). It has been highlighted that HH patients may exhibit distortion in both the visual and auditory space. However, in the bimodal approach, they would cancel each other out, and as a consequence, the cross-modal abilities might be preserved (34).
3- Audio-visual integration in patients with visual field defects:
The anatomical correlation of audio-visual integration was investigated by a comparison between patients with hemianopia and patients with spatial neglect in two studies (n=36, 22 males and 14 females) (13, 33). Both studies showed that after adaptation to spatially coincident AV stimuli, both HH patients and neglect patients exhibited significant reduction in auditory and visual localization errors. A possible explanation for these effects is the function of multisensory neurons in the SC, which can be activated when the stimuli from different sensory modalities at close spatial proximity interact (13). Thus, the results indicated that damaged brain areas (striate and parieto-temporal areas) in HH and neglect patients were not contributory in this specific form of perceptual learning (33). In other words, visual information is capable of calibrating auditory space, even without the involvement of those brain areas, as long as visual and auditory information are spatially coincident. Passamonti, Làdavas (33) found that adaptation to spatially disparate stimuli invokes the geniculo-striate circuit to correct and reduce the discrepancy registration However, adaptation to spatially aligned stimuli invoke the collicular-extrastriate circuit to reduce the localization errors. Therefore, the multisensory enhancement should be observed in both neglect and HH patients as the collicular-extrastriate circuit is spared in both patients (33).
By contrast, in patients suffering from hemianopia and neglect, multisensory integration did not occur (13) . It has been reported that integrative multisensory effect depends on the extension and/or the localization of the lesion. Lesions causing neglect are mainly confined to the fronto-temporal and parietal lobe (visuospatial attentional system) whilst lesions causing HH are mainly confined to the occipital lobe (the primary sensory visual system) and, for patients with both neglect and HH, the lesion could involve both areas (13). A possible explanation provided was that the influence of these cortical areas modulates the ability of SC to synthesize cross-modal inputs, preventing the cross-modal integration in patients with both deficits.
Childhood population
The possibility of inducing long-lasting amelioration after AVT in children with chronic HH due to acquired brain lesions was investigated by only one study (21). The study included three children (one male, two females aged between 9 and 17 years). The training duration was one and a half hours daily for 4 weeks. Outcome measures consisted of correct number of visual detections, visual search ability and reading speed. The visual search test consisted of six different subsets; the apple, frog, smile, E-F, triangles and numbers tests (21). Each subject was tested before and after the training period and after a follow-up period of one month, and in one case further follow-up was obtained after 12 months. The authors found a marked improvement in detections and response times only when subjects used explorative eye movements, but not with fixed eyes on a central point (21). This suggests that the enhancement in visual perception induced by training is mediated by the oculomotor system, reinforcing orientation towards the blind hemifield. For all the tests, the main factor session was significant when response times were considered (21).
Improvement in reading speed after training was observed for the single word reading performance for all subjects. The results of this study confirmed that AVT can also induce activation of visual responsiveness of the oculomotor system in children and adolescents with visual field deficits as the visual search behaviour became more efficient and faster after treatment. Tinelli, Purpura (21) argued that this manifests the important role of the multisensory integration especially the SC in this type of ocular compensation and in the plasticity of the visual system in the presence of ‘blindsight i.e. residual visual capacity but without acknowledged perceptual awareness after lesions of the striate cortex’ even when the occipital cortex is completely damaged. Long-lasting effect of the treatment was reported in both one and 12 months follow-up tests (21).