Why Are We Fooled by the Ventriloquist?

By Deric Bownds @DericBownds

As we watch the movement's of a dummy's mouth while it is sitting in a ventriloguist's lap, we perceive the speech as coming from the dummy's mouth, rather than it's master's voice. Berger and Ehrsson show that this illusory translocation is associated with increased activity the left superior temporal sulcus (L. STS). This is the region that has been shown to be central in determining the spatial coordinates of our experienced self. (It is associated also, for example, with the out of body illusion.)

It is well understood that the brain integrates information that is provided to our different senses to generate a coherent multisensory percept of the world around us, but how does the brain handle concurrent sensory information from our mind and the external world? Recent behavioral experiments have found that mental imagery—the internal representation of sensory stimuli in one's mind—can also lead to integrated multisensory perception; however, the neural mechanisms of this process have not yet been explored. Here, using functional magnetic resonance imaging and an adapted version of a well known multisensory illusion (i.e., the ventriloquist illusion), we investigated the neural basis of mental imagery-induced multisensory perception in humans. We found that simultaneous visual mental imagery and auditory stimulation led to an illusory translocation of auditory stimuli and was associated with increased activity in the left superior temporal sulcus (L. STS), a key site for the integration of real audiovisual stimuli. This imagery-induced ventriloquist illusion was also associated with increased effective connectivity between the L. STS and the auditory cortex. These findings suggest an important role of the temporal association cortex in integrating imagined visual stimuli with real auditory stimuli, and further suggest that connectivity between the STS and auditory cortex plays a modulatory role in spatially localizing auditory stimuli in the presence of imagined visual stimuli.