Differences in auditory-visual speech perception between Japanese and Americans: McGurk effect as a function of incompatibility.

要旨

In English speaking cultures, it has been reported that when auditory speech is presented in synchrony with discrepant visual (lip-read) speech, the subjects often report hearing sounds that integrate information from the two modalities (the “McGurk effect”). However, Sekiyama and Tohkura recently showed that Japanese subjects are less influenced by discrepant visual cues than Americans. This study examined the interlanguage differences in terms of the relationship between the magnitude of the McGurk effect and the frequency of incompatibility. The stimulus materials were ten syllables (/ba, pa, ma, wa, da, ta, na, ga, ka, ra/) pronounced by a Japanese and an American speaker. The ten auditory and ten visual syllables pronounced by a speaker were cross-dubbed resulting in 100 auditory-visual stimuli. Japanese syllables were presented to 14 Japanese and 10 American subjects. English syllables were presented to different groups of subjects, 12 Japanese and 10 American. The stimuli were presented in both quiet and noise-added conditions. The subjects were asked to check incompatibility between what they heard and what they saw as well as to report what they heard. The results showed that the magnitude of the McGurk effect correlated highly negatively with the frequency of incompatibility. It was suggested that the small McGurk effect in the Japanese subjects listening to Japanese speech is due to the higher sensitivity of the Japanese to auditory-visual discrepancy.

主題