Have you read me yet?

by Paula Parungao

So, if you've read the pas blog entries, you'll find out that visual cues like lip reading actually helps us understand speech. You'd also hear about the McGurk effect, which are illusory speech sounds that arise when there is a discrepancy between the auditory stimuli and the visual stimuli. If you haven't heard of these topics yet, I suggest you go back and read the blog entries that revolve around these terms. Sayang naman effort namin, diba? If you've already read about them in our entries, good for you! I give you an 1.0 as my favorite blog reader.

Moving on. What I'm gonna talk about is more on the last topic just discussed in the previous paragraph: the McGurk effect. When studies had been replicated concerning this illusion, it revolved mainly on nonsense words like "baba" or "meme". But what about words that are actually in an individual's vocabulary? Would they be messed up too? We know that known words can affect phonetic identification in auditory speech perception. The aim of the current study, was to investigate lexical influences on the McGurk Effect using three letter consonant-vowel-consonant (CVC) words (real and pseudo), and whether the lexical influences on the McGurk Effect are dependent on the positioning of the audiovisual discrepancy prior to a vowel or following a vowel.

Okay, so for the experiment, they let participants watch a video of a female enunciating CVC's of 10 real words and 10 pseudowords. The consonants used for the effect were /b/ over the lip movements of /g/. (If you don't get the method, I really suggest you read our previous blog entries. :) ) Three control conditions were used: visual only, auditory only, and audiovisual matching. The participants were made to watch, listen, or both, to the clips for about and hour and twenty minutes. Imagine having to listen to a girl say three letter CVC's over and over again. My head would probably implode. Thankfully, the heads of the participants stayed intact long enough for the researchers to obtain the following resu

lts:

1) Accuracy of heard speech was greatest for audiovisual matching stimuli (which is consistent with the study that said that visual cues help us discern speech).

2) McGurk effect still works (hooray for replications!)

3) For real words, percentages of the responses consistent with the McGurk effect were significantly greater when audiovisual discrepancy was placed at the onset (before vowel) than offset (after vowel) of real words.

4) For pseudowords, nada. Discrepancies placed at the onset or offset had no influence on the responses.

5) Participants often reported the phonetic equivalents of the visual components of discrepant audiovisual stimuli even though they were only instructed to report what they heard. This means that visual interference on auditory speech perception has a pretty heavy influence.

So what does this show? Basically, in the end, the McGurk Effect wins. Even when real words are used, how they are presented to the individual overrides their vocabulary. (eg. in the experiment, "bet" became "zet"; wth is "zet"?) Also, the effect is dependent on the positioning of audiovisual discrepancy. BUT there was a reduction in the frequency of these responses which suggest that when sufficient information is available, word knowledge does have an influence on auditory phonetic identification.

This study shows how much influence our internal vocabulary, or our lexicon, has on us. Even though we may be presented with words that don't make sense, given the proper information, we may perceive these words as how internally, we know them. This is increasingly helpful especially in moments where individuals mumble or seemingly don't make sense. By piecing together the right cues, we may be able to communicate properly even when presented with such erroneous stimuli.

Reference:

Barutchu, A., Crewther, S.G., et. al. (2008). When /b/ill with /g/ill becomes /d/ill: Evidence for a lexical effect in audiovisual speech perception. European Journal of Cognitive Psychology, 20(I), 1-11.