Encoding of Visual Speaker Attributes and Recognition Memory for Spoken Words

Author: Helena M. Saldaña, Lynne C. Nygaard, and David B. Pisoni

Abstract:
An experiment was designed to assess the extent to which visual articulatory information is encoded in memory. In a recent study, Palmeri, Goldinger, and Pisoni (1993) reported that listeners were more accurate at recognizing previously presented words when they were presented in the same voices as at test than when they were presented in a different voice. this result suggests that detailed voice information is not stripped away by a normalization process during the early stages of spoken word recognition; instead information about the talker's voice is encoded into long-term memory and may later facilitate recognition of spoken words. the present investigation was designed to determine whether detailed cross-modal linguistic information is also retained in long-term memory. Subjects were presented with two audio-visual speakers producing lists of isolated words. The words were presented at three signal-to-noise ratios ranging from+5 dB to -5 dB. In the low signal-to-noise conditions, listeners are forced to attend more carefully to the visual information presented in order to extract the linguistic content. It is proposed that this manipulation will cause dynamic visual information to be encoded along with the spoken words in memory. If it is the case that detailed visual information about the talkers articulation is retained in long term memory, them we would expect that listeners will be better at recognizing "old" words when they are presented with the same visual speaker at test. No significant effect of visual articulation was found on recognition memory, however, a consistent trend was observed. The present results have implications for current theories of spoken word recognition and the nature of the representation of words stored in the mental lexicon.