In the early stages of learning to speak, infants and young children focus on a person’s face when that person is talking to them. They get as much information from the way the person’s face looks and moves as from the words themselves – more, in fact, since word understanding is usually extremely limited at first. The tone of voice and whether a person looks angry or happy will provide more clues than the actual words. In many ways, we do not lose this method of comprehension as we get older, even when we understand the words. If you inform someone with a cheery smile and a laughing tone that your house just burned down, they’ll think you’re joking, because your words don’t match your tone or the expression on your face. In fact, they might not even understand what you’ve said, and ask you to repeat yourself. Students of a foreign language know how important it is to watch a native speaker’s mouth as they pronounce new vocabulary words, especially if the new language has different phonic sounds than their own.

This tendency to look for visual as well as audible clues when communicating can sometimes cause problems, however, when what we see doesn’t match what we hear. In a phenomenon called “the McGurk Effect,” researchers have proved that in some cases what we see actually changes what we hear. They’re incorporating the results of their studies into new technology for speech recognition software. Take a look at this video of a BBC program, and see how susceptible you are to this phenomenon.

When you’re speaking to someone, it’s important to remember that communication happens through the eyes as well as the ears.