Nonverbal synchrony, media, and emotion

Jun, H., & Bailenson, J. (2021). Nonverbal synchrony, media, and emotion. In Routledge International Handbook of Emotions and Media (pp. 303-315). Routledge. doi:10.4324/9780429465758


Media often have a nonverbal component to it. Whether using simple emoticons attached to text messages, or facial expressions which are conveyed via videoconferences, the success of mediated interpersonal interactions often relies on understanding the emotions of others. While understanding the psychological mechanisms behind emotion are intricately complex, most people can effortlessly recognize the emotional state of others, and when appropriate, generate an empathetic response. This empathic ability is assisted by our ability to automatically simulate other people’s behavior and emotions (Iacoboni, 2009). The so-called “mirror-neurons” that scholars have isolated support the existence of this automatic ability. When a person sees others exhibiting a particular behavior or emotion, these neurons respond as if the person is experiencing such behavior or emotion. For example, babies cry when they see other babies cry (De Vignemont & Singer, 2006).
Long before the discovery of mirror neurons, communication researchers demonstrated that people in interpersonal interactions follow temporal patterns. The timing of verbal and
nonverbal messages conveys critical information, above and beyond the content of those messages. If messages fall within a specific temporal pattern of communication, researchers have described the timing as synchrony. As researchers seek to understand both the role of emotion and communication, examining the mechanisms behind mirroring behavior provides insights into the process and outcomes of interpersonal interactions.
Synchrony was first documented by Condon and Ogston (1966). In their pioneering work, they filmed two subjects talking to each other and then meticulously studied individual
film frames to observe subjects’ nonverbal behavior. The authors documented a consistent timing between utterances and nonverbal behavior. According to Condon and Ogston, “Nonverbal behavior displays a consistent harmony between speech and body motion which suggests a highly integrated organism” (p. 345). They argued that this coordinated behavior occurs automatically, as opposed to a process about which participants were aware. Building on their work, Kendon (1970) studied interpersonal communication in a larger group, and confirmed that movements of listeners were coordinated to the speech and movements of the speaker.