Will I speak louder if I see you struggling to understand? Speech modifications in response to non-verbal visual cues of listening effort.

During face-to-face interactions, overt non-verbal cues from conversational partners can reflect the quality of the conversation. Here, we investigated the impact of specific visual non-verbal cues (i.e., facial expressions of confusion, approaching movements towards the speaker), on the speakers' acoustic-phonetic adaptations typically employed to enhance clarity (i.e., speech rate, voice intensity, and pitch). Participants read sentences to a listener seated in a separate room and visible through a head-mounted display. Crucially, no auditory feedback was provided. Unknown to participants, the listener was a confederate whose behavior was pre-recorded in silence using a 360° video camera. The confederate simulated three different levels of listening effort: easy, medium, and hard. After each trial, participants rated the estimated listening effort and comprehension of the confederate, as well as their own speaking effort. In Experiment 1 (N = 22), participants were told their voice was transmitted with varying background noise levels, providing a rationale for potential listener difficulties. In Experiment 2 (N = 12), no mention of background noise was made, to further isolate the effects of visual cues. In both experiments, participants adjusted their speech to the seen listener's behavior. Experiment 1 showed adaptations across all measured parameters, and significant modulations persisted for most parameters in Experiment 2. These adaptations were already present in the earliest phase of exposure to the visual effort cues and remained stable across the mini-block, suggesting a rapid and highly sensitive monitoring mechanism. These findings indicate that visible non-verbal cues of listening effort can elicit acoustic-phonetic adaptations, even in the absence of acoustic feedback or explicit communicative signals, highlighting their role in regulating communication and maintaining speech clarity.