Shape information used for face identity and expression recognition is highly versatile and context specific.

We recognize faces every day to help us gauge social situations, facilitate communication, and retain relationships. A common goal in face perception research is to understand what specific face features are important during such tasks, and how shape information specific to one task (e.g., identity recognition) is influenced by that specific to another (e.g., expression recognition). We addressed these questions using reverse correlation in an anatomically-interpretable face shape space. Participants viewed pairs of randomly sampled faces and were tasked to select the face closest to a target identity or expression. Through averaging chosen noise patterns, we obtained an estimate of the target’s representation, or template, conducted tests to recover face features significantly associated with recognition, and compared templates across changes in face context. We find that recognition of identity and expression depend on subtle changes in shape distributed across the face, and that the shape information is flexible and specific to contextual changes. In particular, emotional expression changes our representation of identity, and face identity changes our representation of emotional expression. These results provide insight into the adaptability of the visual system’s use of sensory information.