The neural computations underlying real-world social interaction recognition
Abstract: Humans perceive the world in rich social detail. We effortlessly recognize not only objects and people in our environment, but also social interactions between people. The ability to perceive and understand social interactions is critical for functioning in our social world. We recently identified a brain region that selectively represents others’ social interactions in the posterior superior temporal sulcus (pSTS) in a manner that is distinct from other visual and social processes, like face recognition and theory of mind. However, it is unclear how social interactions are processed in the real world where they co-vary with many other sensory and social features. In the first part of my talk, I will discuss new work using naturalistic movie fMRI paradigms and novel machine learning analyses to understand how humans process social interactions in real-world settings. We find that social interactions and are selectively processed in the pSTS, even after controlling for the effects of other co-varying perceptual and social information, including faces, voices, and theory of mind. In the second part of my talk, I will discuss the computational implications of social interaction selectivity in the brain, and present a novel graph neural network model, SocialGNN, that instantiates these insights. SocialGNN reproduces human social interaction judgements in both controlled and natural videos using only visual information, without any explicit model of agents’ minds or the physical world, but requires relational, graph structure and processing to do so. Together, this work suggests that social interaction recognition is a core human ability that relies on specialized, structured visual representations.
This seminar series is devoted to highlighting innovative, disruptive advances in cognitive neuroscience. Innovators in Cognitive Neuroscience also is dedicated to leveraging science as a vehicle for social justice. Through this seminar series we hope to recognize outstanding research conducted by historically underrepresented groups (HURG) in Cognitive Neuroscience and related fields, including women, BIPOC, LGBTQ+, and people/individuals with disabilities. Innovators in Cognitive Neuroscience is a formal collaboration between Dartmouth College, University of Pennsylvania, Yale University, Massachusetts Institute of Technology, Princeton University, Harvard University, Columbia University, and Gallaudet University. It is funded by the Center for Cognitive Neuroscience at Dartmouth College. ASL interpretation is provided for all ICN talks.