From the Cover: Whole-agent selectivity within the macaque face-processing system

The primate brain contains a set of face-selective areas, which are thought to extract the rich social information that faces provide, such as emotional state and personal identity. The nature of this information raises a fundamental question about these face-selective areas: Do they respond to a face purely because of its visual attributes, or because the face embodies a larger social agent? Here, we used functional magnetic resonance imaging to determine whether the macaque face patch system exhibits a whole-agent response above and beyond its responses to individually presented faces and bodies. We found a systematic development of whole-agent preference through the face patches, from subadditive integration of face and body responses in posterior face patches to superadditive integration in anterior face patches. Superadditivity was not observed for faces atop nonbody objects, implying categorical specificity of face–body interaction. Furthermore, superadditivity was robust to visual degradation of facial detail, suggesting whole-agent selectivity does not require prior face recognition. In contrast, even the body patches immediately adjacent to anterior face areas did not exhibit superadditivity. This asymmetry between face- and body-processing systems may explain why observers attribute bodies’ social signals to faces, and not vice versa. The development of whole-agent selectivity from posterior to anterior face patches, in concert with the recently described development of natural motion selectivity from ventral to dorsal face patches, identifies a single face patch, AF (anterior fundus), as a likely link between the analysis of facial shape and semantic inferences about other agents.Making sense of the world around us is a computationally daunting challenge and one of the brain’s great feats. The face patch system of the macaque monkey (ref. 1; Fig. 1A) is a uniquely valuable model for understanding how the brain constructs a meaningful representation of the visual world (2). This network of areas along the superior temporal sulcus (STS) of the temporal lobe responds selectively to the sight of faces, parses their visual properties, and supports their perception (3, 4). However, face recognition is just one part of a greater perceptual goal. Faces are singularly rich sources of social information: They convey the identities (5), emotions (6), and intentions (7) of entire agents. This unique aspect of faces raises a critical question: Do face patches respond to a face because they are fundamentally selective for facial shapes, or because these areas represent the larger agent embodied by the face?Open in a separate windowFig. 1.Experimental plan. (A) Schematic of the macaque face and body patches along the STS of a right hemisphere, with the anterior temporal pole on the right. (B) Three hypothetical responses of a model face patch to a face and body viewed together. Red and blue bars indicate the response of the patch to lone faces and headless bodies, respectively. Purple outlines show possible whole-agent response signatures given the responses to face and body. A strictly selective response to whole monkeys would be equivalent to the response to isolated faces, an additive response would equal the sum of the responses to individually presented faces and bodies, and a superadditive response would exceed this sum. Note that whereas a strictly selective or additive response could be created by a mix of selective face and body cells, a superadditive response is highly indicative of joint representation of face and body by single neurons (18).Currently, it is thought that two distinct temporal lobe networks—the face patches and a parallel system of body-selective patches (8–10)—separately process the visual features of social agents. However, the sight of a body can alter the perception of a face (11), and some neurons in the upper STS have been found to integrate face and body information (12). Although face patch neurons respond poorly to images of headless bodies (3), an untested possibility exists that they integrate information from a paired face and body to represent an agent as a whole.Learning whether face patches represent whole agents would address the larger question of where the machinery of facial perception transitions into the machinery of social cognition—the ability to combine posture, motion, and other social information to understand and interact with others (13, 14). One indication that this transition may occur within the face system is that dorsal patches are specifically sensitive to the naturalness of facial motion (15), an important quality for social understanding (16). Integration of face and body is an ideal signature with which to confirm whether social processing begins in the face patches because, unlike traits such as “theory of mind” that are challenging to test with precision (17), it is easily operationalized. The hypothesis that dorsal face areas represent social agents, and not merely facial features, predicts a ventral-to-dorsal development of whole-agent integration across the temporal lobe. In contrast, the prevailing view that face patches are strictly face-selective modules predicts that adding a body to a face will produce no effect in any patch. Identifying how each face patch responds to whole agents would test these two possibilities, and clarify if and how social knowledge evolves within the face-processing system.In this study, we set out to determine whether macaque face or body patches integrate face and body information to represent whole animals. We did this by pairing pictures of faces with pictures of bodies or matched nonbody objects (Fig. S1), then using functional magnetic resonance imaging (fMRI) to observe the response signatures that these stimuli evoked (Fig. 1B). If face patches were strictly category-selective for faces, they would respond to pictures of whole monkeys just as much as they responded to faces alone. If, alternatively, face patches were composed of independently face- and body-selective cells, their response would be additive, the sum of the separate responses to lone faces and lone bodies. However, a superadditive response to whole monkeys would argue against these models of strict category selectivity and for synergistic whole-animal processing within the face patches (18). By observing whether the whole-animal responses differed from the responses to faces on top of nonbody objects, we determined the selectivity of face patches’ responses to whole animals. Finally, we tested whether the face patches integrate the form of whole animals even in the absence of identifiable facial features, observing how they responded to visually obscured faces paired with bodies and nonbody objects. Comparing the results of these three experiments, we could determine whether face patches may be strictly concerned with the visual details of faces, or whether they sometimes process social information gathered from across an agent.Open in a separate windowFig. S1.Sample stimuli from the whole agent stimulus set. These stimuli represent all of the stimuli derived from the picture of a single monkey; the full stimulus set contained 14 additional examples of each stimulus type. Objects presented in the lower visual field were chosen to match the overall shape and size of the bodies. The empty/empty condition was presented in baseline blocks.