Unique semantic space in the brain of each beholder predicts perceived similarity

The unique way in which each of us perceives the world must arise from our brain representations. If brain imaging could reveal an individual’s unique mental representation, it could help us understand the biological substrate of our individual experiential worlds in mental health and disease. However, imaging studies of object vision have focused on commonalities between individuals rather than individual differences and on category averages rather than representations of particular objects. Here we investigate the individually unique component of brain representations of particular objects with functional MRI (fMRI). Subjects were presented with unfamiliar and personally meaningful object images while we measured their brain activity on two separate days. We characterized the representational geometry by the dissimilarity matrix of activity patterns elicited by particular object images. The representational geometry remained stable across scanning days and was unique in each individual in early visual cortex and human inferior temporal cortex (hIT). The hIT representation predicted perceived similarity as reflected in dissimilarity judgments. Importantly, hIT predicted the individually unique component of the judgments when the objects were personally meaningful. Our results suggest that hIT brain representational idiosyncrasies accessible to fMRI are expressed in an individual''s perceptual judgments. The unique way each of us perceives the world thus might reflect the individually unique representation in high-level visual areas.Everyone’s perception of the world is unique. Psychologists and psychotherapists, using methods including questionnaires and free association, have long attempted to peer into an individual’s subjective experiential world. The unique aspects of our experience coexist with a shared experiential component. We can all recognize the objects that surround us and name them in a common language. Consistent with this shared component of experience, there is evidence that visual stimuli are processed similarly in the brains of different individuals (1). However, the unique way in which each of us perceives an object also must arise from brain activity. Is there an individually unique component to our brain representations?Unidimensional aspects of subjective visual percepts, ranging from estimates of object size, color, vividness, and emotional valence, have separately been found to correlate with interindividual variation in both univariate regional-average activation and cortical anatomy (2, 3). However, it remains unclear how a person’s multidimensional subjective percept reflects the multivariate brain-activity pattern that represents a particular object.Functional magnetic resonance imaging (fMRI) studies of object vision have focused largely on commonalities among subjects and category averages across particular stimuli. These studies have revealed regions in human inferior temporal cortex (hIT) that preferentially respond to specific categories (4–9) as well as widely distributed category information (10). More recently, similarity analyses of response patterns to particular stimulus images have revealed exemplar-specific representations (11–14), clustering of response patterns by natural categories (15–17), and reinstatement of neural representations during memory recall (18, 19). These prior studies either tacitly assumed similar representations across individuals or explicitly demonstrated commonalities between individuals and even between species (14, 20–29).Previous studies have shown that the hIT representation has a semantic component (23) and is reflected in perception at the level of group averages (30). Here we tested the hypothesis that an individual’s hIT representation predicts idiosyncrasies in his or her perception of natural objects. Because of hIT’s reciprocal connections to the memory regions of the medial temporal lobe (31), we further predicted that personally meaningful objects elicit individually unique mnemonic associations and are more distinctly represented in each individual.We presented familiar and unfamiliar object images to subjects during fMRI and investigated whether early visual cortex (EVC) and hIT exhibit individually unique representations. We characterized the representational geometry of each region by the dissimilarity matrix of activity patterns elicited by particular object images. This matrix is called the “representational dissimilarity matrix” (RDM) (16). To address whether the detailed representational geometries are reflected in behavior, we tested whether individual idiosyncrasies in similarity judgments can be predicted on the basis of a subject’s brain RDM.