Task context impacts visual object processing differentially across the cortex

Perception reflects an integration of “bottom-up” (sensory-driven) and “top-down” (internally generated) signals. Although models of visual processing often emphasize the central role of feed-forward hierarchical processing, less is known about the impact of top-down signals on complex visual representations. Here, we investigated whether and how the observer’s goals modulate object processing across the cortex. We examined responses elicited by a diverse set of objects under six distinct tasks, focusing on either physical (e.g., color) or conceptual properties (e.g., man-made). Critically, the same stimuli were presented in all tasks, allowing us to investigate how task impacts the neural representations of identical visual input. We found that task has an extensive and differential impact on object processing across the cortex. First, we found task-dependent representations in the ventral temporal and prefrontal cortex. In particular, although object identity could be decoded from the multivoxel response within task, there was a significant reduction in decoding across tasks. In contrast, the early visual cortex evidenced equivalent decoding within and across tasks, indicating task-independent representations. Second, task information was pervasive and present from the earliest stages of object processing. However, although the responses of the ventral temporal, prefrontal, and parietal cortex enabled decoding of both the type of task (physical/conceptual) and the specific task (e.g., color), the early visual cortex was not sensitive to type of task and could only be used to decode individual physical tasks. Thus, object processing is highly influenced by the behavioral goal of the observer, highlighting how top-down signals constrain and inform the formation of visual representations.Perception reflects not only the external world but also our internal goals and biases. Even the simplest actions and decisions about visual objects require a complex integration between “top-down” (internally generated) and “bottom-up” (sensory-driven) signals (1). For example, the information used for object categorization depends on top-down signals arising from the spatial (2) or conceptual (3) context in which the object appears, the prior experience of the observers (4, 5), and the specific task (6, 7). Despite such strong behavioral evidence, the neural correlates of this integration remain unclear, both in terms of the cortical regions involved and the extent of the integration within those regions. Here, we investigate the impact of diverse behavioral goals on the neural architecture that supports object processing.Object recognition is known to depend on the ventral visual pathway, a set of interconnected cortical regions extending from early visual areas (e.g., V1/V2) into the anterior inferotemporal cortex (8). It has been argued that object processing along this pathway can largely be captured in feed-forward hierarchical frameworks without the need for top-down signals (9–12). For example, in the HMAX model (10), the integration of top-down signals is largely constrained to the extrinsic targets of the pathway (8), and in particular the lateral prefrontal cortex (LPFC) (13–17). However, there is strong evidence that top-down signals, such as attention and task, modulate the magnitude of response to simple visual stimuli (e.g., gratings) in early visual areas (18–23) and the response to objects in extrastriate regions (24–30).Although these prior studies provide evidence of an effect of top-down signals on object processing, they afford only limited insight because they tested only the modulation of overall activity and not the impact of top-down signals on fine-grained object information available in the response. The importance of this distinction between gross modulation versus fine-grained information is apparent in functional MRI (fMRI) investigations of working memory, where not all regions that evidence activity modulations contain information about the maintained objects (31, 32). Crucially, quantifying object information allows for a direct test of whether object representations are task-independent (equivalent information within and across tasks) or task-dependent (reduced information across compared with within tasks). Without this test, it remains unclear whether top-down signals, such as task, fundamentally alter the representations of objects or simply scale the response to them.To investigate the full range of task effects, we presented a broad set of objects in six separate tasks, half of which probed physical properties of the stimulus (e.g., color: red/blue) and half its conceptual properties (e.g., content: manmade/natural). This paradigm overcomes a limitation of previous studies, which often treated task and stimulus as simple dichotomous variables (26–30), making it difficult to generalize beyond the limited range of tasks and objects tested. Furthermore, previous studies often manipulated only whether an object was attended or not, and therefore could not establish how different types of information are extracted from the same attended stimuli. In contrast, by presenting an identical set of object images under multiple tasks, all requiring attention to the images, and extracting the response to each combination of task and object, we were able to directly test the effect of task on object responses.Our results revealed that task context has a pervasive effect on visual representations throughout the early visual cortex (EVC), the ventral visual pathway, and the LPFC. Task modulated both response magnitude and multivariate response patterns throughout these regions. Critically, responses in the ventral object-selective as well as the LPFC, were task-dependent, with reduced object information across tasks compared with within task. In contrast, object information in the EVC was task-independent, despite large task-related activity modulations. Together, these findings demonstrate that top-down signals directly contribute to and constrain visual object representations in the ventral object-selective and LPFC. Such effects strongly support a recurrent, highly interactive view of visual object processing within the ventral visual pathway that contrasts with many primarily bottom-up frameworks (9, 10).