Memory-based detection of task-irrelevant visual changes

Colored grating patterns were presented to 8 participants in a passive oddball condition (standard, 87.5% and deviant, 12.5%, differing in their color). In the corresponding multicolor condition, grating patterns of eight different colors were presented, their probabilities set equal both to each other and to that of the deviant in the oddball task. Compared with the ERP response elicited by the standard stimulus, the deviant response was negatively displaced over posterior areas, the difference wave peaking at 136 ms. A similar negative wave was obtained when the ERP response to the deviant was compared with the ERP elicited by the same stimulus in the multicolor condition. This result rules out stimulus- (color-) specific refractoriness as a major factor in the generation of the deviance-related posterior negativity. The observed posterior negativity can therefore be regarded as a visual analog of the mismatch negativity (vMMN).     

Impairment in processing visual information at the pre-attentive stage in patients with a major depressive disorder: A visual mismatch negativity study

We compared the pre-attentive processing in patients with a major depressive disorder (MDD) and matched healthy controls as indexed by the visual mismatch negativity (vMMN) elicited by exposure duration of visual stimuli randomly presented on both peripheral visual fields. To obtain the memory-comparison-based visual MMN, the role of standard and deviant stimuli was reversed in separate blocks. Compared with healthy participants, MDD patients exhibited decreased MMN amplitudes of long duration deviant only and this deficit was not correlated with the depression severity. These data suggests functional impairment of pre-attentive basic visual information processing in MDD patients.     

Can illusory deviant stimuli be used as attentional distractors to record vMMN in a passive three stimulus oddball paradigm?

A passive three stimulus oddball paradigm was used to investigate Visual Mismatch Negativity (vMMN) a component of the Event Related Potential (ERP) believed to represent a central pre-attentive change mechanism. Responses to a change in orientation were recorded to monochrome stimuli presented to subjects on a computer screen. One of the infrequent stimuli formed an illusory figure (Kanizsa Square) aimed to capture spatial attention in the absence of an active task. Nineteen electrodes (10–20 system) were used to record the electroencephalogram in fourteen subjects (ten females) mean age 34.5 years. ERPs to all stimuli consisted of a positive negative positive complex recorded maximally over lateral occipital areas. The negative component was greater for deviant and illusory deviant compared to standard stimuli in a time window of 170–190 ms. A P3a component over frontal/central electrodes to the illusory deviant but not to the deviant stimulus suggests the illusory figure was able to capture attention and orientate subjects to the recording. Subtraction waveforms revealed visual discrimination responses at occipital electrodes, which may represent vMMN. In a control study with 13 subjects (11 females; mean age 29.23 years), using an embedded active attention task, we confirmed the existence of an earlier (150–170 ms) and attenuated vMMN. Recordings from an intracranial case study confirmed separation of N1 and discrimination components to posterior and anterior occipital areas, respectively. We conclude that although the illusory figure captured spatial attention in its own right it did not draw sufficient attentional resources from the standard–deviant comparison as revealed when using a concurrent active task.     

Visual mismatch negativity: New evidence from the equiprobable paradigm

In visual oddball studies, deviant compared to standard stimuli elicited a posterior negative ERP at around 100–250 ms. To determine the underlying processes of the negativity, we used the equiprobable sequence in which bar stimuli of five types of orientation were presented with equal probabilities (control 20% each) as well as the oddball sequence in which two stimuli with the closest orientation were presented with different probabilities (deviant 20% and standard 80%). Deviant compared to standard stimuli elicited two negativities at around 100–150 ms with no hemispheric dominance and 200–250 ms with right hemispheric dominance, while deviant compared to control stimuli elicited only a negativity at around 200–250 ms with right hemispheric dominance. These results suggest that the early negativity reflects refractory effect, while the late negativity reflects memory-comparison-based change detection effect (visual mismatch negativity).     

Unnoticed regularity violation elicits change-related brain activity

Event-related brain electric activity (ERP) was investigated to unnoticed visual changes. The orientation of grid elements (vertical or horizontal) changed after the presentation of 10–15 identical stimuli. The grid patterns were task irrelevant, and were presented in the background of a shape discrimination task. During the first half of the session, participants were unaware of the stimulus change. However, in comparison to the ERPs to the fifth identical stimuli, stimulus change elicited posterior negativities in the 270–375 ms range (visual mismatch negativity, vMMN). With participants instructed on the stimulus change, negativities emerged with earlier onset and with wider distribution. When stimulus change was preceded by only two identical stimuli, there were no such ERP effects. As the results show, a longer sequence of identical unattended stimuli may establish the memory representation of stimulus regularity, and violation of regularity is indicated by posterior negative ERP components (vMMN).     

Brain activity from stimuli that are not perceived: Visual mismatch negativity during binocular rivalry suppression

Predictive coding explains visual perception as the result of an interaction between bottom‐up sensory input and top‐down generative models at each level of the visual hierarchy. Evidence for this comes from the visual mismatch negativity (vMMN): a more negative ERP for rare, unpredictable visual stimuli—deviants, than for frequent, predictable visual stimuli—standards. Here, we show that the vMMN does not require conscious experience. We measured the vMMN from monocular luminance‐decrement deviants that were either perceived or not during binocular rivalry dominance or suppression, respectively. We found that both sorts of deviants elicited the vMMN at about 250 ms after stimulus onset, with perceived deviants eliciting a bigger vMMN than not‐perceived deviants. These results show that vMMN occurs in the absence of consciousness, and that consciousness enhances the processing underlying vMMN. We conclude that generative models of visual perception are tested, even when sensory input for those models is not perceived.     

ERPs and deviance detection: visual mismatch negativity to repeated visual stimuli

Previous studies showed a visual analogue of the mismatch negativity (MMN) event-related potential (ERP), which is elicited by violating some sensory regularity. Separating physical change from violating a regularity, here we show that the visual MMN (vMMN) is elicited by regularity violations that do not involve physical stimulus change. Adult participants were presented with a series of red-black and green-black checkerboard patterns delivered regularly in an RRGGRRGG ... order. Infrequently (p=0.1) this regularity was broken by repeating a stimulus one additional time (e.g. RRGGRRR). ERPs elicited by irregular stimulus repetitions were negatively displaced compared to those elicited both by regular repetition and regular change in two latency ranges: 100-140 ms and 220-260 ms. Whereas the first of these two negative ERP differences appears to be sensitive to stimulus repetition per se, the second difference can be identified as a vMMN response to violating the sequential regularity. Thus, similarly to its auditory counterpart, vMMN reflects deviance-(regularity-violation) rather than change-detection processes.     

Magnetic Source Localization of Early Visual Mismatch Response

Previous studies have reported a visual analogue of the auditory mismatch negativity (MMN) response that is based on sensory memory. The neural generators and attention dependence of the visual MMN (vMMN) still remain unclear. We used magnetoencephalography (MEG) and spatio-temporal source localization to determine the generators of the sensory-memory-based vMMN response to non-attended deviants. Ten participants were asked to discriminate between odd and even digits presented at the center of the visual field while grating patterns with different spatial frequencies were presented outside the focus of attention. vMMN was calculated as the difference between MEG responses to infrequent gratings in oddball blocks and the same gratings in equiprobable blocks. The peak latency of the vMMN response was between 100 and 160 ms. The neuromagnetic sources of the vMMN localized in the occipital cortex differed from the sources evoked by the equiprobable gratings and were stimulus-dependent. Our results suggest the existence of separate neural systems for pre-attentive memory-based detection of visual change and provide new evidence that the vMMN is feature-specific.     

Color categories affect pre-attentive color perception

Categorical perception (CP) of color is the faster and/or more accurate discrimination of colors from different categories than equivalently spaced colors from the same category. Here, we investigate whether color CP at early stages of chromatic processing is independent of top-down modulation from attention. A visual oddball task was employed where frequent and infrequent colored stimuli were either same- or different-category, with chromatic differences equated across conditions. Stimuli were presented peripheral to a central distractor task to elicit an event-related potential (ERP) known as the visual mismatch negativity (vMMN). The vMMN is an index of automatic and pre-attentive visual change detection arising from generating loci in visual cortices. The results revealed a greater vMMN for different-category than same-category change detection when stimuli appeared in the lower visual field, and an absence of attention-related ERP components. The findings provide the first clear evidence for an automatic and pre-attentive categorical code for color.     

Deviance detection in congruent audiovisual speech: Evidence for implicit integrated audiovisual memory representations

Detection of deviant speech syllables embedded in continuous noise was investigated in an oddball paradigm. Behavioral results showed improvement of detecting and identifying the syllables when congruent visual speech accompanied the utterances. A centrally maximal negative ERP difference wave peaking at approximately 290 ms post-stimulus was elicited by audiovisual but not by auditory- or visual-only task-irrelevant deviant syllables. Whereas the circumstances of the elicitation of this ERP response are similar to those of the mismatch negativity component (MMN and its visual counterpart, vMMN), its scalp distribution differs from that of both unimodal MMNs. Elicitation of an MMN-like ERP response (termed here as the audiovisual MMN: avMMN) suggests that detection of the audiovisual deviants involved integrated audiovisual memory representations. The pattern of behavioral and ERP results suggest that the formation of such cross-modal memory representation does not require voluntary operations and may even proceed for stimuli outside the focus of attention.     

Dysfunction of processing task-irrelevant emotional faces in major depressive disorder patients revealed by expression-related visual MMN

We compared the pre-attentive processing of emotional faces in major depressive disorder (MDD) patients and matched healthy controls as indexed by the visual mismatch negativity (vMMN) elicited by emotional faces presented in upright and inverted orientations. Compared with standard neutral faces, infrequent emotional faces elicited posterior negativities between 100 ms and 350 ms. Early vMMN was smaller, while late vMMN was absent in the MDD group, as compared with the normal group. Face inversion decreased the vMMN in the normal group but had no effect on MDD patients. These data suggest the functional dysfunction of pre-attentive expressional information processing in MDD patients, coupled with the disorder of high-level perceptual strategy of processing emotional faces.     

The quest for the genuine visual mismatch negativity (vMMN): Event-related potential indications of deviance detection for low-level visual features

Research shows that the visual system monitors the environment for changes. For example, a left-tilted bar, a deviant, that appears after several presentations of a right-tilted bar, standards, elicits a classic visual mismatch negativity (vMMN): greater negativity for deviants than standards in event-related potentials (ERPs) between 100 and 300 ms after onset of the deviant. The classic vMMN is contributed to by adaptation; it can be distinguished from the genuine vMMN that, through use of control conditions, compares standards and deviants that are equally adapted and physically identical. To determine whether the vMMN follows similar principles to the auditory mismatch negativity (MMN), in two experiments we searched for a genuine vMMN from simple, physiologically plausible stimuli that change in fundamental dimensions: orientation, contrast, phase, and spatial frequency. We carefully controlled for attention and eye movements. We found no evidence for the genuine vMMN, despite adequate statistical power. We conclude that either the genuine vMMN is a rather unstable phenomenon that depends on still-to-be-identified experimental parameters, or it is confined to visual stimuli for which monitoring across time is more natural than monitoring over space, such as for high-level features. We also observed an early deviant-related positivity that we propose might reflect earlier predictive processing.     

Finding the missing stimulus mismatch negativity (MMN): emitted MMN to violations of an auditory gestalt

Deviations from repetitive auditory stimuli evoke a mismatch negativity (MMN). Counterintuitively, omissions of repetitive stimuli do not. Violations of patterns reflecting complex rules also evoke MMN. To detect a MMN to missing stimuli, we developed an auditory gestalt task using one stimulus. Groups of six pips (50 ms duration, 330 ms stimulus onset asynchrony [SOA], 400 trials), were presented with an intertrial interval (ITI) of 750 ms while subjects (n=16) watched a silent video. Occasional deviant groups had missing 4th or 6th tones (50 trials each). Missing stimuli evoked a MMN (p<.05). The missing 4th (-0.8 μV, p<.01) and the missing 6th stimuli (-1.1 μV, p<.05) were more negative than standard 6th stimuli (0.3 μV). MMN can be elicited by a missing stimulus at long SOAs by violation of a gestalt grouping rule. Patterned stimuli appear more sensitive to omissions and ITI than homogenous streams.     

Visual change detection: event-related potentials are dependent on stimulus location in humans

Infrequent colored patterns within sequences of patterns of frequent color elicited a posterior negative event-related potential component only in case of lower half-field stimulation. This negativity in the 140-200 ms latency range is considered as a correlate of automatic visual change detection (visual mismatch negativity, vMMN). Retinotopic prestriate visual areas are suggested to be the generating loci of vMMN.     

Event-Related Potentials and the Identification of Deviant Visual Stimuli

The effects of deviant visual stimuli on event-related potentials were investigated in a counting task (Experiment 1) and in a reaction time task (Experiment 2). In Experiment 2 the interstimulus interval was either short or long (340 vs. 1020 ms). The stimuli (two angles within a frame) were frequent (Standard) or deviant (differing from the Standard either in the orientation of the two angles, or in the thickness of the frame, or in both of these features). In various conditions the target stimuli were defined by one of the deviant features or by the conjunction of these features. Subjects were more accurate in the counting task, and the reaction time was shortest when the target feature was the deviant angle orientation. Performance was lowest for the conjunction of the deviant features. The deviant angle orientation elicited a posterior negative wave in the 140-180 ms range. As the interstimulus interval increased, the magnitude of this component decreased. All stimuli with relevant (attended) deviant features elicited another posterior negative wave in the 180-260 ms range, as well as an anterior positivity with similar latency. When the interstimulus interval was short, and the only target was the Conjunction Deviant, the summed occipital activity to the relevant features of deviant nontarget stimuli was larger than the negativity to the Conjunction Deviant. Target stimuli elicited late positive waves, which were sometimes preceded by central negativity.     

Underlying mechanisms of the P3a task-difficulty effect

In three-stimulus oddball studies, even typical deviant stimuli elicited a large P3a event-related brain potential (ERP) when target/standard discrimination was difficult. To investigate the underlying mechanisms, the effects of task difficulty on early deviant-related ERPs were assessed. Four visual stimuli defined by an orthogonal combination of task-relevant size (nontarget 80%, target 20%) and task-irrelevant luminance (standard 80%, deviant 20%) were presented randomly, where two task difficulties (easy, difficult) were defined by target/nontarget discriminability. An increase in task difficulty enhanced P3a as well as a posterior negativity (change-related negativity) and an anterior positivity (frontal positivity) elicited by deviant nontarget stimuli. These results suggest that attentional modulation of refractoriness-based rareness detection and an attention-triggering process underlie the P3a task-difficulty effect.     

A Visual Analog of Mismatch Negativity When Stimuli Differ in Duration

Event-linked brain potentials were studied in ten essentially healthy volunteers (six men, four women) aged 18–24 years. Subjects were presented with rare deviant and frequent standard visual stimuli in the standard odd-ball paradigm in conditions of active attention to stimulation and in conditions of distracted attention. Differences between deviant stimuli (50, 100, and 150 msec) and standard (200 msec) stimuli ranged from 50 to 150 msec. Negative difference waves (deviant stimulus with distracted attention minus control stimulus of the same duration) were seen at differences of 100 and 150 msec and these were most marked in the right temporal cortex 200–400 msec from stimulus onset. As this wave was seen in conditions of distracted attention, it appears to be a visual analog of mismatch negativity and reflects the process of the automatic detection of stimuli with deviant durations.     

Probability-independent and -dependent ERPs reflecting visual change detection

In ERP studies, two posterior components with different polarities have been identified as ERP correlates of visual change detection. To compare these components in terms of sensitivity to the preceding stimulus sequence, two peripheral stimuli of different colors (red and blue) were presented with equal (50:50) or different probabilities (20:80 or 80:20), while 12 participants performed shape discrimination at a central location. A posterior positivity at around 90-140 ms was observed with similar amplitude to all stimuli immediately preceded by a different stimulus. In contrast, a posterior negativity at around 140-180 ms was observed to increase in amplitude with increasing number of preceding different stimuli. These results suggest the existence of probability-independent and -dependent change processing in the human visual system. The functional significance is discussed in terms of memory-based comparison and stimulus-specific refractoriness.     

Short‐term trained lexical categories affect preattentive shape perception: Evidence from vMMN

Perceptual processing of colors and shapes in the right visual field is modulated by the lexical category information of the stimuli, a phenomenon known as the lateralized Whorfian effect. For color stimuli, lateralized Whorfian effect is characterized by preattentive occurrence and dependency on acquired lexical information, but it remains unknown whether these key features are generalizable to other domains of perceptual processing. Here, we investigated whether lateralized Whorfian effect in the shape perception domain also depends on acquired lexical category and occurs preattentively using ERPs. Participants were trained to associate novel, irregular polygons with lexical category labels via short‐term intensive training. Using the visual oddball paradigm, we found stronger visual mismatch negativity (vMMN) component elicited by the deviant stimuli whose lexical category differed from the standard stimuli when the deviant was presented in the right visual field, indicating higher perceptual conspicuity for between‐category stimuli. These findings provide direct evidence of similar preattentive lexical category‐contingent modulation on shape perception akin to color perception, suggesting that the lateralized Whorfian effect is not epiphenomenal but rather might reflect the interaction between higher‐level lexical processing and the lower‐level perceptual processing more broadly.     

Separate memory-related processing for auditory frequency and patterns.

Detecting deviant, and potentially meaningful, auditory events depends on transient representations of preceding stimuli. Here, we examined whether the neural circuitry underlying deviance detection system varied as a function of deviance type. In different blocks of trials, participants were presented with a sequence that included standard and deviant tones differing in frequency or a sequence of tones that alternated regularly in frequency with occasional deviant repetitions. Both frequency- and pattern-deviant stimuli elicited a mismatch negativity (MMN) peaking between 120 and 175 ms poststimulus. The MMN amplitude distribution was more frontal for frequency-deviant than for pattern-deviant stimuli. There are two possible explanations for these results. Both frequency- and pattern-deviation MMNs might arise in the same set of generators whose relative strength of activation varies. Alternatively, frequency- and pattern-deviation MMNs could originate in different generators. These alternatives were investigated using principal component analysis and signal identification methods. These methods revealed that no common signal space could account for both of the MMNs, indicating different generator sources for the analysis of frequency and pattern deviance. The results suggest separate memory-related processing for auditory frequency and patterns and indicate that the neural circuit of deviance detection varies as a function of the perceptual context.