Non-invasive brain–computer interface system: Towards its application as assistive technology

The quality of life of people suffering from severe motor disabilities can benefit from the use of current assistive technology capable of ameliorating communication, house-environment management and mobility, according to the user's residual motor abilities. Brain–computer interfaces (BCIs) are systems that can translate brain activity into signals that control external devices. Thus they can represent the only technology for severely paralyzed patients to increase or maintain their communication and control options.Here we report on a pilot study in which a system was implemented and validated to allow disabled persons to improve or recover their mobility (directly or by emulation) and communication within the surrounding environment. The system is based on a software controller that offers to the user a communication interface that is matched with the individual's residual motor abilities. Patients (n = 14) with severe motor disabilities due to progressive neurodegenerative disorders were trained to use the system prototype under a rehabilitation program carried out in a house-like furnished space. All users utilized regular assistive control options (e.g., microswitches or head trackers). In addition, four subjects learned to operate the system by means of a non-invasive EEG-based BCI. This system was controlled by the subjects’ voluntary modulations of EEG sensorimotor rhythms recorded on the scalp; this skill was learnt even though the subjects have not had control over their limbs for a long time.We conclude that such a prototype system, which integrates several different assistive technologies including a BCI system, can potentially facilitate the translation from pre-clinical demonstrations to a clinical useful BCI.     

Non-invasive brain-computer interface system: towards its application as assistive technology

The quality of life of people suffering from severe motor disabilities can benefit from the use of current assistive technology capable of ameliorating communication, house-environment management and mobility, according to the user's residual motor abilities. Brain-computer interfaces (BCIs) are systems that can translate brain activity into signals that control external devices. Thus they can represent the only technology for severely paralyzed patients to increase or maintain their communication and control options. Here we report on a pilot study in which a system was implemented and validated to allow disabled persons to improve or recover their mobility (directly or by emulation) and communication within the surrounding environment. The system is based on a software controller that offers to the user a communication interface that is matched with the individual's residual motor abilities. Patients (n=14) with severe motor disabilities due to progressive neurodegenerative disorders were trained to use the system prototype under a rehabilitation program carried out in a house-like furnished space. All users utilized regular assistive control options (e.g., microswitches or head trackers). In addition, four subjects learned to operate the system by means of a non-invasive EEG-based BCI. This system was controlled by the subjects' voluntary modulations of EEG sensorimotor rhythms recorded on the scalp; this skill was learnt even though the subjects have not had control over their limbs for a long time. We conclude that such a prototype system, which integrates several different assistive technologies including a BCI system, can potentially facilitate the translation from pre-clinical demonstrations to a clinical useful BCI.     

Emulation of computer mouse control with a noninvasive brain-computer interface

Brain-computer interface (BCI) technology can provide nonmuscular communication and control to people who are severely paralyzed. BCIs can use noninvasive or invasive techniques for recording the brain signals that convey the user's commands. Although noninvasive BCIs are used for simple applications, it has frequently been assumed that only invasive BCIs, which use electrodes implanted in the brain, will be able to provide multidimensional sequential control of a robotic arm or a neuroprosthesis. The present study shows that a noninvasive BCI using scalp-recorded electroencephalographic (EEG) activity and an adaptive algorithm can provide people, including people with spinal cord injuries, with two-dimensional cursor movement and target selection. Multiple targets were presented around the periphery of a computer screen, with one designated as the correct target. The user's task was to use EEG to move a cursor from the center of the screen to the correct target and then to use an additional EEG feature to select the target. If the cursor reached an incorrect target, the user was instructed not to select it. Thus, this task emulated the key features of mouse operation. The results indicate that people with severe motor disabilities could use brain signals for sequential multidimensional movement and selection.     

Brain-computer interfaces for communication and control.

For many years people have speculated that electroencephalographic activity or other electrophysiological measures of brain function might provide a new non-muscular channel for sending messages and commands to the external world - a brain-computer interface (BCI). Over the past 15 years, productive BCI research programs have arisen. Encouraged by new understanding of brain function, by the advent of powerful low-cost computer equipment, and by growing recognition of the needs and potentials of people with disabilities, these programs concentrate on developing new augmentative communication and control technology for those with severe neuromuscular disorders, such as amyotrophic lateral sclerosis, brainstem stroke, and spinal cord injury. The immediate goal is to provide these users, who may be completely paralyzed, or 'locked in', with basic communication capabilities so that they can express their wishes to caregivers or even operate word processing programs or neuroprostheses. Present-day BCIs determine the intent of the user from a variety of different electrophysiological signals. These signals include slow cortical potentials, P300 potentials, and mu or beta rhythms recorded from the scalp, and cortical neuronal activity recorded by implanted electrodes. They are translated in real-time into commands that operate a computer display or other device. Successful operation requires that the user encode commands in these signals and that the BCI derive the commands from the signals. Thus, the user and the BCI system need to adapt to each other both initially and continually so as to ensure stable performance. Current BCIs have maximum information transfer rates up to 10-25bits/min. This limited capacity can be valuable for people whose severe disabilities prevent them from using conventional augmentative communication methods. At the same time, many possible applications of BCI technology, such as neuroprosthesis control, may require higher information transfer rates. Future progress will depend on: recognition that BCI research and development is an interdisciplinary problem, involving neurobiology, psychology, engineering, mathematics, and computer science; identification of those signals, whether evoked potentials, spontaneous rhythms, or neuronal firing rates, that users are best able to control independent of activity in conventional motor output pathways; development of training methods for helping users to gain and maintain that control; delineation of the best algorithms for translating these signals into device commands; attention to the identification and elimination of artifacts such as electromyographic and electro-oculographic activity; adoption of precise and objective procedures for evaluating BCI performance; recognition of the need for long-term as well as short-term assessment of BCI performance; identification of appropriate BCI applications and appropriate matching of applications and users; and attention to factors that affect user acceptance of augmentative technology, including ease of use, cosmesis, and provision of those communication and control capacities that are most important to the user. Development of BCI technology will also benefit from greater emphasis on peer-reviewed research publications and avoidance of the hyperbolic and often misleading media attention that tends to generate unrealistic expectations in the public and skepticism in other researchers. With adequate recognition and effective engagement of all these issues, BCI systems could eventually provide an important new communication and control option for those with motor disabilities and might also give those without disabilities a supplementary control channel or a control channel useful in special circumstances.     

A high performance sensorimotor beta rhythm-based brain-computer interface associated with human natural motor behavior

To explore the reliability of a high performance brain-computer interface (BCI) using non-invasive EEG signals associated with human natural motor behavior does not require extensive training. We propose a new BCI method, where users perform either sustaining or stopping a motor task with time locking to a predefined time window. Nine healthy volunteers, one stroke survivor with right-sided hemiparesis and one patient with amyotrophic lateral sclerosis (ALS) participated in this study. Subjects did not receive BCI training before participating in this study. We investigated tasks of both physical movement and motor imagery. The surface Laplacian derivation was used for enhancing EEG spatial resolution. A model-free threshold setting method was used for the classification of motor intentions. The performance of the proposed BCI was validated by an online sequential binary-cursor-control game for two-dimensional cursor movement. Event-related desynchronization and synchronization were observed when subjects sustained or stopped either motor execution or motor imagery. Feature analysis showed that EEG beta band activity over sensorimotor area provided the largest discrimination. With simple model-free classification of beta band EEG activity from a single electrode (with surface Laplacian derivation), the online classifications of the EEG activity with motor execution/motor imagery were: >90%/ approximately 80% for six healthy volunteers, >80%/ approximately 80% for the stroke patient and approximately 90%/ approximately 80% for the ALS patient. The EEG activities of the other three healthy volunteers were not classifiable. The sensorimotor beta rhythm of EEG associated with human natural motor behavior can be used for a reliable and high performance BCI for both healthy subjects and patients with neurological disorders. Significance: The proposed new non-invasive BCI method highlights a practical BCI for clinical applications, where the user does not require extensive training.     

A brain-computer interface using electrocorticographic signals in humans

Brain-computer interfaces (BCIs) enable users to control devices with electroencephalographic (EEG) activity from the scalp or with single-neuron activity from within the brain. Both methods have disadvantages: EEG has limited resolution and requires extensive training, while single-neuron recording entails significant clinical risks and has limited stability. We demonstrate here for the first time that electrocorticographic (ECoG) activity recorded from the surface of the brain can enable users to control a one-dimensional computer cursor rapidly and accurately. We first identified ECoG signals that were associated with different types of motor and speech imagery. Over brief training periods of 3-24 min, four patients then used these signals to master closed-loop control and to achieve success rates of 74-100% in a one-dimensional binary task. In additional open-loop experiments, we found that ECoG signals at frequencies up to 180 Hz encoded substantial information about the direction of two-dimensional joystick movements. Our results suggest that an ECoG-based BCI could provide for people with severe motor disabilities a non-muscular communication and control option that is more powerful than EEG-based BCIs and is potentially more stable and less traumatic than BCIs that use electrodes penetrating the brain.     

A large clinical study on the ability of stroke patients to use an EEG-based motor imagery brain-computer interface

Brain-computer interface (BCI) technology has the prospects of helping stroke survivors by enabling the interaction with their environ ment through brain signals rather than through muscles, and restoring motor function by inducing activity-dependent brain plasticity. This paper presents a clinical study on the extent of detectable brain signals from a large population of stroke patients in using EEG-based motor imagery BCI. EEG data were collected from 54 stroke patients whereby finger tapping and motor imagery of the stroke-affected hand were performed by 8 and 46 patients, respectively. EEG data from 11 patients who gave further consent to perform motor imagery were also collected for second calibration and third independent test sessions conducted on separate days. Off-line accuracies of classifying the two classes of EEG from finger tapping or motor imagery of the stroke-affected hand versus the EEG from background rest were then assessed and compared to 16 healthy subjects. The mean off-line accuracy of detecting motor imagery by the 46 patients (mu=0.74) was significantly lower than finger tapping by 8 patients (mu=0.87, p=0.008), but not significantly lower than motor imagery by healthy subjects (mu=0.78, p=0.23). Six stroke patients performed motor imagery at chance level, and no correlation was found between the accuracies of detecting motor imagery and their motor impairment in terms of Fugl-Meyer Assessment (p=0.29). The off-line accuracies of the 11 patients in the second session (mu=0.76) were not significantly different from the first session (mu=0.72, p=0.16), or from the on-line accuracies of the third independent test session (mu=0.82, p=0.14). Hence this study showed that the majority of stroke patients could use EEG-based motor imagery BCI.     

Signals from intraventricular depth electrodes can control a brain-computer interface

A brain-computer interface (BCI) is a device that enables severely disabled people to communicate and interact with their environments using their brain waves. Most research investigating BCI in humans have used scalp-recorded electroencephalography (EEG). We have recently demonstrated that signals from intracranial electrocorticography (ECoG) and stereotactic depth electrodes (SDE) in the hippocampus can be used to control a BCI P300 Speller paradigm. We report a case in which stereotactic depth electrodes positioned in the ventricle were able to obtain viable signals for a BCI. Our results demonstrate that event-related potentials from intraventricular electrodes can be used to reliably control the P300 Speller BCI paradigm.     

EEG-based brain-computer interfaces: an overview of basic concepts and clinical applications in neurorehabilitation

Some patients are no longer able to communicate effectively or even interact with the outside world in ways that most of us take for granted. In the most severe cases, tetraplegic or post-stroke patients are literally 'locked in' their bodies, unable to exert any motor control after, for example, a spinal cord injury or a brainstem stroke, requiring alternative methods of communication and control. But we suggest that, in the near future, their brains may offer them a way out. Non-invasive electroencephalogram (EEG)-based brain-computer interfaces (BCI) can be characterized by the technique used to measure brain activity and by the way that different brain signals are translated into commands that control an effector (e.g., controlling a computer cursor for word processing and accessing the internet). This review focuses on the basic concepts of EEG-based BCI, the main advances in communication, motor control restoration and the downregulation of cortical activity, and the mirror neuron system (MNS) in the context of BCI. The latter appears to be relevant for clinical applications in the coming years, particularly for severely limited patients. Hypothetically, MNS could provide a robust way to map neural activity to behavior, representing the high-level information about goals and intentions of these patients. Non-invasive EEG-based BCIs allow brain-derived communication in patients with amyotrophic lateral sclerosis and motor control restoration in patients after spinal cord injury and stroke. Epilepsy and attention deficit and hyperactive disorder patients were able to downregulate their cortical activity. Given the rapid progression of EEG-based BCI research over the last few years and the swift ascent of computer processing speeds and signal analysis techniques, we suggest that emerging ideas (e.g., MNS in the context of BCI) related to clinical neurorehabilitation of severely limited patients will generate viable clinical applications in the near future.     

Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface

People with severe motor disabilities can maintain an acceptable quality of life if they can communicate. Brain-computer interfaces (BCIs), which do not depend on muscle control, can provide communication. Four people severely disabled by ALS learned to operate a BCI with EEG rhythms recorded over sensorimotor cortex. These results suggest that a sensorimotor rhythm-based BCI could help maintain quality of life for people with ALS.     

Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface

ObjectiveThis study investigates the impact of a continuously presented visual feedback in the form of a grasping hand on the modulation of sensorimotor EEG rhythms during online control of a brain–computer interface (BCI).MethodsTwo groups of participants were trained to use left or right hand motor imagery to control a specific output signal on a computer monitor: the experimental group controlled a moving hand performing an object-related grasp (‘realistic feedback’), whereas the control group controlled a moving bar (‘abstract feedback’). Continuous feedback was realized by using the outcome of a real-time classifier which was based on EEG signals recorded from left and right central sites.ResultsThe classification results show no difference between the two feedback groups. For both groups, ERD/ERS analysis revealed a significant larger ERD during feedback presentation compared to an initial motor imagery screening session without feedback. Increased ERD during online BCI control was particularly found for the lower alpha (8–10 Hz) and for the beta bands (16–20, 20–24 Hz).ConclusionsThe present study demonstrates that visual BCI feedback clearly modulates sensorimotor EEG rhythms. When the feedback provides equivalent information on both the continuous and final outcomes of mental actions, the presentation form (abstract versus realistic) does not influence the performance in a BCI, at least in initial training sessions.SignificanceThe present results are of practical interest for classifier development and BCI use in the field of motor restoration.     

Brain-computer interfaces (BCIs): detection instead of classification

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer through brain-computer interfaces (BCIs). These devices operate by recording signals from the brain and translating these signals into device commands. They can be used by people who are severely paralyzed to communicate without any use of muscle activity. One of the major impediments in translating this novel technology into clinical applications is the current requirement for preliminary analyses to identify the brain signal features best suited for communication. This paper introduces and validates signal detection, which does not require such analysis procedures, as a new concept in BCI signal processing. This detection concept is realized with Gaussian mixture models (GMMs) that are used to model resting brain activity so that any change in relevant brain signals can be detected. It is implemented in a package called SIGFRIED (SIGnal modeling For Real-time Identification and Event Detection). The results indicate that SIGFRIED produces results that are within the range of those achieved using a common analysis strategy that requires preliminary identification of signal features. They indicate that such laborious analysis procedures could be replaced by merely recording brain signals during rest. In summary, this paper demonstrates how SIGFRIED could be used to overcome one of the present impediments to translation of laboratory BCI demonstrations into clinically practical applications.     

Investigating the effects of visual distractors on the performance of a motor imagery brain-computer interface

Objectives

Brain-computer interfaces (BCIs) allow users to operate a device or application by means of cognitive activity. This technology will ultimately be used in real-world environments which include the presence of distractors. The purpose of the study was to determine the effect of visual distractors on BCI performance.

A Voting Optimized Strategy Based on ELM for Improving Classification of Motor Imagery BCI Data

This paper presents an approach to classifying electroencephalogram (EEG) signals for brain–computer interfaces (BCI). To eliminate redundancy in high-dimensional EEG signals and reduce the coupling among different classes of EEG signals, we use principle component analysis and linear discriminant analysis to extract features that represent the raw signals. Next, we introduce the voting-based extreme learning machine to classify the features. Experiments performed on real-world data from the 2003 BCI competition indicate that our classification method outperforms state-of-the-art methods in speed and accuracy.     

Neuronal mechanisms underlying control of a brain-computer interface

Brain-computer interfaces (BCIs) enable humans or animals to communicate or control external devices without muscle activity using electric brain signals. The BCI used here is based on self-regulation of slow cortical potentials (SCPs), a skill that most people and paralyzed patients can acquire with training periods of several hours up to months. The neurophysiological mechanisms and anatomical sources of SCPs and other event-related brain potentials have been described but the neural mechanisms underlying the self-regulation skill for the use of a BCI are unknown. To uncover the relevant areas of brain activation during regulation of SCPs, the BCI was combined with functional magnetic resonance imaging. The electroencephalogram was recorded inside the magnetic resonance imaging scanner in 12 healthy participants who learned to regulate their SCP with feedback and reinforcement. The results demonstrate activation of specific brain areas during execution of the brain regulation skill allowing a person to activate an external device; a successful positive SCP shift compared with a negative shift was closely related to an increase of the blood oxygen level-dependent response in the basal ganglia. Successful negativity was related to an increased blood oxygen level-dependent response in the thalamus compared with successful positivity. These results may indicate learned regulation of a cortico-striatal-thalamic loop modulating local excitation thresholds of cortical assemblies. The data support the assumption that human subjects learn the regulation of cortical excitation thresholds of large neuronal assemblies as a prerequisite for direct brain communication using an SCP-driven BCI. This skill depends critically on an intact and flexible interaction between the cortico-basal ganglia-thalamic circuits.     

Classification of selective attention to auditory stimuli: toward vision-free brain-computer interfacing

Brain-computer interface (BCI) is a developing, novel mode of communication for individuals with severe motor impairments or those who have no other options for communication aside from their brain signals. However, the majority of current BCI systems are based on visual stimuli or visual feedback, which may not be applicable for severe locked-in patients that have lost their eyesight or the ability to control their eye movements. In the present study, we investigated the feasibility of using auditory steady-state responses (ASSRs), elicited by selective attention to a specific sound source, as an electroencephalography (EEG)-based BCI paradigm. In our experiment, two pure tone burst trains with different beat frequencies (37 and 43 Hz) were generated simultaneously from two speakers located at different positions (left and right). Six participants were instructed to close their eyes and concentrate their attention on either auditory stimulus according to the instructions provided randomly through the speakers during the inter-stimulus interval. EEG signals were recorded at multiple electrodes mounted over the temporal, occipital, and parietal cortices. We then extracted feature vectors by combining spectral power densities evaluated at the two beat frequencies. Our experimental results showed high classification accuracies (64.67%, 30 commands/min, information transfer rate (ITR) = 1.89 bits/min; 74.00%, 12 commands/min, ITR = 2.08 bits/min; 82.00%, 6 commands/min, ITR = 1.92 bits/min; 84.33%, 3 commands/min, ITR = 1.12 bits/min; without any artifact rejection, inter-trial interval = 6s), enough to be used for a binary decision. Based on the suggested paradigm, we implemented a first online ASSR-based BCI system that demonstrated the possibility of materializing a totally vision-free BCI system.     

Decoding two-dimensional movement trajectories using electrocorticographic signals in humans

Signals from the brain could provide a non-muscular communication and control system, a brain-computer interface (BCI), for people who are severely paralyzed. A common BCI research strategy begins by decoding kinematic parameters from brain signals recorded during actual arm movement. It has been assumed that these parameters can be derived accurately only from signals recorded by intracortical microelectrodes, but the long-term stability of such electrodes is uncertain. The present study disproves this widespread assumption by showing in humans that kinematic parameters can also be decoded from signals recorded by subdural electrodes on the cortical surface (ECoG) with an accuracy comparable to that achieved in monkey studies using intracortical microelectrodes. A new ECoG feature labeled the local motor potential (LMP) provided the most information about movement. Furthermore, features displayed cosine tuning that has previously been described only for signals recorded within the brain. These results suggest that ECoG could be a more stable and less invasive alternative to intracortical electrodes for BCI systems, and could also prove useful in studies of motor function.     

Long-term evaluation of a 4-class imagery-based brain–computer interface

ObjectiveThe study aimed to improve brain–computer interface (BCI)-usability by using distinct control strategies and evaluating performance, brain activity and psychological variables on a long-term basis over several months.MethodsFourteen able-bodied users participated in 10 sessions, plus a follow-up session. Users were trained to control an EEG-based 4-class BCI with the mental tasks, word association, mental subtraction, spatial navigation, and motor imagery.ResultsEight users reached mean accuracies of 61–72% and managed to control all 4 classes above chance in single-sessions. Performance and brain patterns stayed stable over 10 weeks without training. Motor imagery showed the best performance and most distinct brain patterns. Participants’ fear of incompetence decreased while the quality of their imagery and task ease increased over sessions. The evaluation of feedback differed between tasks and correlated with performance.ConclusionUsers can control a real-time 4-class BCI, driven by distinct mental tasks, with stable performance over months. However, general performance was rather low for effective BCI control in daily life. Possibilities for future optimizations to increase performance are discussed.SignificanceThe evaluation of alternatives to motor imagery, long-term BCI use, and psychological variables is important to improve usability for mental imagery-based BCIs.     

Learning to move machines with the mind

Brain-computer interfaces (BCIs) extract signals from neural activity to control remote devices ranging from computer cursors to limb-like robots. They show great potential to help patients with severe motor deficits perform everyday tasks without the constant assistance of caregivers. Understanding the neural mechanisms by which subjects use BCI systems could lead to improved designs and provide unique insights into normal motor control and skill acquisition. However, reports vary considerably about how much training is required to use a BCI system, the degree to which performance improves with practice and the underlying neural mechanisms. This review examines these diverse findings, their potential relationship with motor learning during overt arm movements, and other outstanding questions concerning the volitional control of BCI systems.     

Towards adaptive classification for BCI

Non-stationarities are ubiquitous in EEG signals. They are especially apparent in the use of EEG-based brain-computer interfaces (BCIs): (a) in the differences between the initial calibration measurement and the online operation of a BCI, or (b) caused by changes in the subject's brain processes during an experiment (e.g. due to fatigue, change of task involvement, etc). In this paper, we quantify for the first time such systematic evidence of statistical differences in data recorded during offline and online sessions. Furthermore, we propose novel techniques of investigating and visualizing data distributions, which are particularly useful for the analysis of (non-)stationarities. Our study shows that the brain signals used for control can change substantially from the offline calibration sessions to online control, and also within a single session. In addition to this general characterization of the signals, we propose several adaptive classification schemes and study their performance on data recorded during online experiments. An encouraging result of our study is that surprisingly simple adaptive methods in combination with an offline feature selection scheme can significantly increase BCI performance.