Robust Nonlinear Autoregressive Moving Average Model Parameter Estimation Using Stochastic Recurrent Artificial Neural Networks

In this study, we introduce a new approach for estimating linear and nonlinear stochastic autoregressive moving average (ARMA) model parameters, given a corrupt signal, using artificial recurrent neural networks. This new approach is a two-step approach in which the parameters of the deterministic part of the stochastic ARMA model are first estimated via a three-layer artificial neural network (deterministic estimation step) and then reestimated using the prediction error as one of the inputs to the artificial neural networks in an iterative algorithm (stochastic estimation step). The prediction error is obtained by subtracting the corrupt signal of the estimated ARMA model obtained via the deterministic estimation step from the system output response. We present computer simulation examples to show the efficacy of the proposed stochastic recurrent neural network approach in obtaining accurate model predictions. Furthermore, we compare the performance of the new approach to that of the deterministic recurrent neural network approach. Using this simple two-step procedure, we obtain more robust model predictions than with the deterministic recurrent neural network approach despite the presence of significant amounts of either dynamic or measurement noise in the output signal. The comparison between the deterministic and stochastic recurrent neural network approaches is furthered by applying both approaches to experimentally obtained renal blood pressure and flow signals. © 1999 Biomedical Engineering Society. PAC99: 8710+e, 8719Uv, 0705Mh     

Application of fast orthogonal search to linear and nonlinear stochastic systems

Standard deterministic autoregressive moving average (ARMA) models consider prediction errors to be unexplain able noise sources. The accuracy of the estimated ARMA model parameters depends on producing minimum prediction errors. In this study, an accurate algorithm is developed for estimating linear and nonlinear stochastic ARMA model parameters by using a method known as fast orthogonal search, with an extended model containing prediction errors as part of the model estimation process. The extended algorithm uses fast orthogonal search in a two-step procedure in which deterministic terms in the non-linear difference equation model are first identified and then reestimated, this time in a model containing the prediction errors. Since the extended algorithm uses an orthogonal procedure, together with automatic model order selection criteria, the significant model terms are estimated efficiently and accurately. The model order selection criteria developed for the extended algorithm are also crucial in obtaining accurate parameter estimates. Several simulated examples are presented to demonstrate the efficacy of the algorithm. This study was supported by the National Institutes of Health Grant HL45623 and grants from the Natural Sciences and Engineering Research Council of Canada, the Danish Heart Association, the Danish Medical Research Council (12-3164-1), and the Novo-Nordisk Foundation     

Identification of nonlinear biological systems using laguerre expansions of kernels

Identification of nonlinear dynamic systems using the Volterra-Wiener approach requires the estimation of system kernels from input-output data. A kernel estimation technique, originally proposed by Wiener (1958) and recently studied by Ogura (1986), employs Laguerre expansions of the kernels and estimates the unknown expansion coefficients via time-averaging of covariance samples. This paper presents another implementation of the technique which utilizes least-squares fitting instead of covariance time-averaging and provides for the proper selection of the intrinsic Laguerre parameter “α”. Results from simulation examples demonstrate that this implementation can yield accurate kernel estimates up to 3rd-order from short input-output data records. Furthermore, it is shown that this implementation remains effective in the presence of noise and when the spectral characteristics of the input signal deviate somewhat from the theoretical requirements of whiteness. the computational requirements and the overall performance of this technique compare favorably to existing methods, especially in cases where the system kernels can be represented with a relatively small number of Laguerre basis functions.     

Nonlinearity identified by neural network models inP CO 2 control system in humans

The nonlinearity included in the P_{CO 2} control system in humans is evaluated using the degree of nonlinearity based on a difference of residuals. An autoregressive moving average (ARMA) model and neural networks (linear and nonlinear) are employed to model the system, and three types of network (Jordan, Elman and fully interconnected) are compared. As the Jordan-type linear network cannot approximate respiratory data accurately, the other two types and the ARMA model are used for the evaluation of the nonlinearity. The results of the evaluation indicate that the linear assumption for the P_{CO 2} control system is invalid for three subjects out of seven. In particular, strong nonlinearity was observed for two subjects.     

Classification of normal and abnormal electrogastrograms using multilayer feedforward neural networks

A neural network approach is proposed for the automated classification of the normal and abnormal EGG. Two learning algorithms, the quasi-Newton and the scaled conjugate gradient method for the multilayer feedforward neural networks (MFNN), are introduced and compared with the error backpropagation algorithm. The configurations of the MFNN are determined by experiment. The raw EGG data, its power spectral data, and its autoregressive moving average (ARMA) modelling parameters are used as the input to the MFNN and compared with each other. Three indexes (the percent correct, sumsquared error and complexity per iteration) are used to evaluate the performance of each learning algorithm. The results show that the scaled conjugate gradient algorithm performs best, in that it is robust and provides a super-linear convergence rate. The power spectral representation and the ARMA modelling parameters of the EGG are found to be better types of the input to the network for this specific application, both yielding a percent correctness of 95% on the test set. Although the results are focused on the classification of the EGG, this paper should provide useful information for the classification of other biomedical signals.     

Dual-component model of respiratory motion based on the periodic autoregressive moving average (periodic ARMA) method

A new approach to the problem of modelling and predicting respiration motion has been implemented. This is a dual-component model, which describes the respiration motion as a non-periodic time series superimposed onto a periodic waveform. A periodic autoregressive moving average algorithm has been used to define a mathematical model of the periodic and non-periodic components of the respiration motion. The periodic components of the motion were found by projecting multiple inhale-exhale cycles onto a common subspace. The component of the respiration signal that is left after removing this periodicity is a partially autocorrelated time series and was modelled as an autoregressive moving average (ARMA) process. The accuracy of the periodic ARMA model with respect to fluctuation in amplitude and variation in length of cycles has been assessed. A respiration phantom was developed to simulate the inter-cycle variations seen in free-breathing and coached respiration patterns. At +/-14% variability in cycle length and maximum amplitude of motion, the prediction errors were 4.8% of the total motion extent for a 0.5 s ahead prediction, and 9.4% at 1.0 s lag. The prediction errors increased to 11.6% at 0.5 s and 21.6% at 1.0 s when the respiration pattern had +/-34% variations in both these parameters. Our results have shown that the accuracy of the periodic ARMA model is more strongly dependent on the variations in cycle length than the amplitude of the respiration cycles.     

Separable least squares identification of nonlinear Hammerstein models: application to stretch reflex dynamics

The Hammerstein cascade, consisting of a zero-memory nonlinearity followed by a linear filter, is often used to model nonlinear biological systems. This structure can represent some high-order nonlinear systems accurately with relatively few parameters. However, it is not possible, in general, to estimate the parameters of a Hammerstein cascade in closed form. The most effective method available to date uses an iterative approach, which alternates between estimating the linear element from a crosscorrelation, and then fitting a polynomial to the nonlinearity via linear regression. This paper proposes the use of separable least squares optimization methods to estimate the linear and nonlinear elements simultaneously in a least squares framework. A separable least squares algorithm for the identification of Hammerstein cascades is developed and used to analyze stretch reflex electromyogram data from two experimental subjects. The results show that in each case the proposed algorithm produced a better model, in that it predicted the systems response to novel inputs more accurately, than did models estimated using the traditional iterative algorithm. Monte-Carlo simulations demonstrated that when the input is a non-Gaussian, nonwhite signal, as is often the case experimentally, the traditional iterative identification approach produces biased models, whereas the separable least squares approach proposed in this paper does not. © 2001 Biomedical Engineering Society. PAC01: 8719Ff, 8719Nn, 8710+e     

A neuro-fuzzy inference system for modeling and prediction of heart rate variability in the neuro-intensive care unit

In the neurological intensive care unit (NICU), prediction of impending changes in patient condition would be highly beneficial. In this paper, we employ a neuro-fuzzy inference system (NFIS) for short-term prediction of heart rate variability in the NICU. An NFIS was selected because it allows for a “gray-box” approach through which a system identification procedure is used in conjunction with fuzzy modeling. The NFIS is described in detail and is compared to an auto-regressive moving average (ARMA) model for its ability to model both simulated and measured data from NICU patients. We found that the NFIS is capable of predicting changes in heart rate to a reasonable extent, and that the NFIS has both advantages and limitations over the ARMA model. The NFIS may therefore be a reasonable technique to consider for more extensive prediction purposes in ICU settings.     

A Robust Time-Varying Identification Algorithm Using Basis Functions

We extend a recently developed time invariant (TIV) model order search criterion named the optimal parameter search algorithm (OPS) for identification of time varying (TV) autoregressive (AR) and autoregressive moving average (ARMA) models. Using the TV algorithm is facilitated by the fact that expanding each TV coefficient onto a finite set of basis sequences permits TV parameters to become TIV. Taking advantage of this TIV feature of expansion parameters exploits the features of the OPS, which has been shown to provide accurate model order selection as well as extraction of only the significant model terms. Another advantage of the new algorithm is its ability to discriminate insignificant basis sequences thereby reducing the number of expansion parameters to be estimated. Due to these features, the resulting algorithm can accurately estimate TV AR or ARMA models and determine their orders. Indeed, comparison via computer simulations of AR models between the proposed method and one of the well-known iterative methods, recursive least squares, shows the greater capability of the new method to track TV parameters. Furthermore, application of the new method to experimentally obtained renal blood flow signals shows that the new method provides higher-resolution time-varying spectral capability than does the short-time Fourier transform (STFT), concomitant with fewer spurious frequency peaks than obtained with the STFT spectrogram. © 2003 Biomedical Engineering Society. PAC2003: 8710+e, 8719Uv, 8780Tq     

The Topography of Non-Linear Cortical Dynamics at Rest, in Mental Calculation and Moving Shape Perception

Differential cortical activation by cognitive processing was studied using dimensional complexity, a measure derived from nonlinear dynamics that indicates the degrees of freedom (complexity) of a dynamic system. We examined the EEG of 32 healthy subjects at rest, during a visually presented calculation task, and during a moving shape perception task. As a nonlinear measure of connectivity, the mutual dimension of selected electrode pairs was used. The first Lyapunov coefficient was also calculated. Data were tested for non-linearity using a surrogate data method and compared to spectral EEG measures (power, coherence). Surrogate data testing confirmed the presence of nonlinear structure in the data. Cognitive activation led to a highly significant rise in dimensional complexity. While both tasks activated central, parietal and temporal areas, mental arithmetic showed frontal activation and an activity maximum at T3, while the moving shape task led to occipital activation and a right parietal activity maximum. Analysis of mutual dimension showed activation of a bilateral temporal-right frontal network in calculation. The Lyapunov coefficent showed clear topographic variation, but was not significantly changed by mental tasks (p<.09). While dimensional complexity was almost unrelated to power values, nonlinear (mutual dimension) and linear (coherence) measures of connectivity shared up to 37% of variance. Data are interpreted in terms of increased cortical complexity as a result of recruitment of asynchronously active, distributed neuronal assemblies in cognition. The topography of nonlinear dynamics are related to neuropsychological and neuroimaging findings on mental calculation and moving shape perception.     

Nonstationarity of dynamic cerebral autoregulation

Dynamic cerebral autoregulation (dCA), the transient response of cerebral blood flow (CBF) to rapid changes in arterial blood pressure (BP), is usually quantified by parameters extracted from time- or frequency-domain analysis. Reproducibility studies of dCA parameters and consideration of the physiological determinants of the dynamic BP-CBF relationship provide strong indications that dCA is a nonstationary process. As a consequence, new analytical approaches are needed to estimate dCA parameters with greater temporal resolution thus allowing its longitudinal patterns of variability to be assessed in health and disease states. Techniques proposed for this task include ARMA models with moving windows, recursive least-squares, Laguerre–Volterra networks, wavelet phase synchronisation, and multimodal pressure-flow analysis. Initial results with these techniques have revealed the influence of some key determinants of dCA nonstationarity, such as PaCO₂, as well as their ability to reflect dCA impairment in different clinical conditions. One key priority for future work is the development and validation of multivariate time-varying techniques to minimise the influence to the many co-variates which contribute to dCA nonstationarity.     

A nonlinear trimmed moving averaging-based system with its application to real-time QRS beat classification

In this paper, a real-time QRS beat classification system based on a nonlinear trimmed moving average filter is presented. This nonlinear system aims to identify abnormal beats of ventricular origin. The proposed beat classifier is designed to work in parallel with a real-time QRS detector, allowing the task of beat diagnosis to be performed immediately after a QRS complex is detected. Algorithm performance was evaluated against the ECG recordings drawn from the MIT-BIH arrhythmia database. Numerical results demonstrated that a beat classification rate of over 99.5% can be achieved by the algorithm.     

A nonlinear trimmed moving averaging-based system with its application to real-time QRS beat classification

In this paper, a real-time QRS beat classification system based on a nonlinear trimmed moving average filter is presented. This nonlinear system aims to identify abnormal beats of ventricular origin. The proposed beat classifier is designed to work in parallel with a real-time QRS detector, allowing the task of beat diagnosis to be performed immediately after a QRS complex is detected. Algorithm performance was evaluated against the ECG recordings drawn from the MIT-BIH arrhythmia database. Numerical results demonstrated that a beat classification rate of over 99.5% can be achieved by the algorithm.     

Nonlinear analysis of renal autoregulation under broadband forcing conditions

Linear analysis of renal blood flow fluctuations, induced experimentally in rats by broad-band (pseudorandom) arterial blood pressure forcing at various power levels, has been unable to explain fully the dynamics of renal autoregulation at low frequencies (1). This observation has suggested the possibility of nonlinear mechanisms subserving renal autoregulation at frequencies below 0.2 Hz. This paper presents results of 3rd-order Volterra-Wiener analysis that appear to explain adequately the nonlinearities in the pressure-flow relation below 0.2 Hz in rats. The contribution of the 3rd-order kernel in describing the dynamic pressure-flow relation is found to be important. Furthermore, the dependence of 1st-order kernel waveforms on the power level of broadband pressure forcing indicates the presence of nonlinear feedback (of sigmoid type) based on previously reported analysis of a class of nonlinear feedback systems (11).     

Detection of the QRS complex by linear prediction

The electrocardiogram (ECG) represents the electrical activity of the heart. It is characterized by its recurrent or periodic behaviour with each beat. Each recurrence is composed of a wave sequence consisting of P, QRS and T-waves, where the most characteristic wave set is the QRS complex. In this paper, we have developed an algorithm for detection of the QRS complex. The algorithm consists of several steps: signal-to-noise enhancement, linear prediction for ECG signal analysis, nonlinear transform, moving window integrator, centre-clipping transformation and QRS detection. Linear prediction determines the coefficients of a forward linear predictor by minimizing the prediction error by a least-square approach. The residual error signal obtained after processing by the linear prediction algorithm has very significant properties which will be used to localize and detect QRS complexes. The detection algorithm is tested on ECG signals from the universal MIT-BIH arrhythmia database and compared with the Pan and Tompkins QRS detection method. The results we obtain show that our method performs better than this method. Our algorithm results in fewer false positives and fewer false negatives.     

Assessment of Granger Causality by Nonlinear Model Identification: Application to Short-term Cardiovascular Variability

A method for assessing Granger causal relationships in bivariate time series, based on nonlinear autoregressive (NAR) and nonlinear autoregressive exogenous (NARX) models is presented. The method evaluates bilateral interactions between two time series by quantifying the predictability improvement (PI) of the output time series when the dynamics associated with the input time series are included, i.e., moving from NAR to NARX prediction. The NARX model identification was performed by the optimal parameter search (OPS) algorithm, and its results were compared to the least-squares method to determine the most appropriate method to be used for experimental data. The statistical significance of the PI was assessed using a surrogate data technique. The proposed method was tested with simulation examples involving short realizations of linear stochastic processes and nonlinear deterministic signals in which either unidirectional or bidirectional coupling and varying strengths of interactions were imposed. It was found that the OPS-based NARX model was accurate and sensitive in detecting imposed Granger causality conditions. In addition, the OPS-based NARX model was more accurate than the least squares method. Application to the systolic blood pressure and heart rate variability signals demonstrated the feasibility of the method. In particular, we found a bilateral causal relationship between the two signals as evidenced by the significant reduction in the PI values with the NARX model prediction compared to the NAR model prediction, which was also confirmed by the surrogate data analysis. Furthermore, we found significant reduction in the complexity of the dynamics of the two causal pathways of the two signals as the body position was changed from the supine to upright. The proposed is a general method, thus, it can be applied to a wide variety of physiological signals to better understand causality and coupling that may be different between normal and diseased conditions.     

Sensitivity Analysis of Kernel Estimates: Implications in Nonlinear Physiological System Identification

Many techniques have been developed for the estimation of the Volterra/Wiener kernels of nonlinear systems, and have found extensive application in the study of various physiological systems. To date, however, we are not aware of methods for estimating the reliability of these kernels from single data records. In this study, we develop a formal analysis of variance for least-squares based nonlinear system identification algorithms. Expressions are developed for the variance of the estimated kernel coefficients and are used to place confidence bounds around both kernel estimates and output predictions. Specific bounds are developed for two such identification algorithms: Korenberg's fast orthogonal algorithm and the Laguerre expansion technique. Simulations, employing a model representative of the peripheral auditory system, are used to validate the theoretical derivations, and to explore their sensitivity to assumptions regarding the system and data. The simulations show excellent agreement between the variances of kernel coefficients and output predictions as estimated from the results of a single trial compared to the same quantities computed from an ensemble of 1000 Monte Carlo runs. These techniques were validated with white and nonwhite Gaussian inputs and with white Gaussian and nonwhite non-Gaussian measurement noise on the output, provided that the output noise source was independent of the test input.     

Automatic algorithm for filtering kinematic signals with impacts in the Wigner representation

An automatic filtering algorithm is proposed for the accurate estimation of the second derivatives of kinematic signals with impacts. The impacts considered here occur when a moving object hits a rigid surface. The algorithm performs time-frequency filtering in the Wigner representation, to deal efficiently with the non-stationarities caused by such impacts, and adjusts the parameters of its time-frequency filtering function so that the filtering process adapts to the individual characteristics of the signal in hand. Performance analysis and comparative evaluation with experimentally acquired kinematic impact signals demonstrated a higher accuracy, with performance advantages over two widely used conventional automatic methods: linear phase autoregressive model-based derivative assessment (LAMBDA) and generalised cross-validation using quintic splines (GCVQS). For high impacts, the average absolute relative error in estimating the peak acceleration was 5.7% with the proposed method, 17.2% with a Butterworth low-pass filter optimised to yield minimum overall acceleration RMS error (best-case result), 18.3% with the LAMBDA method, and 37.2% with the GCVQS method. For signals with low impacts, the average absolute relative error was 19.4%, 6.9%, 8.3% and 19.1%, respectively, in each case, which indicates that, for signals with a low-frequency content, there is no need for such time-frequency filtering.     

Detection of the QRS complex by linear prediction

The electrocardiogram (ECG) represents the electrical activity of the heart. It is characterized by its recurrent or periodic behaviour with each beat. Each recurrence is composed of a wave sequence consisting of P, QRS and T-waves, where the most characteristic wave set is the QRS complex. In this paper, we have developed an algorithm for detection of the QRS complex. The algorithm consists of several steps: signal-to-noise enhancement, linear prediction for ECG signal analysis, nonlinear transform, moving window integrator, centre-clipping transformation and QRS detection. Linear prediction determines the coefficients of a forward linear predictor by minimizing the prediction error by a least-square approach. The residual error signal obtained after processing by the linear prediction algorithm has very significant properties which will be used to localize and detect QRS complexes. The detection algorithm is tested on ECG signals from the universal MIT-BIH arrhythmia database and compared with the Pan and Tompkins QRS detection method. The results we obtain show that our method performs better than this method. Our algorithm results in fewer false positives and fewer false negatives.