Finite‐horizon near optimal adaptive control of uncertain linear discrete‐time systems

In this paper, the finite‐horizon near optimal adaptive regulation of linear discrete‐time systems with unknown system dynamics is presented in a forward‐in‐time manner by using adaptive dynamic programming and Q‐learning. An adaptive estimator (AE) is introduced to relax the requirement of system dynamics, and it is tuned by using Q‐learning. The time‐varying solution to the Bellman equation in adaptive dynamic programming is handled by utilizing a time‐dependent basis function, while the terminal constraint is incorporated as part of the update law of the AE. The Kalman gain is obtained by using the AE parameters, while the control input is calculated by using AE and the system state vector. Next, to relax the need for state availability, an adaptive observer is proposed so that the linear quadratic regulator design uses the reconstructed states and outputs. For the time‐invariant linear discrete‐time systems, the closed‐loop dynamics becomes non‐autonomous and involved but verified by using standard Lyapunov and geometric sequence theory. Effectiveness of the proposed approach is verified by using simulation results. The proposed linear quadratic regulator design for the uncertain linear system requires an initial admissible control input and yields a forward‐in‐time and online solution without needing value and/or policy iterations. Copyright © 2014 John Wiley & Sons, Ltd.     

The observer‐based linear quadratic sub‐optimal digital tracker for analog systems with input and state delays

This paper presents a new observer‐based sub‐optimal digital tracker for the continuous‐time system with input and state time delays. We directly convert the continuous‐time input‐state delayed system into an equivalent discrete‐time input‐state delayed model and its extended discrete‐time delay‐free model. In addition, we directly discretize the linear quadratic performance index specified in the continuous‐time domain into an equivalent decoupled discrete‐time performance index using the newly developed extended delay‐free model. As a consequence, the well‐developed discrete‐time optimal control theory for discrete‐time delay‐free system can be applied to determine the optimal digital tracker for the continuous‐time input‐state delayed system. When the states of the continuous‐time input‐state delayed system are not available for measurements, we develop a sub‐optimal digital observer for the original continuous‐time input‐state delayed system using the state‐matching digital redesign technique and the digital‐to‐analog model conversion technique. As a result, the proposed observer‐based linear quadratic digital tracker is able to make the output of the digitally controlled continuous‐time input‐state delayed system sub‐optimally track the desired reference signals. An illustrative example is presented to demonstrate the effectiveness of the proposed design methodology. Copyright © 2003 John Wiley & Sons, Ltd.     

LQ‐optimal control of positive linear systems

The LQ₊ problem, i.e. the finite‐horizon linear quadratic optimal control problem with nonnegative state constraints, is studied for positive linear systems in continuous time and in discrete time. Necessary and sufficient optimality conditions are obtained by using the maximum principle. These conditions lead to a computational method for the solution of the LQ₊ problem by means of a corresponding Hamiltonian system. In addition, the necessary and sufficient conditions are proved for the LQ₊‐optimal control to be given by the standard LQ‐optimal state feedback law. Then sufficient conditions are established for the positivity of the LQ‐optimal closed‐loop system. In particular, such conditions are obtained for the problem of minimal energy control with penalization of the final state. Moreover, a positivity criterion for the LQ‐optimal closed‐loop system is derived for positive discrete‐time systems with a positively invertible (dynamics) generator. The main results are illustrated by numerical examples. Copyright © 2010 John Wiley & Sons, Ltd.     

A new optimal linear quadratic observer‐based tracker under input constraint for the unknown system with a direct feed‐through term

This paper develops a new optimal linear quadratic observer‐based tracker with input constraint for the linear unknown system with a direct transmission term from input to measured output. The off‐line observer/Kalman filter identification method is used to determine the linear sampled‐data model with a direct feed‐through term. On the basis of this model, a high‐gain optimal linear quadratic analog observer‐based tracker is proposed, so that it can effectively induce a high quality performance on the state estimation and the trajectory tracking design. Besides, the prediction‐based digital redesign method is utilized to obtain a relatively low‐gain and implementable observer and digital tracker from the theoretically well‐designed high‐gain analog observer and tracker for the linear system with a direct transmission term from input to output. To reduce the magnitude of control input, which is caused by the high‐gain property to fit the requirement of the input constraint, the modified linear quadratic analog tracker is proposed. Thus, the control input can be compressed effectively without losing the original high performance of tracking much. Copyright © 2014 John Wiley & Sons, Ltd.     

Jump linear quadratic Gaussian problem for a class of nonhomogeneous Markov jump linear systems

This paper concerns with the jump linear quadratic Gaussian problem for a class of nonhomogeneous Markov jump linear systems (MJLSs) in the presence of process and observation noise. By assuming that mode transition rate matrices (MTRMs) are piecewise homogeneous whose variation is subjected to a high‐level Markov process, two Markov processes are proposed to model the characteristics of nonhomogeneous MJLSs: the variation of system mode is governed by a low‐level Markov process, while the variation of MTRM is governed by a high‐level one. Based on this model, a mode‐MTRM‐based optimal filter is firstly given where filter gain can be obtained via coupled Riccati equations. Secondly, we extend the separation principle of the linear quadratic problem to the nonhomogeneous MJLSs case. An optimal controller is then designed to minimize the quadratic system cost. Finally, a potential application in solar boiler system is given to illustrate the developed theoretical methods. Copyright © 2016 John Wiley & Sons, Ltd.     

H∞ output tracking control over networked control systems with Markovian jumping parameters

The problem of H_∞ output tracking control over networked control systems (NCSs) with communication limits and environmental disturbances is studied in this paper. A wide range of time‐varying stochastic problem arising in networked tracking control system is reduced to a standard convex optimization problem involving linear matrix inequalities (LMIs). The closed‐loop hybrid NCS is modeled as a Markov jump linear system in which random time delays and packet dropouts are described as two stochastic Markov chains. Gridding approach is introduced to guarantee the finite value of the sequences of transmission delays from sensor to actuator. Sufficient conditions for the stochastic stabilization of the hybrid NCS tracking system are derived by the LMI‐based approach through the computation of the optimal H_∞ performance. The mode‐dependent robust H_∞ output tracking controller is obtained by the optimal iteration method. Numerical examples are given to demonstrate the effectiveness of the proposed robust output tracking controller for NCS. Copyright © 2016 John Wiley & Sons, Ltd.     

Linear quadratic regulation for discrete‐time systems with multiplicative noise and multiple input delays

This paper is concerned with the optimal linear quadratic regulation problem for discrete‐time systems with state and control dependent noises and multiple delays in the input. We show that the problem admits a unique solution if and only if a sequence of matrices, which are determined by coupled difference equations developed in this paper, are positive definite. Under this condition, the optimal feedback controller and the optimal cost are presented via some coupled difference equations. Our approach is based on the stochastic maximum principle. The key technique is to establish relations between the costate and the state. Copyright © 2016 John Wiley & Sons, Ltd.     

Optimal control of nonlinear aeroelastic system with non-semi-simple eigenvalues at Hopf bifurcation points

This study discusses an efficient method of the Hopf bifurcation control for nonlinear aeroelastic system. The nonlinear aeroelastic system whose linear part has multiple non-semi-simple eigenvalues at critical point gives rise to Hopf bifurcations. The method of the multiple scales and the well-known linear quadratic regulator method are used to deal with the optimal control of the nonlinear system at Hopf bifurcation points. The modal optimal control equation and modal Riccati equation of the nonlinear system are developed to simplify the computations. The conventional Potter's algorithm is extended to solve modal Riccati equation for the modal Riccati matrix of the Hopf bifurcation control. The first-order approximation solutions are developed, which include the gain vectors and inputs. By the way of optimal control, the admissible control input and trajectory of the linear part of the nonlinear aeroelastic system are obtained to minimize the performance measure. Then, we set the appropriate first-order gain vector to adjust the convergence speed of this nonlinear system.     

Near‐optimal control for multiparameter singularly perturbed stochastic systems

In this paper, the linear quadratic optimal stochastic control problem is investigated for multiparameter singularly perturbed stochastic systems in which N lower‐level fast subsystems are interconnected by a higher‐level slow subsystem. After establishing the asymptotic structure of the solution for the multiparameter stochastic algebraic Riccati equation (MSARE), a near‐optimal controller that is independent of small unknown parameters is obtained by neglecting these parameters. The stability of a closed‐loop stochastic system is investigated. Furthermore, it is shown that the resulting controller achieves an O(∥ν∥²) approximation to the optimal cost of the original optimal control problem. Finally, in order to demonstrate the efficiency of the proposed algorithm, a numerical example—a practical multi‐area power system—is solved. Copyright © 2010 John Wiley & Sons, Ltd.     

Linear quadratic optimal sampled data control of linear systems with unknown switched modes and stochastic disturbances

It is nontrivial to control a dynamic system that is switched consistently with a completely unknown switched modes. This problem is further complicated if the system is subject to stochastic disturbance. This paper studies the linear quadratic optimal control problem of linear continuous systems with stochastic disturbance and unknown switched process. By integrating one‐step adaptive estimator with optimal control theory, a linear quadratic optimal stabilization controller based on sampled feedback is developed for systems that are continuous in nature yet switched consistently with unknown modes. It is shown that with the proposed control scheme, both parameter estimate error and system stabilization error are ensured to be bounded, and the existence of the upper bound is explicitly confirmed. The results compliment and extend the existing works on digital feedback control of switched linear continuous systems with unknown switched processes and stochastic disturbances. Copyright © 2015 John Wiley & Sons, Ltd.     

An optimal control approach to spacecraft rendezvous on elliptical orbit

In this paper, we consider a linear quadratic regulator control problem for spacecraft rendezvous in an elliptical orbit. A new spacecraft rendezvous model is established. On the basis of this model, a linear quadratic regulator control problem is formulated. A parametric Lyapunov differential equation approach is used to design a state feedback controller such that the resulting closed‐loop system is asymptotically stable, and the performance index is minimized. By an appropriate choice of the value of a parameter, an approximate state feedback controller is obtained from a solution to the periodic Lyapunov differential equation, where the periodic Lyapunov differential equation is solved on the basis of a new numerical algorithm. The spacecraft rendezvous mission under the controller obtained will be accomplished successfully. Several illustrative examples are provided to show the effectiveness of the proposed control design method. Copyright © 2014 John Wiley & Sons, Ltd.     

Optimal control of non‐linear chemical reactors via an initial‐value Hamiltonian problem

The problem of designing strategies for optimal feedback control of non‐linear processes, specially for regulation and set‐point changing, is attacked in this paper. A novel procedure based on the Hamiltonian equations associated to a bilinear approximation of the dynamics and a quadratic cost is presented. The usual boundary‐value situation for the coupled state–costate system is transformed into an initial‐value problem through the solution of a generalized algebraic Riccati equation. This allows to integrate the Hamiltonian equations on‐line, and to construct the feedback law by using the costate solution trajectory. Results are shown applied to a classical non‐linear chemical reactor model, and compared against suboptimal bilinear‐quadratic strategies based on power series expansions. Since state variables calculated from Hamiltonian equations may differ from the values of physical states, the proposed control strategy is suboptimal with respect to the original plant. Copyright © 2005 John Wiley & Sons, Ltd.     

Optimal control for discrete‐time singular stochastic systems with input delay

The finite‐horizon linear quadratic regulation problem is considered in this paper for the discrete‐time singular systems with multiplicative noises and time delay in the input. Firstly, the extremum principle is discussed, and a stationary condition is derived for the singular stochastic system. Then, based on the relationships established between the state and the costate variables, the stationary condition is also shown to be a sufficient criterion assuring the existence of the solution for the stochastic control problem. The optimal controller is designed as a linear function of the current state and the past inputs information, which can be recursively calculated by effective algorithms. With the designed optimal controllers, the explicit expression is also derived for the minimal value of the performance index. One numerical example is provided in the end of the paper to illustrate the effectiveness of the obtained results. Copyright © 2016 John Wiley & Sons, Ltd.     

Optimal portfolio of continuous‐time mean‐variance model with futures and options

This paper is concerned with the portfolio selection of options that have different sale and purchase prices based on the continuous‐time mean‐variance criterion in financial markets. The optimal investment problem is formulated as a continuous‐time mathematical model. These price processes follow jump‐diffusion processes (the Weiner process and the Poisson process). With the theory of stochastic linear‐quadratic control and viscosity solutions, the corresponding Hamilton‐Jacobi‐Bellman equation of the problem is represented and its solutions are obtained in different conditions. The optimal investment strategies are presented. In addition, the efficient frontier is also illustrated. Finally, an example and some discussions illustrating these results are also presented.     

Delay‐dependent feedforward control of time‐delay systems with parametric uncertainties via dynamic IQCs

This paper studies the design problem of a robust delay‐dependent H _∞ feedforward controller design for a class of linear uncertain time‐delay system having state and control delays when the system is subject to ‐type disturbances. The proposed controller scheme involves two main controllers, which are static state‐feedback and dynamic feedforward controllers. The state‐feedback controller is used for stabilizing the delay and uncertainty‐free system, whereas the feedforward controller performs disturbance attenuation. Dynamic type integral quadratic constraints (IQCs), which consist of frequency‐dependent multipliers, have been introduced to represent the delays and parametric uncertainties in the system where the degree of the multiplier used in IQC representation is in an adjustable nature. This scheme allows the designer to obtain less conservative controllers with increasing precision. Sufficient delay‐dependent criteria in terms of linear matrix inequalities are obtained such that the uncertain linear time‐delay system is guaranteed to be globally, uniformly, asymptotically stable with a minimum disturbance attenuation level. Several numerical examples together with the simulation studies provided at the end illustrate the usefulness of the proposed design. Copyright © 2012 John Wiley & Sons, Ltd.     

Optimal control for linear system using genetic programming

In this paper, optimal control for a linear system with quadratic performance is obtained using genetic programming (GP). The goal is to find the optimal control with reduced calculus effort using non‐traditional methods. The obtained GP solution is compared with the traditional Runge–Kutta method. To obtain optimal control, the solution of matrix Riccati differential equation is computed based on grammatical evolution. The accuracy of the solution of the GP approach to the problem is qualitatively better than traditional methods. An illustrative numerical example is presented for the proposed method. Copyright © 2008 John Wiley & Sons, Ltd.     

Stabilization of positive coupled differential‐difference equations with unbounded time‐varying delays

This paper researches the static output‐feedback stabilization of single‐input single‐output (SISO) positive coupled differential‐difference equations (CDDEs) with unbounded time‐varying delays. First, a necessary and sufficient condition is provided for the positivity and asymptotical stability of CDDEs with unbounded time‐varying delays. For this type of system, based on the constructed estimates of its solution, a necessary and sufficient condition on asymptotical stability is provided. Then, based on this criterion, for CDDEs with unbounded time‐varying delays, a kind of static output‐feedback controller is designed to ensure the positivity and asymptotical stability of the corresponding closed‐loop systems. It is also worth pointing out that the controller is designed by the linear programming method without parameterization technique. This design approach can also be applied to the static state feedback stabilization problem of CDDEs with unbounded time‐varying delays. Finally, two illustrative examples are given to show the effectiveness of our results.