Optimal control of discrete-time nonlinear heterogeneous multi-agent systems via a distributed DISOPE algorithm

A distributed offline DISOPE algorithm for optimal state synchronization of leader-follower systems with nonlinear discrete-time dynamics is considered, which integrates the model optimization idea and parameter estimation technique together. It can be seen that the convergent solutions of modified linear optimal control problems satisfy the optimality conditions of the original nonlinear optimization problem with non-LQ performance indices. The heterogeneous agents can cooperate and exchange information via network communication. Based on DISOPE algorithm, a distributed optimal control policy is obtained to assure state synchronization and minimize performance indices in finite time horizon. Finally, a simulation example is provided to illustrate the effectiveness of the distributed DISOPE algorithm.     

Stochastic model predictive control for tracking linear systems

This note presents a stochastic formulation of the model predictive control for tracking (MPCT), based on the results of the work of Lorenzen et al. The proposed controller ensures constraints satisfaction in probability, and maintains the main features of the MPCT, that are feasibility for any changing setpoints and enlarged domain of attraction, even larger than the one delivered by Lorenzen et al, thanks to the use of artificial references and relaxed terminal constraints. The asymptotic stability (in probability) of the minimal robust positively invariant set centered on the desired setpoint is guaranteed. Simulations on a DC-DC converter show the benefits and the properties of the proposal.     

On linear quadratic optimal control of discrete-time complex-valued linear systems

We study in this paper the linear quadratic optimal control (linear quadratic regulation, LQR for short) for discrete-time complex-valued linear systems, which have several potential applications in control theory. Firstly, an iterative algorithm was proposed to solve the discrete-time bimatrix Riccati equation associated with the LQR problem. It is shown that the proposed algorithm converges to the unique positive definite solution (bimatrix) to the bimatrix Riccati equation with appropriate initial conditions. With the help of this iterative algorithm, the LQR problem for the antilinear system, which is a special case of complex-valued linear system, was carefully examined and three different Riccati equations–based approaches were provided, namely, bimatrix Riccati equation, anti-Riccati equation, and normal Riccati equation. The established approach is then used to solve the LQR problem for a discrete-time time-delay system with one-step state delay, and a numerical example was used to illustrate the effectiveness of the proposed methods.     

Switching optimal adaptive trajectory tracking control of quantum systems

In this paper, a switching optimal adaptive controller for tracking a time‐dependent trajectory in finite‐dimensional closed quantum systems is proposed. The issue of intrinsic singularities in trajectory tracking control of quantum systems leads to a sharp rise in the control amplitude. To overcome this drawback, a switching optimal adaptive quantum controller is designed based on Lyapunov stability theory and optimal quantum control approach. A state‐dependent strategy is considered to select the switching signal. The new switching controller adjusts the quantum state so that its population traces the desired dynamic trajectory and simultaneously eliminates the effects of singularities and reduces the control amplitude. The proposed controller is tested successfully for population transfer in a 4‐level closed quantum system in a simulation experiment. Both issues of reduction of the tracking error and control intensity along with a significant decrease in the number of singular points are well illustrated by simulation experiment as the advantages of the proposed method.     

Finite-time disturbance attenuation control problem for singularly perturbed discrete-time systems

In this paper, we consider the problem of finite-time H_∞-optimal control of linear, singularly perturbed, discrete-time systems. The problem is addressed from the game theoretic approach. This leads to a singularly perturbed, matrix Riccati difference equation, the solution of which is given in terms of an outer series solution, and a boundary-layer correction series solution. We show that the disturbance attenuation level achieved by the singular perturbation method, compared to the full-order solution, depends on the order of approximation. The theory is illustrated by considering two examples. © 1998 John Wiley & Sons, Ltd.     

A novel neural network discrete-time optimal control design for nonlinear time-delay systems using adaptive critic designs

In this article, a novel neural network (NN) optimal control approach using adaptive critic designs is developed for nonlinear discrete-time (DT) systems with time delays. First, to eliminate the delay term of control input, a time-delay matrix function is developed by designing a M network. Furthermore, the cost function is approximated by the critic NN, and the control signal can be obtained directly by using the information of critic NN according to the equilibrium condition. In addition, to shorten the learning time and reduce the computational burden in the control process, a novel control strategy with less adjustable parameters for the time-delay DT nonlinear systems is proposed in this article, in which the norm of the weight estimations of critic NN is updated to generate a novel long-term performance function. The proposed control algorithm using adaptive critic designs has the advantage of reducing adaptive learning parameters and lessening calculative burden. The Lyapunov stability analysis shows that the time-delay DT controlled systems can be uniformly ultimately bounded stable. Finally, three simulations are presented to demonstrate the control performance of the developed method.     

Optimal control of linear discrete systems via hahn polynomials

This paper presents a method of finite series expansion using Hahn polynomials for the finite time optimal control of linear discrete time systems with a quadratic performance index. The method converts the discrete two-point boundary value Euler—Lagrange difference equations into a set of linear algebraic equations. As a result, the computation is simple and straightforward.     

Optimal control of linearly constrained linear systems via state parametrization

This paper presents a general computational tool for determining the near-optimal trajectories of linear, lumped parameter, dynamic systems subjected to linear constraints. In the proposed approach each state variable is approximated by the sum of a third-order polynomial and a finite term Fourier-type series. This enables a linearly constrained optimal control problem to be converted into a linearly constrained mathematical programming problem. Simulation results demonstrate that the approach accurately predicts the optimal performance index and the optimal state and control trajectories.     

A reinforcement learning‐based scheme for direct adaptive optimal control of linear stochastic systems

Reinforcement learning where decision‐making agents learn optimal policies through environmental interactions is an attractive paradigm for model‐free, adaptive controller design. However, results for systems with continuous state and action variables are rare. In this paper, we present convergence results for optimal linear quadratic control of discrete‐time linear stochastic systems. This work can be viewed as a generalization of a previous work on deterministic linear systems. Key differences between the algorithms for deterministic and stochastic systems are highlighted through examples. The usefulness of the algorithm is demonstrated through a nonlinear chemostat bioreactor case study. Copyright © 2009 John Wiley & Sons, Ltd.     

Inverse optimal control for asymptotic trajectory tracking of discrete‐time stochastic nonlinear systems in block controllable form

This paper concerns an inverse optimal control–based trajectory tracking of discrete‐time stochastic nonlinear systems. It is assumed that the nonlinear system can be transformed to the so called nonlinear block controllable form. Additionally, the synthesized control law minimizes a cost functional, which is posteriori determined. In contrast to the optimal control technique, this scheme avoids to solve the stochastic Hamilton‐Jacobi‐Bellman equation, which is not an easy task. Based on a discrete‐time stochastic control Lyapunov function, the proposed optimal controller is analyzed. The proposed approach is applied successfully to the two degrees‐of‐freedom helicopter with uncertainties in real time.     

Controller design for optimal tracking response in discrete‐time systems

The problem of designing a controller, which results in a closed‐loop system response with optimal time‐domain characteristics, is considered. In the approach presented in this paper, the controller order is fixed (higher than pole‐placement order) and we seek a controller that results in closed‐loop poles at certain desired and pre‐specified locations; while at the same time the output tracks the reference input in an optimal way. The optimality is measured by requiring certain norms on the error sequence—between the reference and output signals—to be minimum. Several norms are used. First, l₂‐norm is used and the optimal solution is computed in one step of calculations. Second, l_∞‐norm (i.e. minimal overshot) is considered and the solution is obtained by solving a constrained affine minimax optimization problem. Third, the l₁‐norm (which corresponds to the integral absolute error‐(IAE)‐criterion) is used and linear programming techniques are utilized to solve the problem. The important case of finite settling time (i.e. deadbeat response) is studied as a special case. Examples that illustrate the different design algorithms and demonstrate their feasibility are presented. Copyright © 2007 John Wiley & Sons, Ltd.     

Optimality principles and decomposition of tracking controllers for weakly dual redundant systems

In this paper, the problem of optimizing the output regulation of a weakly dual redundant plant is addressed. When the system is underactuated, only a subset of the outputs can be arbitrarily controlled and the remaining ones are constrained. With a specific focus on the asymptotic output tracking problem for single‐input systems, we investigate the connection between the overall optimal input and the individually optimal controllers, which leads to the perfect tracking of each output reference. Such relationship is analyzed both for open‐loop and closed‐loop controllers for different cost functions and types of references, and the optimality principles are established, provided that suitable structural conditions are met by the plant matrices.     

Inverse optimal control for discrete‐time nonlinear systems via passivation

This paper presents an inverse optimal control approach for stabilization and trajectory tracking of discrete‐time nonlinear systems, avoiding to solve the associated Hamilton–Jacobi–Bellman equation, and minimizing a meaningful cost functional. The proposed controller is based on a discrete‐time control Lyapunov function and passivity theory; its applicability is illustrated via simulations for an unstable nonlinear system and a planar robot. Copyright © 2013 John Wiley & Sons, Ltd.     

Optimal control approach for discontinuous dynamical systems

In this work, we consider a wide class of discontinuous dynamical systems, discontinuity of which is based on the sign (for short sgn) function. We propose a smooth optimal control problem to solve the main discontinuous system. By solving some numerical examples in mechanical engineering, we show the efficiency of our approach with respect to 2 smoothing methods for discontinuous systems.     

Investment portfolio tracking using model predictive control

The composition of an investment portfolio aiming to increase the financial returns while reducing the exposure to risk is a topic of growing interest in the world. In this direction, we propose a model predictive control (MPC) strategy in order to optimize the investment portfolio selection taking into account the tracking of a reference portfolio with desired return. In addition, an analysis comparing different sizes of the prediction horizon according to VPH-MPC (Varying Predictive Horizon-MPC) and FPH-MPC (Fixed Predictive Horizon-MPC) algorithms is conducted. Finally, we propose an optimal control problem using the tracking error as a function of loss of CVaR (Conditional Value at Risk) measurement. Numerical experiments are run based on Brazilian Stock Exchange data. The experimental results are compared with the Markowitz portfolio optimization model, a conventional tracking strategy, and benchmarks from the Brazilian financial market. This comparison indicates a good tracking performance obtained by the proposed MPC in the two versions while satisfying the constraints.     

Calculation of discrete-time process noise statistics for hybrid continuous/discrete-time applications

Discrete-time models of continuous-time plants are commonly required owing to the popular use of computers to implement control and estimation algorithms. When stochastic design techniques such as the discrete-time Kalman filter are utilized, it is necessary to determine the equivalent discrete-time process noise statistics from the continuous-time process noise statistics. Herein we present a new solution for the required transformation.     

Optimal control of Markovian queueing systems

Dynamic optimization of queueing systems is treated by optimal control theory. This work is based on modelling the queueing problem as a time-varying continuous Markov chain. Necessary and sufficient conditions are given for a broad class of problems which include both scalar and Markovian dynamic programming control structures. Continuity of the switching function is used to characterize optimality near the end points of the horizon. Special properties of the model are exploited to ensure the absence of singular subarcs. Numerical results based on the use of a gradient algorithm report the effect of increasing the system capacity, a comparison of scalar versus Markovian dynamic programming controls, and an application to a multiprogrammed computer system.     

Optimal control of continuous systems with impulse controls

Impulse control problems, in which a continuously evolving state is modified by discrete control actions, have applications in epidemiology, medicine, and ecology. In this paper, we present a simple method for solving impulse control problems for systems of differential equations. In particular, we show how impulse control problems can be reformulated and solved as discrete optimal control problems. The method is illustrated with two examples. Published 2014. This article has been contributed to by US Government employees and their work is in the public domain in the USA.     

Inverse optimal neural control of a class of nonlinear systems with constrained inputs for trajectory tracking

This paper focusses on recurrent adaptive neural control applied to unknown nonlinear systems with input constraints. A recurrent high‐order neural network is used in order to identify the unknown system and a learning law is obtained using the Lyapunov methodology. Then a control law, which stabilizes the reference tracking error dynamics, is developed using the inverse optimal control approach. Tracking error stability is established as a function of design parameters. Chaotic systems are considered to demonstrate the applicability of the approach, via simulations, as for their stabilization and synchronization. The obtained simulation results illustrates the proposed approach capabilities for trajectory tracking of unknown nonlinear systems, whose inputs are constrained. Copyright © 2011 John Wiley & Sons, Ltd.