A novel neural network discrete-time optimal control design for nonlinear time-delay systems using adaptive critic designs

In this article, a novel neural network (NN) optimal control approach using adaptive critic designs is developed for nonlinear discrete-time (DT) systems with time delays. First, to eliminate the delay term of control input, a time-delay matrix function is developed by designing a M network. Furthermore, the cost function is approximated by the critic NN, and the control signal can be obtained directly by using the information of critic NN according to the equilibrium condition. In addition, to shorten the learning time and reduce the computational burden in the control process, a novel control strategy with less adjustable parameters for the time-delay DT nonlinear systems is proposed in this article, in which the norm of the weight estimations of critic NN is updated to generate a novel long-term performance function. The proposed control algorithm using adaptive critic designs has the advantage of reducing adaptive learning parameters and lessening calculative burden. The Lyapunov stability analysis shows that the time-delay DT controlled systems can be uniformly ultimately bounded stable. Finally, three simulations are presented to demonstrate the control performance of the developed method.     

Online identifier–actor–critic algorithm for optimal control of nonlinear systems

In this paper, a novel identifier–actor–critic optimal control scheme is developed for discrete‐time affine nonlinear systems with uncertainties. In contrast to traditional adaptive dynamic programming methodology, which requires at least partial knowledge of the system dynamics, a neural‐network identifier is employed to learn the unknown control coefficient matrix working together with actor–critic‐based scheme to solve the optimal control online. The critic network learns the approximate value function at each step. The actor network attempts to improve the current policy based on the approximate value function. The weights of all neural networks are updated at each sampling instant. Lyapunov theory is utilized to prove the stability of closed‐loop system. It shows that system states and neural network weights are uniformly ultimately bounded. Finally, simulations are provided to illustrate the effectiveness of the developed method. Copyright © 2016 John Wiley & Sons, Ltd.