Guaranteed cost solution of optimal control and game problems for uncertain systems

A class of systems having white noise parameter variations is considered. Solutions of the infinite-time quadratic optimal control (LQOC) and differential game (LQDC) problems are considered. It is shown that guaranteed cost solutions of these problems are possible if the effects of parameter uncertainties are first taken into account in a specified manner. The method is illustrated through a numerical example.     

On the optimal control of the Vidale-Wolfe advertising model

The Vidale-Wolfe advertising model is a singular optimal control problem with a non negative control. Sufficient conditions on a generalized time-varying market, for which a solution can be found, are given. Time is parametrized to describe impulsive optimal trajectories in a conventional manner. The solution is then found and verified by using the Hamilton-Jacobi-Bellman equation on the parametrized problem. © 1997 John Wiley & Sons, Ltd.     

Singular perturbation analysis of the closed-loop discrete optimal control problem

The closed-loop optimal control of a linear, time-invariant, singularly perturbed discrete system is considered. The resulting matrix Riccati difference equation is formulated in the singularly perturbed structure. It is observed that the degeneration affects some of the final conditions of the Riccati equation. A singular perturbation method is developed to obtain approximate solutions in terms of an outer series and a final correction series. The outer series takes advantage of the order reduction associated with degeneration and the correction series takes care of the affected final conditions. Two examples are given to illustrate the proposed method.     

The infinite-order singular problem

In singular optimal control problems, the optimal control is determined by solving the algebraic equation which results by successively differentiating the switching function until the control appears explicitly. In certain classes of problems, the control never appears, and such problems are termed infinite-order singular problems. It is shown that this class has many useful properties with respect to the theory and computation of optimal controls. In particular, it is shown that for the time-invariant, singular, linear-quadratic problem: (i) the singular order is infinity or less than or equal to the state dimension, (ii) infinite-order problems can arise only from exact differential type cost functions, (iii) the range of the second-variation operator (Hessian) is finite-dimensional, (iv) the computational method converges strongly, and (v) conjugate direction methods converge in a finite number of steps. The latter property is especially useful in the generation of test problems for optimal control computation schemes.     

A primal-dual method for linear-quadratic gaussian control problems with quadratic constraints

Linear-quadratic Gaussian control with quadratic state and input constraints is studied. A primal-dual method is considered in which the solution of a dual maximization problem is found by solving a sequence of unconstrained linear-quadratic Gaussian control problems. Second derivatives are derived for the dual problem and Newton-type methods are applied to its solution. Non-negativity constraints on the Lagrange multipliers of the primal problem lead to corresponding constraints in the dual problem, and for these both an active set strategy and a method based on quadratic subproblems are considered. A numerical example is presented to illustrate the performance of the numerical algorithms.     

A combined state and state estimate feedback solution to the ship positioning control problem

A stochastic linear quadratic optimal control problem is considered in which some of the plant states may be measured without a measurement noise component. This set of states are assumed to be associated with the plant inputs and force transducers. The optimal controller is shown to include state feedback from this part of the system. The states which cannot be measured are assumed to be combined in noisy output signal. The optimal controller corresponding to this second subsystem is shown to include a Kalman filter and state-estimate feedback. The combination of state and state-estimate feedback has the advantage that the dimension of the Kalman filter is equal to that of the second subsystem mentioned above. In the conventional solution to this problem, no states are assumed measurable, and the dimension of the Kalman filter is equal to the dimension of the complete system. In many industrial control problems, the combined control law enables a significant reduction in the dimension of the filter to be achieved. The technique has been proposed for use in dynamic ship positioning control systems, and this problem is discussed.     

A Riccati-equation-based algorithm for continuous-time optimal control problems

In this paper we consider continuous-time unconstrained optimal control problems. We propose a computational method which is essentially based on the closed-loop solutions of the linear quadratic optimal control problems. In the proposed algorithm, Riccati differential equations play an important role. We prove that accumulation points generated by the present algorithm, if they exist, satisfy the weak necessary conditions for optimality, under some assumptions including Kalman's sufficient conditions for the bounded Riccati solutions. In addition, we also propose the simple but effective technique to guarantee the boundedness of the solutions of Riccati equations. Lastly, we illustrate the usefulness of the present algorithm through simulation experiences. Copyright © 1998 John Wiley & Sons, Ltd.     

A Newton-like approximation algorithm for the steady-state solution of the riccati equation for time-varying systems

An approximation technique is developed for the steady-state solution of the time-varying matrix Riccati equation. We show how the Newton-type algorithm of Kleinman, developed for computing the steady solution to the algebraic Riccati equation for time-invariant systems, can be extended for time-varying linear systems. The time-varying case is considerably more involved than the time-invariant one. Consider a linear time-varying system x (t) = F (t) x (t) + G (t) u (t). If ( F , G ) is uniformly completely controllable, we show how one can construct a recursive sequence of matrix functions (using linear techniques) which converge to the steady-state solution of the associated time-varying matrix Riccati equation (a non-linear object). At each successive state, the next approximation is in terms of the steady-state solution to a linear Lyapunov differential equation (which is the extension of the algebraic Lyapunov equations used by Kleinman) for which an explicit expression exists. This provides an approximation technique for obtaining infinite-time, linear-quadratic, optimal controllers and steady-state Kalman—Bucy filters for time-varying systems using purely linear techniques. Thus, we provide new types of suboptimal stabilizing feedback laws for linear time-varying systems.     

The solution of finite-time optimal control problems with control time delays

The design of deterministic LQP optimal control systems is considered for plants with control delays and a finite optimization interval. First, a solution is obtained to the fixed end-point optimal control problem. An expression for the open-loop optimal control is derived in the s-domain. The closed-loop optimal time-varying gain matrix is then calculated from the s-domain results. The open-loop and closed-loop solutions to the free end-point optimal control problem are also given. The constant gain, infinite-time, feedback control law is obtained as a limiting case of these results. The receding-horizon optimal control problem for plants with control signal delays is also considered. The solution to this problem yields a constant feedback gain matrix which has obvious advantages for implementation. This gain matrix is not the same as the constant solution to the infinite-time problem. The receding-horizon control laws are derived for both the fixed and free end-point problems, and these are shown to produce asymptotically stable closed-loop systems.     

Second-order necessary optimality conditions for a discrete optimal control problem with nonlinear state equations

In this article, we study second-order necessary optimality conditions for a discrete optimal control problem with a nonconvex cost function, nonlinear state equations and mixed constraints. In order to achieve these conditions, we first establish an abstract result on the second-order necessary optimality conditions for a mathematical programming problem and then we derive the second-order necessary optimality conditions for a discrete optimal control problem. The main result of this article is illustrated by two examples.     

Singular perturbation method applied to the closed-loop discrete optimal control problem

An improved singular perturbation method for the singularly perturbed, closed-loop discrete optimal control problem with two terminal boundary layer correction series is presented.     

Direct solution to an optimal control problem of econometric systems

A direct solution is proposed to an optimal control problem of linear econometric systems with a quadratic welfare loss function when there are linear equality constraints on the control variables. The direct solution proposed here eliminates the problem of non-uniqueness of the optimal solution, which is present when this optimal control problem is solved using the recursive algorithm proposed by Chow,¹ Pindyck² and Tan.³ If a unique solution to the optimal control problem exists, then the direct solution and the recursive solution coincide.     

An extended penalty function approach to the numerical solution of constrained optimal control problems

This paper presents the extended penalty function method for solving constrained optimal control problems. Here, equality and inequality constraints on the state and control variables are considered. Using the extended penalty function method, the original constrained optimal control problem is transformed into a sequence of optimal control problems without inequality constraints. This is accomplished by adding to the cost functional a penalty term that takes on large values when the inequality constraints are violated and small values when the constraints are satisfied. Also presented is a continuation method for solving the sequence of differential-algebraic boundary value problems arising from the transformed optimal control problems. The effectiveness of the approach is demonstrated via examples.     

A novel continuous genetic algorithm for the solution of optimal control problems

In this paper, the Continuous Genetic Algorithm (CGA), previously developed by the principal author, is applied for the solution of optimal control problems. The optimal control problem is formulated as an optimization problem by the direct minimization of the performance index subject to constraints, and is then solved using CGA. In general, CGA uses smooth operators and avoids sharp jumps in the parameter values. This novel approach possesses two main advantages when compared to other existing direct and indirect methods that either suffer from low accuracy or lack of robustness. First, our method can be applied to optimal control problems without any limitation on the nature of the problem, the number of control signals, and the number of mesh points. Second, high accuracy can be achieved where the performance index is globally minimized while satisfying the constraints. The applicability and efficiency of the proposed novel algorithm for the solution of different optimal control problems is investigated. Copyright © 2010 John Wiley & Sons, Ltd.     

On linear quadratic optimal control of discrete-time complex-valued linear systems

We study in this paper the linear quadratic optimal control (linear quadratic regulation, LQR for short) for discrete-time complex-valued linear systems, which have several potential applications in control theory. Firstly, an iterative algorithm was proposed to solve the discrete-time bimatrix Riccati equation associated with the LQR problem. It is shown that the proposed algorithm converges to the unique positive definite solution (bimatrix) to the bimatrix Riccati equation with appropriate initial conditions. With the help of this iterative algorithm, the LQR problem for the antilinear system, which is a special case of complex-valued linear system, was carefully examined and three different Riccati equations–based approaches were provided, namely, bimatrix Riccati equation, anti-Riccati equation, and normal Riccati equation. The established approach is then used to solve the LQR problem for a discrete-time time-delay system with one-step state delay, and a numerical example was used to illustrate the effectiveness of the proposed methods.     

Solvability for indefinite mean-field stochastic linear quadratic optimal control with random jumps and its applications

In this article, we study the open-loop and closed-loop solvability for indefinite mean-field stochastic linear quadratic (MF-SLQ) optimal control problem and its application in finance, where the controlled stochastic system is driven by a Brownian motion and a Poisson random martingale measure and also disturbed by some stochastic processes. The intrinsic property of stochastic systems results in the inequivalence of those two solvabilities, which is different from deterministic case. Based on a well-posedness result of the problem, it is shown that the uniform convexity of cost functional is sufficient for the open-loop solvability of the problem. By a matrix minimum principle, the necessity condition of regular solvability for a decoupled Riccati equation is established, meanwhile, the closed-loop solvability is turned out to be equivalent to the regular solvability of Riccati equations with some constraints on the solutions of disturbances equations, moreover, the optimal closed-loop strategy is characterized by regular solutions of Riccati equations and adapted solutions of disturbances equations. And then a mean-variance model is considered to solve optimal portfolio selection strategy problem of the insurance company with liability. Our study fills a gap in the field of solvability for mean-field stochastic optimal control with random jumps and provides a different way for solving mean-variance portfolio selection model.     

On a control problem governed by a linear partial differential equation with a smooth functional

In this paper, the optimal control problem for the Helmholtz equation with non‐local boundary conditions is considered. The necessary and sufficient conditions of optimality in a maximum principle form have been obtained. We note that this problem is basically different from classical‐type problems because it is impossible to use Green's formula and we cannot rewrite it in the variational form widely used in the literature. So it is impossible to use all the theory that has been developed for optimal control problems with classical boundary conditions. Copyright © 2009 John Wiley & Sons, Ltd.     

Nonlinear optimal control of washing machine based on approximate solution of HJB equation

Performance of a washing machine depends on different parameters, namely water temperature, degree of soiling, water volume, and motor speed. In this paper, firstly, a nonlinear model is derived based on the empirical data. In addition, validity of the proposed model is also verified. Secondly, to optimize the performance, a quadratic energy function is considered, which is minimized subject to the nonlinear model of the Washing Machine. Finding the corresponding control law requires solving a partial differential equation called Hamiltonian–Jacobi–Bellman (HJB) equation. An approximate solution for HJB equation using the second‐, third‐, and fourth‐order terms of Taylor's series expansion is utilized. Simulation results reveal that the higher‐order controller leads not only to a lower cost function but also to a better performance. Copyright © 2007 John Wiley & Sons, Ltd.     

Semi‐decentralized approximation of optimal control of distributed systems based on a functional calculus

This paper discusses a new approximation method for operators that are solution to an operational Riccati equation. The latter is derived from the theory of optimal control of linear problems posed in Hilbert spaces. The approximation is based on the functional calculus of self‐adjoint operators and the Cauchy formula. Under a number of assumptions, the approximation is suitable for implementation on a semi‐decentralized computing architecture in view of real‐time control. Our method is particularly applicable to problems in optimal control of systems governed by partial differential equations with distributed observation and control. Some relatively academic applications are presented for illustration. More realistic examples relating to microsystem arrays have already been published. Copyright © 2014 John Wiley & Sons, Ltd.     

Minimizing the maximum heating of a re-entering space shuttle: An optimal control problem with multiple control constraints

This paper presents a numerical investigation of an optimal re-entry manoeuvre under several control and control-state constraints. The essential aim of the optimization is the minimization of the maximal skin temperature of an orbiter. It is demonstrated that the interaction of different solution techniques is indispensable in order to successfully treat such a highly constrained problem. The reduction of the skin temperature is significant. Moreover, the maximum heat flux and the integrated heat flux are also reduced considerably by the optimization.