Semi‐decentralized approximation of optimal control of distributed systems based on a functional calculus

This paper discusses a new approximation method for operators that are solution to an operational Riccati equation. The latter is derived from the theory of optimal control of linear problems posed in Hilbert spaces. The approximation is based on the functional calculus of self‐adjoint operators and the Cauchy formula. Under a number of assumptions, the approximation is suitable for implementation on a semi‐decentralized computing architecture in view of real‐time control. Our method is particularly applicable to problems in optimal control of systems governed by partial differential equations with distributed observation and control. Some relatively academic applications are presented for illustration. More realistic examples relating to microsystem arrays have already been published. Copyright © 2014 John Wiley & Sons, Ltd.     

Structure‐preserving local optimal control of mechanical systems

While system dynamics are usually derived in continuous time, respective model‐based optimal control problems can only be solved numerically, ie, as discrete‐time approximations. Thus, the performance of control methods depends on the choice of numerical integration scheme. In this paper, we present a first‐order discretization of linear quadratic optimal control problems for mechanical systems that is structure preserving and hence preferable to standard methods. Our approach is based on symplectic integration schemes and thereby inherits structure from the original continuous‐time problem. Starting from a symplectic discretization of the system dynamics, modified discrete‐time Riccati equations are derived, which preserve the Hamiltonian structure of optimal control problems in addition to the mechanical structure of the control system. The method is extended to optimal tracking problems for nonlinear mechanical systems and evaluated in several numerical examples. Compared to standard discretization, it improves the approximation quality by orders of magnitude. This enables low‐bandwidth control and sensing in real‐time autonomous control applications.     

A variable time transformation method for mixed‐integer optimal control problems

The article discusses a variable time transformation method for the approximate solution of mixed‐integer non‐linear optimal control problems (MIOCP). Such optimal control problems enclose real‐valued and discrete‐valued controls. The method transforms MIOCP using a discretization into an optimal control problem with only real‐valued controls. The latter can be solved efficiently by direct shooting methods. Numerical results are obtained for a problem from automobile test‐driving that involves a discrete‐valued control for the gear shift of the car. The results are compared to those obtained by Branch&Bound and show a drastic reduction of computation time. This very good performance makes the suggested method applicable even for many discretization points. Copyright © 2006 John Wiley & Sons, Ltd.     

Time‐optimal control of automobile test drives with gear shifts

We present a numerical method and results for a recently published benchmark problem (Optim. Contr. Appl. Met. 2005; 26 :1–18; Optim. Contr. Appl. Met. 2006; 27 (3):169–182) in mixed‐integer optimal control. The problem has its origin in automobile test‐driving and involves discrete controls for the choice of gears. Our approach is based on a convexification and relaxation of the integer controls constraint. Using the direct multiple shooting method we solve the reformulated benchmark problem for two cases: (a) As proposed in (Optim. Contr. Appl. Met. 2005; 26 :1–18), for a fixed, equidistant control discretization grid and (b) As formulated in (Optim. Contr. Appl. Met. 2006; 27 (3):169–182), taking into account free switching times. For the first case, we reproduce the results obtained in (Optim. Contr. Appl. Met. 2005; 26 :1–18) with a speed‐up of several orders of magnitude compared with the Branch&Bound approach applied there (taking into account precision and the different computing environments). For the second case we optimize the switching times and propose to use an initialization based on the solution of (a). Compared with (Optim. Contr. Appl. Met. 2006; 27 (3):169–182) we were able to reduce the overall computing time considerably, applying our algorithm. We give theoretical evidence on why our convex reformulation is highly beneficial in the case of time‐optimal mixed‐integer control problems as the chosen benchmark problem basically is (neglecting a small regularization term). Copyright © 2009 John Wiley & Sons, Ltd.     

Solution of a specific class of nonlinear fractional optimal control problems including multiple delays

This research provides a new framework based on a hybrid of block‐pulse functions and Legendre polynomials for the numerical examination of a special class of scalar nonlinear fractional optimal control problems involving delay. The concepts of the fractional derivative and the fractional integral are employed in the Caputo sense and the Riemann‐Liouville sense, respectively. In accordance with the notion of the Riemann‐Liouville integral, we derive a new integral operator related to the proposed basis called the operational matrix of fractional integration. By employing two significant operators, namely, the delay operator and the integral operator connected to the hybrid basis, the system dynamics of the primal optimal control problem converts to another system involving algebraic equations. Consequently, the optimal control problem under study is reduced to a static optimization one that is solved by existing well‐established optimization procedures. Some new theoretical results regarding the new basis are obtained. Various kinds of fractional optimal control problems containing delay are examined to measure the accuracy of the new method. The simulation results justify the merits and superiority of the devised procedure over the existing optimization methods in the literature.     

State‐constrained optimal control applied to cell‐cycle‐specific cancer chemotherapy

In this paper, the optimal drug injection problem arising in cancer treatment by cell‐cycle‐specific chemotherapy is investigated. The optimal control problem is state constrained in which the stage cost reflects the concern of maximal drug injection, while the state constraint imposes a lower bound on the total number of cells in the bone marrow. It is shown that this problem can be approximately solved up to any desired precision by using an indexed family of state‐unconstrained optimal control problems. The state constraint is fulfilled for any member of the family. The existence of solutions is proved and the resulting approximation is characterized by appropriate two‐sided inequalities. Simulations are provided to show the efficiency and relevance of the proposed formulation. Copyright © 2007 John Wiley & Sons, Ltd.     

Optimal control problems with Atangana‐Baleanu fractional derivative

In this paper, we study fractional‐order optimal control problems (FOCPs) involving the Atangana‐Baleanu fractional derivative. A computational method based on B‐spline polynomials and their operational matrix of Atangana‐Baleanu fractional integration is proposed for the numerical solution of this class of problems. With this numerical technique, the FOCPs are reduced to a system of equations which are solved for the unknown parameters with the help of Mathematica® software. Our results show the applicability and usefulness of the numerical technique.     

A novel approach for the numerical investigation of optimal control problems containing multiple delays

This paper deals with the numerical solution of optimal control problems with multiple delays in both state and control variables. A direct approach based on a hybrid of block‐pulse functions and Lagrange interpolating polynomials is used to convert the original problem into a mathematical programming one. The resulting optimization problem is then solved numerically by the Lagrange multipliers method. The operational matrix of delay for the presented framework is derived. This matrix plays an imperative role to transfer information between 2 consecutive switching points. Furthermore, 2 upper bounds on the error with respect to the L²‐norm and infinity norm are established. Several optimal control problems containing multiple delays are carried out to illustrate the various aspects of the proposed approach. The simulation results are compared with either analytical or numerical solutions available in the literature.     

A back propagation through time‐like min–max optimal control algorithm for nonlinear systems

This paper presents a conjugate gradient‐based algorithm for feedback min–max optimal control of nonlinear systems. The algorithm has a backward‐in‐time recurrent structure similar to the back propagation through time (BPTT) algorithm. The control law is given as the output of the one‐layer NN. Main contribution of the paper includes the integration of BPTT techniques, conjugate gradient methods, Adams method for solving ODEs and automatic differentiation, to provide an effective, numerically robust algorithm for solving optimal min–max control problems. The proposed algorithm is evaluated on a robotic system with two DOFs. Copyright © 2012 John Wiley & Sons, Ltd.     

The role of symplectic integrators in optimal control

For general optimal control problems, Pontryagin's maximum principle gives necessary optimality conditions, which are in the form of a Hamiltonian differential equation. For its numerical integration, symplectic methods are a natural choice. This article investigates to which extent the excellent performance of symplectic integrators for long‐time integrations in astronomy and molecular dynamics carries over to problems in optimal control. Numerical experiments supported by a backward error analysis show that for problems in low dimension close to a critical value of the Hamiltonian, symplectic integrators have a clear advantage. This is illustrated using the Martinet case in sub‐Riemannian geometry. For problems like the orbital transfer of a spacecraft or the control of a submerged rigid body, such an advantage cannot be observed. The Hamiltonian system is a boundary value problem and the time interval is in general not large enough so that symplectic integrators could benefit from their structure preservation of the flow. Copyright © 2008 John Wiley & Sons, Ltd.     

An efficient scheme for minimax solutions of multiple linear‐quadratic control

Optimal control is one of the most important methodologies for studies of dynamic systems in many areas of sciences, engineering and economics. Minimax optimal control is a special topic in the general framework of multiple optimal control problems. Minimax optimal control can be considered as a dynamic game with multiple players under the same system. In this paper, we develop a fast search for a minimax solution of multiple linear‐quadratic control problems. The algorithm improves the existing solution scheme by adjusting the multiple weighting coefficients in each iteration and also including updates for step‐size control. Copyright © 2005 John Wiley & Sons, Ltd.     

Inverse optimal controller based on extended Kalman filter for discrete‐time nonlinear systems

In this study, we present an inverse optimal control approach based on extended Kalman filter (EKF) algorithm to solve the optimal control problem of discrete‐time affine nonlinear systems. The main aim of inverse optimal control is to circumvent the tedious task of solving the Hamilton‐Jacobi‐Bellman equation that results from the classical solution of a nonlinear optimal control problem. Here, the inverse optimal controller is based on defining an appropriate quadratic control Lyapunov function (CLF) where the parameters of this candidate CLF were estimated by adopting the EKF equations. The root mean square error of the system states is used as the observed error in the case of classical EKF algorithm application, whereas, here, the EKF tries to eliminate the same root mean square error defined over the parameters by generating a CLF matrix with appropriate elements. The performance and the applicability of the proposed scheme is illustrated through both simulations performed on a nonlinear system model and a real‐time laboratory experiment. Simulation study demonstrate the effectiveness of the proposed method in comparison with 2 other inverse control approaches. Finally, the proposed controller is implemented on a professional control board to stabilize a DC‐DC boost converter and minimize a meaningful cost function. The experimental results show the applicability and effectiveness of the proposed EKF‐based inverse optimal control even in real‐time control systems with a very short time constant.     

Exact penalty functions for optimal control problems I: Main theorem and free‐endpoint problems

In this two‐part study, we develop a general approach to the design and analysis of exact penalty functions for various optimal control problems, including problems with terminal and state constraints, problems involving differential inclusions, and optimal control problems for linear evolution equations. This approach allows one to simplify an optimal control problem by removing some (or all) constraints of this problem with the use of an exact penalty function, thus allowing one to reduce optimal control problems to equivalent variational problems and apply numerical methods for solving, eg, problems without state constraints, to problems including such constraints, etc. In the first part of our study, we strengthen some existing results on exact penalty functions for optimisation problems in infinite dimensional spaces and utilise them to study exact penalty functions for free‐endpoint optimal control problems, which reduce these problems to equivalent variational ones. We also prove several auxiliary results on integral functionals and Nemytskii operators that are helpful for verifying the assumptions under which the proposed penalty functions are exact.     

Minimizing control energy in a class of bounded‐control linear‐quadratic regulator problems

Minimal‐control‐energy strategies are substantiated and illustrated for linear‐quadratic problems with penalized endpoints and no state‐trajectory cost, when bounds in control values are imposed. The optimal solution for a given process with restricted controls, starting at a known initial state, is shown to coincide with the saturated solution to the unrestricted problem that has the same coefficients but starts at a generally different initial state. This result reduces the searching span for the solution: from the infinite‐dimensional set of admissible control trajectories to the finite‐dimensional Euclidean space of initial conditions. An efficient real‐time scheme is proposed here to approximate (eventually to find) the optimal control strategy, based on the detection of the appropriate initial state while avoiding as much as possible the generation and evaluation of state and control trajectories. Numerical (including model predictive control) simulations are provided, compared, and checked against the analytical solution to ‘the cheapest stop of a train’ problem in its pure‐upper‐bounded brake, flexible‐endpoint setting. Copyright © 2013 John Wiley & Sons, Ltd.     

Minimal‐power control of hydrogen evolution reactions

An integral approach to solve finite‐horizon optimal control problems posed by set‐point changes in electrochemical hydrogen reactions is developed. The methodology extends to nonlinear problems with regular, convex Hamiltonians that cannot be explicitly minimized, i.e. where the functional dependence of the H‐minimal control on the state and costate variables is not known. The Lagrangian functions determining trajectory costs will not have special restrictions other than positiveness, but for simplicity the final penalty will be assumed quadratic. The answer to the problem is constructed through the solution to a coupled system of three first‐order quasi‐linear partial differential equations (PDEs) for the missing boundary conditions x(T), λ(0) of the Hamiltonian equations, and for the final value of the control variable u(T). The independent variables of these PDEs are the time‐duration T of the process and the characteristic parameter S of the final penalty. The solution provides information on the whole (T, S)‐family of control problems, which can be used not only to construct the individual optimal control strategies online, but also for global design purposes. Copyright © 2009 John Wiley & Sons, Ltd.     

Adjoints estimation methods for impulsive Moon‐to‐Earth trajectories in the restricted three‐body problem

A study of optimal impulsive Moon‐to‐Earth trajectories is presented in a planar circular restricted three‐body framework. Two‐dimensional return trajectories from circular lunar orbits are considered, and the optimization criterion is the total velocity change. The optimal conditions are provided by the optimal control theory. The boundary value problem that arises from the application of the theory of optimal control is solved using a procedure based on Newton's method. Motivated by the difficulty of obtaining convergence, the search for the initial adjoints is carried out by means of two different techniques: homotopic approach and adjoint control transformation. Numerical results demonstrate that both initial adjoints estimation methods are effective and efficient to find the optimal solution and allow exploring the fundamental tradeoff between the time of flight and required ΔV. Copyright © 2014 John Wiley & Sons, Ltd.     

Solving mixed‐integer optimal control problems by branch&bound: a case study from automobile test‐driving with gear shift

The article discusses the application of the branch&bound method to a mixed integer non‐linear optimization problem (MINLP) arising from a discretization of an optimal control problem with partly discrete control set. The optimal control problem has its origin in automobile test‐driving, where the car model involves a discrete‐valued control function for the gear shift. Since the number of variables in (MINLP) grows with the number of grid points used for discretization of the optimal control problem, the example from automobile test‐driving may serve as a benchmark problem of scalable complexity. Reference solutions are computed numerically for two different problem sizes. A simple heuristic approach suitable for optimal control problems is suggested that reduces the computational amount considerably, though it cannot guarantee optimality anymore. Copyright © 2005 John Wiley & Sons, Ltd.     

Existence and stability analysis of optimal control

By constructing a complete metric space and a compact set of admissible control functions, this paper investigates the existence and stability of solutions of optimal control problems with respect to the right‐hand side functions. On the basis of set‐valued mapping theory, by introducing the notion of essential solutions for optimal control problems, some sufficient and necessary criteria guaranteeing the existence and stability of solutions are established. New derived criteria show that the optimal control problems whose solutions are all essential form a dense residual set, and so every optimal control problem can be closely approximated arbitrarily by an essential optimal control problem. The example shows that not all optimal control problems are stable. However, our main result shows that, in the sense of Baire category, most of the optimal control problems are stable. Copyright © 2013 John Wiley & Sons, Ltd.     

Solving multi‐objective dynamic optimization problems with fuzzy satisfying method

This article proposes a novel algorithm integrating iterative dynamic programming and fuzzy aggregation to solve multi‐objective optimal control problems. First, the optimal control policies involving these objectives are sequentially determined. A payoff table is then established by applying each optimal policy in series to evaluate these multiple objectives. Considering the imprecise nature of decision‐maker's judgment, these multiple objectives are viewed as fuzzy variables. Simple monotonic increasing or decreasing membership functions are then defined for degrees of satisfaction for these linguistic objective functions. The optimal control policy is finally searched by maximizing the aggregated fuzzy decision values. The proposed method is rather easy to implement. Two chemical processes, Nylon 6 batch polymerization and Penicillin G fed‐batch fermentation, are used to demonstrate that the method has a significant potential to solve real industrial problems. Copyright © 2003 John Wiley & Sons, Ltd.