An off-policy approach for model-free stabilization of linear systems subject to input energy constraint and its application to spacecraft rendezvous

This note is concerned with the problem of stabilizing a class of linear continuous-time systems with completely unknown system dynamics subject to input energy constraint. To deal with this problem, a model-based low gain feedback law is designed firstly by establishing a special algebraic Riccati equation. Such a low gain feedback law can semiglobally stabilize the linear systems subject to input energy constraint with the exact system model. In order to relax the assumption that the system model is exactly known, an off-policy reinforcement learning approach is designed to solve the same problem without requiring the completely knowledge of the system dynamics. Finally, in order to verify the effectiveness of the proposed model-free approach, simulation result on the spacecraft rendezvous problem is introduced.     

Nonlinear model predictive control strategy for low thrust spacecraft missions

In this paper, two nonlinear model predictive control (MPC) strategies are applied to solve a low thrust interplanetary rendezvous problem. Each employs a unique, nonclassical parameterization of the control to adapt the nonlinear MPC approach to interplanetary orbital dynamics with low control authority. The approach is demonstrated numerically for a minimum‐fuel Earth‐to‐Mars rendezvous maneuver, cast as a simplified coplanar circular orbit heliocentric transfer problem. The interplanetary transfer is accomplished by repeated solution of an optimal control problem over (i) a receding horizon with fixed number of control subintervals and (ii) a receding horizon with shrinking number of control subintervals, with a doubling strategy to maintain controllability. In both cases, the end time is left unconstrained. The performances of the nonlinear MPC strategies in terms of computation time, fuel consumption, and transfer time are compared for a constant thrust nuclear‐electric propulsion system. For this example, the ability to withstand unmodeled effects and control allocation errors is verified. The second strategy, with shrinking number of control subintervals, is also shown to easily handle the more complicated bounded thrust nuclear‐electric case, as well as a state‐control‐constrained solar‐electric case. Copyright © 2012 John Wiley & Sons, Ltd.     

Time‐optimal reorientation of the rigid spacecraft using a pseudospectral method integrated homotopic approach

This paper proposes a robust algorithm for time‐optimal rigid spacecraft reorientation trajectory generation. Based on the Pontryagin's maximum principle, the first‐order necessary optimality conditions are derived. These optimality conditions are numerically satisfied by adopting a pseudospectral method integrated homotopic approach to solve the associated shooting functions. First, the energy‐optimal reorientation solution is obtained using the Radau pseudospectral method, which has a spectral convergence speed and can give a precise estimation of the initial costates used to start the homotopic approach. Then, a modified homotopy scheme is given to deform the associated energy‐optimal solution to the desired time‐optimal solution continuously. Finally, for the inertially symmetric spacecraft reorientation problem, the newly found time‐optimal solutions are presented. The performance of the algorithm is illustrated by simulating a general asymmetric rigid spacecraft time‐optimal reorientation problem. Copyright © 2014 John Wiley & Sons, Ltd.     

An optimal dynamic output feedback controller for a flexible spacecraft

The design of a low-order dynamic output feedback controller for a flexible spacecraft is carried out using modified linear quadratic Gaussian (LQG) theory. The necessary conditions for optimization are given. The linear equations governing a flexible spacecraft with a central rigid body and two sun-tracking solar panels are presented. The parameters of the Solar Electric Propulsion Spacecraft (SEPS) are selected for analysis and simulation. The optimal gains for the dynamic controller are estimated using an iterative algorithm. The sequential procedure which assures convergence is selected. The initial gains which stabilize the system are chosen on the basis of the principle of a PD controller. A third-order controller for pitch, roll and yaw axes of the 18th-order system, which includes sensor and actuator dynamics, is designed. Numerical simulations carried out to ascertain the performance of the controller show the performance to be satisfactory.     

Lateral approach to the orbit: An anatomical study

The authors have identified and described three lateral approaches to the intraorbital portion of the optic nerve in cadavers. The structures exposed via the superior, inferior or central approaches were directed through the lateral orbital wall. The superior approach in which the lateral rectus muscle is retracted inferiorly exposes the superolateral aspect of the orbit. The inferior approach in which the lateral rectus muscle is retracted superiorly, provided the inferolateral aspect of the optic nerve from the globe to the muscle cone. The central approach in which the lateral rectus muscle was disinserted and retracted provided excellent exposure of the structures of the lateral part of the orbit and the optic nerve. There were no structures blocking access to the ciliary ganglion and two third anterior optic nerve and lateral surface of the globe. The structures encountered in these approaches are demonstrated under magnification in cadavers, and the exposure of the optic nerve is discussed.     

Solving fractional optimal control problems by new Bernoulli wavelets operational matrices

In this article, a new numerical method based on Bernoulli wavelet basis has been applied to give the approximate solution of the fractional optimal control problems. The new operational matrices of multiplication and fractional integration are constructed. The proposed method is applied to reduce the problem to the solution of a system of algebraic equations. The fractional derivative is considered in the Caputo sense. Convergence of the algorithm is proved and some results concerning the error analysis are obtained. Approximate solutions are given and in the cases when we have an exact solution, a comparison with the exact solution is presented to demonstrate the validity and applicability of the proposed method. In addition, we compare the obtained results with the results of other methods. Comparison shows the more accuracy of presented technique in comparison to other published methods.     

An extended penalty function approach to the numerical solution of constrained optimal control problems

This paper presents the extended penalty function method for solving constrained optimal control problems. Here, equality and inequality constraints on the state and control variables are considered. Using the extended penalty function method, the original constrained optimal control problem is transformed into a sequence of optimal control problems without inequality constraints. This is accomplished by adding to the cost functional a penalty term that takes on large values when the inequality constraints are violated and small values when the constraints are satisfied. Also presented is a continuation method for solving the sequence of differential-algebraic boundary value problems arising from the transformed optimal control problems. The effectiveness of the approach is demonstrated via examples.     

Exact penalty function approach to constrained optimal control problems

Given the well known concept that optimal control problems may be solved either by the maximum principle or by the dynamic programming technique which employs many numerical algorithms, this paper attempts to show that the exact penalty function method may be used to transform a constrained optimal control problem into an unconstrained optimal control problem. Under certain conditions the constrained optimal control problem is shown to be equivalent to an unconstrained optimal control problem, which can be easily solved by a numerical technique.     

Analysis of the inertially symmetric rigid spacecraft time‐optimal three‐axis reorientation

This paper investigates the classical time‐optimal rest‐to‐rest three‐axis reorientation of the inertially symmetric rigid spacecraft. First‐order necessary optimality conditions are derived from the Pontryagin's maximum principle. Then, control structures (i.e., switching times and control torques) for the time‐optimal solution with five, six, and seven switches are given. For any five‐switch, six‐switch, or seven‐switch time‐optimal solution, a finite number of control structures exist, and relations among the control structures and their associated time‐optimal solutions are analytically derived. By utilizing the control structure, efficient numerical optimization algorithm based on multiple‐interval Radau pseudospectral method is proposed. Numerical results show that, after rounding to integer, five‐switch and six‐switch time‐optimal solutions exist for rotation angles on the interval [1,180] deg, and s es on the interval [1,72] deg. Finally, time‐optimal solutions for typical rotation angles are given to illustrate and validate the new findings. Copyright © 2016 John Wiley & Sons, Ltd.     

State parameterization approach to the solution of optimal control problems

A direct method for solving optimal control problems by parameterizing state variables is developed. This state parameterization (SP) procedure is first formulated for a general class of problem. Next, a simpler and more efficient alternative version of the SP algorithm is developed for an important special class of problems, namely those involving systems that can be cast into a multivariable non-linear phase-variable form. The modified SP algorithm in particular avoids explicit integration of the system state and costate differential equations. The algorithm is tested on two problems which are parameterized in terms of cubic splines.     

An antiaging and regenerative medicine approach to optimal skin health

The optimal health and appearance of the skin is the result of several factors: extrinsic aging (UV damage), intrinsic aging (genetics, lifestyle), and hormonal aging (primarily estrogen loss). This article intends to take an antiaging and regenerative medicine approach to the aging skin and focus on both the hormonal causes and the intrinsic causes of aging; namely, hormones, diet, and lifestyle. By taking a functional approach to the evaluation and treatment of the skin, we hope to elucidate the causation of the condition of the skin and provide targeted treatments to improve its quality. We will sequentially cover the topics of hormonal decline and the skin, understanding hormone replacement, aging and the skin, and key nutrients for youthful skin. We hope to provide a more comprehensive approach to treating the aging face and skin that will provide patients with more long-lasting and youthful results.     

An Optimal control approach to marketing-production planning

The problem of finding an optimal advertising and production policy in a firm is analysed using a recently proposed model of a marketing-production system. First, it is shown that the optimal control problem underlying the proposed model is a partially singular control problem. Then, a reverse-time parametric solution procedure is designed to determine the optimal advertising and production policy for the proposed model. Finally, it is shown that the results derived from the new model are applicable to problems of capacity expansion in a firm.     

An integrated approach to reduced-order control theory

Linear-Quadratic-Gaussian (LQG) theory generally requires the use of a full-order dynamic observer whose dimension is often prohibitive in applications so that reduced-order controller design techniques are needed. A general formalism is presented here for the optimal design of controllers of arbitrary prescribed order with the usual quadratic cost functionals. The gradients of the cost functional are obtained in explicit form and involves Liapunov equations. A fundamental aspect of this paper is a unifying formalism; direct output feedback and reduced-order controllers, stochastic and deterministic systems, infinite-time and finite-time problems and time-invariant and time-varying problems are considered. Optimization of sensor and actuator locations and parameter-variation-insensitive controller design are also considered within this framework. Examples are treated.     

Bilinear quadratic optimal control: a recursive approach

This paper deals with the time-varying bilinear quadratic optimal control problem. Using Adomian's decomposition method, we shall first derive a functional expansion for the input–output map of the system, then transform the cost functional so that it yields the optimal control in a recursive manner. The optimal tracking problem is considered to illustrate the theory. An alternative method is derived which is proved to be more ‘robust’. © 1997 John Wiley & Sons, Ltd.     

An approach to variable time receding horizon control

This paper proposes a method for the design of a predictive controller with variable step sizes. Instead of using a fixed sampling frequency, the switching times are incorporated into an optimization problem, thus resulting in an adaptive step‐size control process. The controller with the variable time steps is shown to require less tuning and reduces the number of expensive model evaluations. An alternate solution approach had to be implemented to accommodate the new formulation. Stability is proven in a context that does not require terminal cost or constraints. Copyright © 2011 John Wiley & Sons, Ltd.     

Further results on the optimal regulation of spacecraft angular momentum

A direct analytical solution of angular velocities is given for regulating the angular momentum of a spacecraft using reaction jets only. The solution, which appears to be new, may be written down in terms of characteristic values and characteristic vectors. These values and vectors satisfy an algebraic non-linear equation.     

Using constrained optimal control to model a socially sustainable fishery

In the so‐called sole‐owner‐case of a renewable resource harvesting, it is known that the optimal policy includes a period with no harvesting, if the population is insufficiently small. Therefore, the fishermen have no income in this phase. To avoid this problem, the optimal control problem was enhanced in two different ways. First, the capital accumulation rate was bounded from below what generated a global mixed constraint. Second, lower bounds for the capital stock at a finite number of time points were introduced, which at first led to one and afterwards several isolated inequality constraints. For the resulting constrained optimal control problems, the necessary conditions were derived and an iterative algorithm was implemented to calculate the solutions in several numerical examples. Copyright © 2010 John Wiley & Sons, Ltd.     

A decentralized receding horizon optimal approach to formation control of networked mobile robots

This paper presents a receding horizon optimal controller with guaranteed stability for multirobot formation, taking into account collision and obstacle avoidance. The proposed scheme is based on synchronous decentralized strategy wherein all the vehicles that are connected via a packed‐delaying network solve a finite horizon–constrained optimal control problem to obtain their own control action at each sampling instant. First, each robot is modeled by a single integrator dynamics; then, by defining a control law for each robot and considering the effect of communication delay, the closed‐loop dynamics is described as a delay differential equation with tunable parameters. Afterwards, a novel finite‐horizon optimal control setup is established to obtain these adjustable gains such that a desirable formation is achieved. The efficiency and applicability of the suggested scheme are demonstrated by simulation results.     

Unified approach for open‐loop optimal control

We develop a unified approach for the necessary conditions for optimization of open‐loop control systems, starting from the basic principles of calculus of variations. The unified approach results are simultaneously applicable to both shift (q)‐operator‐based, discrete‐time systems and the derivative (d/dt)‐operator‐based, continuous‐time systems. It is shown that the optimal condition results that are now obtained separately for continuous‐time and discrete‐time systems can be easily obtained from the unified approach. An illustrative example is given. Copyright © 2006 John Wiley & Sons, Ltd.     

A semidefinite programming approach for polynomial switched optimal control problems

This paper considers solving optimal control of switched systems with polynomial data globally, where the number of switches and the switching signal are not preassigned. The problem is transformed into an embedded polynomial form in which a continuous variable controls the switching policy. Then, using occupation measures, the embedded optimal control problem is formulated as an infinite‐dimensional linear programming (LP) over the space of measures. A polynomial sum‐of‐squares strengthening corresponding to conic dual of the measure LP problem provides an approximating optimal feedback control for both classical control and the switching signal. Furthermore, the optimal switching signal is calculated without mode projection. Finally, three simulation experiments are included to confirm the theoretical results in the paper.