An online policy iteration for adaptive optimal control of unknown bilinear systems

Bellman's optimality principle states that designing an optimal controller for continuous-time bilinear systems with known system dynamics has a high computational complexity. As a result, controller design typically uses approximation techniques that depend on system dynamics knowledge. This problem will become more challenging when the system dynamics are unknown. Identifying the bilinear system dynamics through identification techniques is the first step toward overcoming this. It is well known that the identification methods give the designer a linear model to use in the controller design, based on the input and output data of the system. This paper proposes a new iterative method to design an optimal controller for a bilinear system whose dynamics are unknown, using an online adaptive policy iteration. In the proposed iterative method, instead of knowing the dynamics of the bilinear system, the optimal controller is designed by using the online input information and measurement of states. Also, by applying noise as an input for the system in a certain time interval, the need to measure the states for the next iterations is eliminated. The convergence of the adaptive iterative process to the optimal controller has been presented and proved in a theorem.