C8.7 Optimal Control (2024-25)
Main content blocks
- Lecturer: Profile: Samuel Cohen
None, but some basic knowledge of stochastic calculus (as would be obtained from B8.2 Continuous Martingales and Stochastic Calculus or B8.3 Mathematical Models of Financial Derivatives) will be assumed. Familiarity with applied PDE (as would be obtained from B5.2 Applied Partial Differential Equations, B6.1 Numerical Solution of Partial Differential Equations, or B7.1 Classical Mechanics) would also be beneficial.
Optimal control is the question of how one should select actions sequentially through time. The problem appears, in various forms, in many applications, from industrial problems and classical mechanics through to problems in biology and finance. This course will study the mathematics required to understand these problems, both in discrete and continuous time, and in settings with and without randomness. The two main perspectives on control -- dynamic programming and the Pontryagin principle -- will be explored, along with how these perspectives lead to equations that describe the optimal action. The numerical solution of these equations will also be considered.
The students will develop an understanding of the classical theory of optimal control, and be able to determine optimal controls within mathematical models. They will be familiar with manipulating the corresponding PDEs, and with reinforcement learning techniques to solve them.
Dynamic programming in discrete time, the Bellman equation and value function. Iteration methods for discrete systems and variations from reinforcement learning. Continuous deterministic systems and the Hamilton--Jacobi equation. Pontryagin maximum principle for deterministic systems. Feynman--Kac theorem and values in stochastic systems, the Linear-Quadratic-Gaussian case. Martingale characterization of optimality and the Hamilton--Jacobi--Bellman equation, maximum principle and verification theorem. Examples from finance and engineering.
Section outline
-
-
Due: Monday, 3 February 2025, 5:00 PM
Corrected version (with fewer typos) now available
-
-
-
-
Registration start: Monday, 13 January 2025, 12:00 PMRegistration end: Friday, 14 February 2025, 12:00 PM
-
Class Tutor's Comments Assignment
Class tutors will use this activity to provide overall feedback to students at the end of the course.
-