General Prerequisites: Basic linear algebra (such as eigenvalues and eigenvectors of real matrices), multivariate real analysis (such as norms, inner products, multivariate linear and quadratic functions, basis) and multivariable calculus (such as Taylor expansions, multivariate differentiation, gradients).
Course Overview: The solution of optimal decision-making and engineering design problems in which the objective and constraints are nonlinear functions of potentially (very) many variables is required on an everyday basis in the commercial and academic worlds. A closely-related subject is the solution of nonlinear systems of equations, also referred to as least-squares or data fitting problems that occur in almost every instance where observations or measurements are available for modelling a continuous process or phenomenon, such as in weather forecasting. The mathematical analysis of such optimization problems and of classical and modern methods for their solution are fundamental for understanding existing software and for developing new techniques for practical optimization problems at hand.
Course Synopsis: Part 1: Unconstrained Optimization
Optimality conditions, steepest descent method, Newton and quasi-Newton methods, General line search methods, Trust region methods, Least squares problems and methods.
Part 2: Constrained Optimization
Optimality/KKT conditions, penalty and augmented Lagrangian for equality-constrained optimization, interior-point/ barrier methods for inequality constrained optimization. SQP methods.
