Course: C6.2 Continuous Optimisation (2021-22)

Lecturer: Profile: Coralia Cartis

Course information

General prerequisites:

Basic linear algebra (such as eigenvalues and eigenvectors of real matrices), multivariate real analysis (such as norms, inner products, multivariate linear and quadratic functions, basis) and multivariable calculus (such as Taylor expansions, multivariate differentiation, gradients).

Course term: Hilary

Course lecture information: 16 lectures

Course weight: 1

Course level: M

Assessment type: Written Examination

Course overview:

The solution of optimal decision-making and engineering design problems in which the objective and constraints are nonlinear functions of potentially (very) many variables is required on an everyday basis in the commercial and academic worlds. A closely-related subject is the solution of nonlinear systems of equations, also referred to as least-squares or data fitting problems that occur in almost every instance where observations or measurements are available for modelling a continuous process or phenomenon, such as in weather forecasting. The mathematical analysis of such optimization problems and of classical and modern methods for their solution are fundamental for understanding existing software and for developing new techniques for practical optimization problems at hand.

Course synopsis:

Part 1: Unconstrained Optimization
Optimality conditions, steepest descent method, Newton and quasi-Newton methods, General line search methods, Trust region methods, Least squares problems and methods.
Part 2: Constrained Optimization
Optimality/KKT conditions, penalty and augmented Lagrangian for equality-constrained optimization, interior-point/ barrier methods for inequality constrained optimization. SQP methods.

Section outline

Select section General

Collapse Expand
General

Collapse all Expand all
- Select activity Announcements
  
  Announcements Forum
- Select activity Discussion Forum
  
  Discussion Forum
Select section Course Materials

Collapse Expand
Course Materials
- Select activity Lectures 1 and 2
  
  Introduction. Optimality conditions for unconstrained problems. (2021 Videos 1-3; Sheet 1)
  
  Lectures 1 and 2
  
  C62_Lecture1a.pdf
  
  C62_Lecture1b.pdf
  
  C62_Lecture2.pdf
- Select activity Lectures 3 and 4
  
  Methods for unconstrained optimization. Linesearch algorithms (2021 Videos 4-6; Sheet 2)
  
  Lectures 3 and 4
  
  C62_Lecture3a.pdf
  
  C62_Lecture3b.pdf
  
  C62_Lecture4.pdf
- Select activity Lecture 5
  
  Steepest descent methods (2021 Video 7, Sheet 2)
  
  Lecture 5
  
  C62_Lecture5.pdf
- Select activity Lecture 6
  
  Newton's method for unconstrained optimization (2021 Videos 8-9; Sheet 2)
  
  Lecture 6
  
  C62_Lecture6a.pdf
  
  C62_Lecture6b.pdf
- Select activity Lecture 7
  
  Quasi-Newton methods. Nonlinear least-squares and Gauss-Newton methods. (2021 Videos 10-11; Sheet 3)
  
  Lecture 7
  
  C62_Lecture7.pdf
- Select activity Lectures 8 and 9
  
  Trust region methods. (2021 Videos 12-14, Sheet 3)
  
  Lectures 8 and 9
  
  C62_Lecture8.pdf
  
  C62_Lecture9a.pdf
  
  C62_Lecture9b.pdf
- Select activity Lectures 10 and 11
  
  Optimality conditions for constrained problems. (2021 Videos 15-16, Sheet 4)
  
  Lectures 10 and 11
  
  C62_Lectures10and11.pdf
- Select activity Lecture 12
  
  Penalty methods for constrained optimization (2021 Videos 17, Sheet 4)
  
  Lecture 12
  
  C6.2_Lecture12.pdf
- Select activity Lecture 13
  
  Augmented Lagrangian methods (2021 Videos 18, Sheet 4)
  
  Lecture 13
  
  C62_Lecture13.pdf
- Select activity Lectures 14 and 15
  
  Interior point methods for inequality constrained optimization problems (2021 Videos 19-20, Sheet 4)
  
  Lectures 14 and 15
  
  C62_Lectures14and15.pdf
- Select activity Lecture 16
  
  SQP methods for constrained optimization (2021 Video 21)
  
  Lecture 16
  
  C62_Lecture16.pdf
- Select activity Additional lecture material - nonexaminable
  
  Complete proof of trust region methods convergence. Proof of second-order optimality conditions for constrained problems.
  
  Additional lecture material - nonexaminable
  
  Lecture9_CompleteProof_Trust_Region.pdf
  
  Lecture11_ProofTheorem19.pdf
- Select activity Additional Resources
  
  Mathematical Background. Resources and bibliography
  
  Additional Resources
  
  C62_MathematicalBackground.pdf
  
  C62_Resources.pdf
- Select activity Optimization Booklet
  
  Optimization Booklet
  
  Gould_Continuous_Optimization_Algorithms.pdf
- Select activity Sheet 1
  
  Sheet 1 Assignment
- Select activity Sheet 2
  
  Sheet 2 Assignment
- Select activity Sheet 3
  
  Sheet 3 Assignment
- Select activity Sheet 4
  
  Sheet 4 Assignment
- Select activity Numerical Laboratories
  
  Numerical Laboratories
  
  C62_NumericalAssignment1-2022.pdf
  
  C62_NumericalAssignment2-2022.pdf
  
  Newton method.m
  
  Rosenbrock function.m
  
  Scaled quadratic function.m
  
  Steepest descent method.m

Main content blocks

Section outline

General

Course Materials