Course: C6.2 Continuous Optimisation (2024-25)

Lecturer: Profile: Coralia Cartis

Course information

General prerequisites:

Basic linear algebra (such as eigenvalues and eigenvectors of real matrices), multivariate real analysis (such as norms, inner products, multivariate linear and quadratic functions, basis) and multivariable calculus (such as Taylor expansions, multivariate differentiation, gradients).

Course term: Hilary

Course lecture information: 16 lectures

Course weight: 1

Course level: M

Assessment type: Written Examination

Course overview:

The solution of optimal decision-making and engineering design problems in which the objective and constraints are nonlinear functions of potentially (very) many variables is required on an everyday basis in the commercial and academic worlds. A closely-related subject is the solution of nonlinear systems of equations, also referred to as least-squares or data fitting problems that occur in almost every instance where observations or measurements are available for modelling a continuous process or phenomenon, such as in weather forecasting. The mathematical analysis of such optimization problems and of classical and modern methods for their solution are fundamental for understanding existing software and for developing new techniques for practical optimization problems at hand.

Course synopsis:

Part 1: Unconstrained Optimization

Optimality conditions, steepest descent method, Newton and quasi-Newton methods, General line search methods, Trust region methods, Least squares problems and methods.

Part 2: Constrained Optimization

Optimality/KKT conditions, penalty and augmented Lagrangian for equality-constrained optimization, interior-point/ barrier methods for inequality constrained optimization. SQP methods.

Section outline

Select section General

Collapse Expand
General

Collapse all Expand all
- Select activity Announcements
  
  Announcements Forum
- Select activity Reading List
  
  Reading List URL
- Select activity Discussion Forum
  
  Discussion Forum
Select section Course Materials

Collapse Expand
Course Materials
- Select activity Lectures 1 and 2
  
  Introduction. Optimality conditions for unconstrained problems. (2021 Videos 1-3; Sheet 1)
  
  Lectures 1 and 2
  
  C62_Lecture1a_2024.pdf
  
  C62_Lecture1b_2024.pdf
  
  C62_Lecture2_2024.pdf
- Select activity Lectures 3 and 4
  
  Methods for unconstrained optimization. Linesearch algorithms (2021 Videos 4-6; Sheet 2)
  
  Lectures 3 and 4
  
  C62_Lecture3a_2024.pdf
  
  C62_Lecture3b_2024.pdf
  
  C62_Lecture4_2024.pdf
- Select activity Lecture 5
  
  Steepest descent methods (2021 Video 7, Sheet 2)
  
  Lecture 5
  
  C62_Lecture5_2024.pdf
- Select activity Lecture 6
  
  Newton's method for unconstrained optimization (2021 Videos 8-9; Sheet 2)
  
  Lecture 6
  
  C62_Lecture6a_2024.pdf
  
  C62_Lecture6b_2024.pdf
- Select activity Lecture 7
  
  Quasi-Newton methods. Nonlinear least-squares and Gauss-Newton methods. (2021 Videos 10-11; Sheet 3)
  
  Lecture 7
  
  C62_Lecture7_2024.pdf
- Select activity Lectures 8 and 9
  
  Trust region methods. (2021 Videos 12-14, Sheet 3)
  
  Lectures 8 and 9
  
  C62_Lecture8.pdf
  
  C62_Lecture9a.pdf
  
  C62_Lecture9b.pdf
- Select activity Lectures 10 and 11
  
  Optimality conditions for constrained problems. (2021 Videos 15-16, Sheet 4)
  
  Lectures 10 and 11
  
  C62_Lecture10and11.pdf
- Select activity Lecture 12
  
  Penalty methods for constrained optimization (2021 Videos 17, Sheet 4)
  
  Lecture 12
  
  C62_Lecture12.pdf
- Select activity Lecture 13
  
  Augmented Lagrangian methods (2021 Videos 18, Sheet 4)
  
  Lecture 13
  
  C62_Lecture13.pdf
- Select activity Lectures 14 and 15
  
  Interior point methods for inequality constrained optimization problems (2021 Videos 19-20, Sheet 4)
  
  Lectures 14 and 15
  
  C62_Lecture14and15.pdf
- Select activity Lecture 16
  
  SQP methods for constrained optimization (2021 Video 21) The content here is nonexaminable.
  
  Lecture 16
  
  C62_Lecture16.pdf
- Select activity Additional lecture material - nonexaminable
  
  Complete proof of trust region methods convergence. Proof of second-order optimality conditions for constrained problems.
  
  Additional lecture material - nonexaminable
  
  Lecture9_CompleteProof_Trust_Region.pdf
  
  Lecture11_ProofTheorem19.pdf
- Select activity Additional Resources
  
  Mathematical Background. Resources and bibliography
  
  Additional Resources
  
  C62_MathematicalBackground.pdf
  
  C62_Resources.pdf
  
  Taylor.pdf
- Select activity Optimization Booklet
  
  Optimization Booklet
  
  Gould_Continuous_Optimization_Algorithms.pdf
  
  Hauser_Cont_Opt_lecturenotes.pdf
- Select activity Sheet 1
  
  Sheet 1 Assignment
  
  Opened: Friday, 17 January 2025, 11:20 PM
- Select activity Sheet 2
  
  Sheet 2 Assignment
  
  Opened: Friday, 17 January 2025, 11:23 PM
- Select activity Sheet 3
  
  Sheet 3 Assignment
  
  Opened: Saturday, 18 January 2025, 9:04 AM
- Select activity Sheet 4
  
  Sheet 4 Assignment
  
  Opened: Saturday, 18 January 2025, 9:07 AM
- Select activity Numerical Laboratories
  
  Numerical Laboratories
  
  C62_NumericalAssignment_1.pdf
  
  C62_NumericalAssignment_2.pdf
  
  Newton method.m
  
  Rosenbrock function.m
  
  Scaled quadratic function.m
  
  Steepest descent method.m
Select section Intercollegiate Classes

Collapse Expand
Intercollegiate Classes
- Select activity Class Signup
  
  Class Signup Class registration
  
  Registration start: Monday, 13 January 2025, 12:00 PM
  
  Registration end: Friday, 14 February 2025, 12:00 PM
- Select activity Class Tutor's Comments
  
  Class Tutor's Comments Assignment
  
  Class tutors will use this activity to provide overall feedback to students at the end of the course.
  
  Not available unless: You have signed up to a class

Main content blocks

Section outline

General

Course Materials

Intercollegiate Classes