B6.2 Optimisation for Data Science (2023-24)
Main content blocks
- Lecturer: Profile: Coralia Cartis
- Lecturer: Profile: Raphael Hauser
Steepest descent method and its convergence analysis in the general case, the convex case and the strongly convex case.
Modelling: least squares, matrix completion, sparse inverse covariance estimation, sparse principal components, sparse plus low rank matrix decomposition, support vector machines, logistic regression, deep learning.
Proximal operators and prox-gradient methods.
Accelerating gradient methods: heavy ball method and Nesterov acceleration.
Oracle complexity and the stochastic gradient descent algorithm.
The variance reduced stochastic gradient descent algorithm.
Dimensionality reduction techniques for large scale optimisation (Johnson-Lindenstrauss Lemma)
Data sketching: linear least squares and sums of functions (batch stochastic gradient)
Parameter sketching: Randomised coordinate descent first order methods and random subspace methods.
Section outline
-
-
Old Lecture notes with Latex source code. Disclaimer: these notes provide an overview of all the material in one place, but they may contain typos that we are fixing in the lecture slides, as the course progresses. At the end of the course we will post updated lecture notes with fixed typos, if required. The lecture slides contain all the lecture materials in their most up-to-date form.
-
-
-
https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Dd0e45590-75c1-48e1-bfbf-ae1601012ce1%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D5ee2e5f3-7769-4f4e-bf95-ae1601012d3a%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D5fc62a2a-6ed4-4fad-a56f-ae1601017362%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D8e795c1b-c7fe-4bb5-920c-ae1601012d4f%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D197d00d3-8a58-4077-854a-ae1601017411%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D19453d24-a26a-4248-9c5a-ae1601012d66%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Db0f90c1c-f415-4fba-bf35-ae16010173b8%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Dfa691284-c06e-454d-9d0e-ae1601012d92%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Dd7905c89-2197-48b5-85a1-ae1601017321%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3De26ae3a2-b0d4-41b8-ab5f-ae160101738e%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D0b9e6334-80ea-4fbe-b54e-ae16010173f7%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3D712a7ad0-d536-44bc-bba5-ae1601017378%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Da54d9a71-ca36-4a04-9af0-ae1601012daa%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Df215947f-a157-4ee3-9302-ae1601012d7b%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3Dd7a94db6-45bd-482b-93dd-ae16010173db%3C%2Fa%3E%3C%2Fspan%3E%3Cbr%3E%3C%2Fp%3E%0A%3Cp%3E%3Cspan%20style%3D"font-size: 0.9375rem;">https://ox.cloud.panopto.eu/Panopto/Pages/Auth/Login.aspx?instance=moodle-maths&support=true&ReturnUrl=https%3A%2F%2Fox.cloud.panopto.eu%2FPanopto%2FPages%2FViewer.aspx%3Fid%3De6931e9e-615f-4c97-9df3-ae1601012dc0%3C%2Fa%3E%3C%2Fspan%3E%3C%2Fp%3E%3Cbr%3E%3Cp%3E%3C%2Fp%3E%3C%2Fdiv%3E
-