- Lecturer: Justin Sirignano
Course Term: Hilary
Course Lecture Information: 16 lectures
Course Overview:
Deep learning has revolutionized image recognition, speech recognition, and natural language processing. There is also growing interest in applying deep learning to finance.
At a high level, deep neural networks are stacks of nonlinear operations, typically with millions of parameters. This produces a highly flexible and powerful model which has proved effective in many applications. The design of network architectures and optimization methods has been the focus of intense research.
This course provides an introduction to deep learning, covering topics such as fully-connected networks, convolution neural networks, residual networks, recurrent neural networks such as LSTMs, generative adversarial networks, and deep reinforcement learning. Optimization methods and distributed training algorithms will also be presented. Students will gain experience in using PyTorch to train deep learning models with GPUs.
Mathematical analysis of neural networks, reinforcement learning, and stochastic gradient descent algorithms will also be covered in the lectures.
At a high level, deep neural networks are stacks of nonlinear operations, typically with millions of parameters. This produces a highly flexible and powerful model which has proved effective in many applications. The design of network architectures and optimization methods has been the focus of intense research.
This course provides an introduction to deep learning, covering topics such as fully-connected networks, convolution neural networks, residual networks, recurrent neural networks such as LSTMs, generative adversarial networks, and deep reinforcement learning. Optimization methods and distributed training algorithms will also be presented. Students will gain experience in using PyTorch to train deep learning models with GPUs.
Mathematical analysis of neural networks, reinforcement learning, and stochastic gradient descent algorithms will also be covered in the lectures.
Course Synopsis:
+ Fully-connected networks, convolution networks, residual networks, and recurrent networks
+ Backpropagation algorithm and stochastic gradient descent
+ Hyperparameter selection and parameter initialization
+ Optimization methods in deep learning
+ PyTorch, automatic differentiation
+ Regularization methods (L2 penalty, dropout, ensembles, data augmentation techniques)
+ Batch normalization, layer normalization
+ Distributed training of models
+ Convergence analysis of stochastic gradient descent and reinforcement learning algorithms
+ Global convergence of neural networks trained with gradient descent
+ Backpropagation algorithm and stochastic gradient descent
+ Hyperparameter selection and parameter initialization
+ Optimization methods in deep learning
+ PyTorch, automatic differentiation
+ Regularization methods (L2 penalty, dropout, ensembles, data augmentation techniques)
+ Batch normalization, layer normalization
+ Distributed training of models
+ Convergence analysis of stochastic gradient descent and reinforcement learning algorithms
+ Global convergence of neural networks trained with gradient descent