Мини-курс по стохастическим градиентным методам

28 сентября в 107 БК (Физтех.Био) пройдет мини-курс визит-профессора МФТИ Питера Рихтарика, ведущего мирового специалиста по методам оптимизации.

Мини-курс будет состоять из 4 лекций и посвящен стохастическим градиентным методам. Стоит отметить, что этот курс читался на международной конференции ICCOPT 2019 в Берлине, и места для регистрации закончились за пару дней. Мини-курс рассчитан на студентов 3+ курса, интересующихся оптимизацией, машинным обучением и нейронными сетями. Язык курса английский.

Лекции будут проходить с 10:45 до 18:30 с перерывом на обед.

Название курса: A Guided Walk Through the ZOO of Stochastic Gradient Descent Methods

Abstract: Stochastic gradient descent (SGD) in one of its many variants is the workhorse method for training modern supervised machine learning models. However, the world of SGD methods is vast and expanding, which makes it hard to understand its landscape and inhabitants. In this tutorial I will offer a guided walk through the ZOO of SGD methods. I will chart the landscape of this beautiful world, and make it easier to understand its inhabitants and their properties. In particular, I will introduce a unified analysis of a large family of variants of proximal stochastic gradient descent (SGD) which so far have required different intuitions, convergence analyses, have different applications, and which have been developed separately in various communities. This framework includes methods with and without the following tricks, and their combinations: variance reduction, data sampling, coordinate sampling, importance sampling, mini-batching and quantization. As a by-product, the presented framework offers the first unified theory of SGD and randomized coordinate descent (RCD) methods, the first unified theory of variance reduced and non-variance-reduced SGD methods, and the first unified theory of quantized and non-quantized methods.