Stochastic Optimization

RAWL 3058 | 8:30 a.m. - 4:00 p.m.

 

Description

This workshop will cover basic theory, methods, and algorithms for solving stochastic optimization problems as they appear within machine learning and "big-data" contexts such as classification, regression, function approximation, clustering, and general parameter estimation. This is in addition to traditional optimization contexts in computational finance, logistics, queuing, telecommunications, and epidemic modeling, where the objective function and constraints can only observed through a Monte Carlo simulation or through numerical quadrature.

The workshop is organized intro three modules. The first part will provide a brief survey of methods to solve (deterministic) convex and non-convex optimization problems. The second module of the workshop will introduce stochastic gradient descent (SGD) methods (or, for the purist, stochastic approximation methods) and the numerous modern variants that focus on variance reduction, incorporation of second-order information, and dynamic sampling from a large dataset of observations. The second module will also introduce sample-average approximation (SAA) and retrospective approximation (RA) methods to solve general stochastic optimization methods. The third module of the workshop will focus on the theoretical aspects, providing a panoramic view of the complexity and convergence rate results that are currently available for SGD and its variants, and for SAA/RA.

The organization of the workshop into three modules is designed to attract (i) graduate students, (ii) advanced undergraduate students, (iii) generally sophisticated researchers who are uninitiated to stochastic optimization, and (iv) "practitioners". Participants identifying with categories (i) and (ii) might find all three modules useful; those falling in category (iii) will likely find the third module useful; and those in category (iv) will find the first two modules useful. An earnest attempt will be made to keep the workshop self-contained so that anyone with knowledge at the level of college calculus should be able to follow much of proceedings of the workshop.

Location TBA.