Probabilistic Machine Learning and Modern Statistics
Organizer and Chair: Vinayak Rao, Assistant Professor of Statistics, Department of Statistics, Purdue University
Speakers
- Babak Shahbaba, Associate Professor, Departments of Statistics and Computer Science, University of California at Irvine
- Jean Honorio, Assistant Professor of Computer Science, Department of Computer Science, Purdue University
- Qiang Liu, Assistant Professor, Department of Computer Science, University of Texas at Austin
- Bharath Sriperumbudur, Assistant Professor, Department of Statistics, Pennsylvania State University
Friday, June 8, 1:30-3:30 p.m. in STEW 214 CD
| Time | Speaker | Title |
|---|---|---|
| 1:30-2:00 p.m. | Babak Shahbaba |
Dynamic Bayesian Models for Neural Data Analysis |
| Abstract: In this talk, I will start with a brief overview of our neurophysiological experiment, where the objective is to understand the basic neuronal mechanisms underlying sequential memory. I will then present our proposed statistical models for analyzing the complex neural data collected based on this experiment. Implementing these models requires sampling from high dimensional and constrained probability distributions. I will describe a computationally efficient sampling algorithm we have recently developed for such problems. | ||
| 2:00-2:30 p.m. | Jean Honorio | Learning linear structural equation models in polynomial time and sample complexity |
| Abstract: The problem of learning structural equation models (SEMs) from data is a fundamental problem in causal inference. We develop a new algorithm --- which is computationally and statistically efficient and works in the high-dimensional regime --- for learning linear SEMs from purely observational data with arbitrary noise distribution. We consider three aspects of the problem: identifiability, computational efficiency, and statistical efficiency. We show that when data is generated from a linear SEM over p nodes and maximum degree d, our algorithm recovers the directed acyclic graph (DAG) structure of the SEM under an identifiability condition that is more general than those considered in the literature, and without faithfulness assumptions. In the population setting, our algorithm recovers the DAG structure in O(p(d^2 + log p)) operations. In the finite sample setting, if the estimated precision matrix is sparse, our algorithm has a smoothed complexity of O(p^3 + p d^7), while if the estimated precision matrix is dense, our algorithm has a smoothed complexity of O(p^5). For sub-Gaussian noise, we show that our algorithm has a sample complexity of O(d^8 log p), while for noise with bounded 4m-th moment, our algorithm has a sample complexity of O(d^8 p^(2/m)). | ||
| 2:30-3:00 p.m. | Qiang Liu |
A Stein variational framework for deep probabilistic modeling |
| Abstract: Modern AI and machine learning techniques increasingly depend on highly complex, hierarchical (deep) probabilistic models to reason with complex relations, and make decisions under uncertain environment. This, however, casts a significant demand on developing efficient computational methods for highly complex probabilistic models in which exact calculation is prohibitive. We discuss a new framework for approximate learning and inference that combines ideas from Stein's method, an advantaged theoretical technique developed by mathematical statistician Charles Stein, with practical machine learning and statistical computation techniques such as variational inference, Monte Carlo, optimal transport and reproducing kernel Hilbert space (RKHS). Our framework provides a new foundation for probabilistic learning and reasoning and allows us to develop a host of new algorithms for a variety of challenging learning and AI tasks, that are significantly different from, and have critical advantages over, traditional methods. Examples of applications include computationally tractable goodness-of-fit tests for evaluating highly complex models, scalable Bayesian computation, deep generative models, and sample efficient policy gradient for deep reinforcement learning. | ||
| 3:00-3:30 p.m. | Bharath Sriperumbudur | On Approximate Kernel PCA Using Random Features: Computational vs. Statistical Trade-off |
| Abstract: Kernel methods are powerful learning methodologies that provide a simple way to construct nonlinear algorithms from linear ones. Despite their popularity, they suffer from poor scalability in big data scenarios. Various approximation methods, including random feature approximation, have been proposed to alleviate the problem. However, the statistical consistency of most of these approximate kernel methods is not well understood except for kernel ridge regression wherein it has been shown that the random feature approximation is not only computationally efficient but also statistically consistent with a minimax optimal rate of convergence. In this work, we investigate the efficacy of random feature approximation in the context of kernel principal component analysis (KPCA) by studying the statistical behavior of approximate KPCA in terms of the convergence of eigenspaces and the reconstruction error. | ||