Georgia Tech SCL Georgia Tech SCL Georgia Tech SCL

Title: Uncertainty quantification for iterative algorithms

Abstract:

This paper investigates the iterates obtained from iterative algorithms in high-dimensional linear regression problems, in the regime where the feature dimension is comparable with the sample size.  The analysis and proposed estimators are applicable to Gradient Descent (GD), proximal GD and their accelerated variants such as Fast Iterative Soft-Thresholding (FISTA).  The paper proposes novel estimators for the generalization error of the iterate for any fixed iteration along the trajectory. These estimators are proved to be root-n consistent under Gaussian designs.  Applications to early-stopping are provided: when the generalization error of the iterates is a U-shape function of the iterations, the estimates allow to select from the data an iteration that achieves the smallest generalization error along the trajectory.  Additionally, we provide a technique for developing debiasing corrections and valid confidence intervals for the components of the true coefficient vector from the iterate at any finite iteration.

Bio:

Pierre C Bellec is an associate professor in the Statistics Department at Rutgers University. He was elected fellow of the Institute of Mathematical Statistics in 2023. In the last few years, his work focused on uncertainty quantifications in regression models, including some recent applications to iterative algorithms and bagging. Before joining Rutgers in 2016, he completed his PhD at ENSAE in Paris, France where he was advised by Alexandre Tsybakov.