Georgia Tech SCL Georgia Tech SCL Georgia Tech SCL

Title: 

Efficient Gradient Estimation for Overparameterized Stochastic Differential Equations

Abstract: 

Overparameterized stochastic differential equation (SDE) models have achieved remarkable success in various complex environments, such as PDE-constrained optimization, stochastic control and reinforcement learning, financial engineering, neural SDEs, and generative AI. These models often feature system evolution coefficients that are parameterized by a high-dimensional vector θ in R^n, aiming to optimize expectations of the SDE, such as a value function, through stochastic gradient ascent. Consequently, designing efficient gradient estimators for which the computational complexity scales well with n is of significant interest. We introduce a novel unbiased stochastic gradient estimator--the generator gradient estimator--for which the computation time remains stable in n. In addition to establishing the validity of our methodology for general SDEs with jumps, we also perform numerical experiments testing our estimator in controlling a multi-class queue where the control policy is parameterized by high-dimensional neural networks. The results show a significant improvement in efficiency compared to previous methods: our estimator achieves near-constant computation times, increasingly outperforms its counterparts as n increases. These empirical findings highlight the potential of our proposed methodology for optimizing SDEs in contemporary applications.

This is a joint work with Jose Blanchet and Peter Glynn.

Bio: 

Shengbo Wang is a fifth-year Ph.D. candidate in Operations Research at Stanford University’s Department of Management Science and Engineering, co-advised by Prof. Peter Glynn and Prof. Jose Blanchet. His research interests span a broad spectrum within applied probability, including stochastic modeling, reinforcement learning, distributionally robust control, and simulation methods for machine learning. He focuses on developing tractable probabilistic models and designing algorithms for data-driven dynamic decision-making under uncertainty, specifically addressing reliability and scalability challenges in modern managerial and engineering applications. Prior to his PhD studies, he earned his B.S. in Operations Research and Information Engineering from Cornell University.