ÍâÍøÌìÌÃ

This event is in-person and open only to Princeton University ID holders

This talk is concerned with the sample efficiency of reinforcement learning (RL). Despite a large number of prior works, a complete picture of the trade-offs between sample complexity and statistical accuracy is yet to be determined.ÌýÌýIn particular, prior results in many basic RL settings suffer from an enormous sample size barrier, in the sense that their claimed statistical optimality holds only when the sample size exceeds a large threshold.ÌýThis, however, presents a major roadblock to achieving efficient RL in sample-limited scenarios.Ìý

In this talk, I will discuss how to overcome such sample size barriers in two basic settings. The first setting assumesÌýaccess to an idealistic generative model (or simulator), and we show that a perturbed model-based approach is minimax optimal for this scenario. The second setting is concerned with online RL that requires balancing exploration and exploitation; we show that a model-free algorithm with early-settled variance reduction is provably sample- and memory-efficient. Our results cover two distinctive RL paradigms and might shed light on the efficacy of these algorithms in more complicated scenarios.Ìý

This talk is based on joint work with Gen Li, Laixi Shi, Yuejie Chi, Yuantao Gu, and Yuting Wei (Ìý²¹²Ô»åÌý).Ìý

YuxinÌýChenÌýis currently an assistant professor in the Department of Electrical and Computer Engineering at Princeton University and is affiliated with Applied and Computational ÍâÍøÌìÌÃ, Computer Science, and Center for Statistics and Machine Learning. Prior to joining Princeton, he was a postdoctoral scholar in the Department of Statistics at Stanford University, and he completed his Ph.D. in Electrical Engineering at Stanford University. His research interests include high-dimensional statistics, mathematical optimization, and reinforcement learning. He has receivedÌýthe Princeton graduate mentoring award, Princeton SEAS junior faculty award, the AFOSRÌýYoung Investigator Award,Ìýthe ARO Young Investigator Award, theÌýICCMÌýbest paper award (gold medal), and was selected as a finalist for the Best Paper Prize for Young Researchers in Continuous Optimization.ÌýÌý