Title: Robust and Uncertainty-Aware Decision Making under Distribution Shifts

Abstract: Decision making tasks like contextual bandit and reinforcement learning often need to be conducted under data distribution shifts. For example, we may need to utilize off-policy data to evaluate a target policy and/or learn an optimal policy utilizing logged data. We may also need to deal with sim2real problem when there is a dynamics shift between training and testing environments. In this talk, I am going to introduce three threads of my work in the domain of robust decision making under distribution shifts. First, I will introduce distributionally robust off-policy evaluation and learning techniques that feature a more conservative uncertainty in the reward estimation component. This pessimistic reward estimation will benefit both off-policy evaluation and learning under various distribution shifts. Second, I will introduce our work in off-dynamics reinforcement learning, where we recognize that the previous methods in off-dynamics reinforcement learning methods can suffer from a lack of exploration and propose a novel model-based approach to it. Finally, I will cover our current work and future work in uncertainty-aware approaches to safe decision-making problems.

Bio: Anqi (Angie) Liu is an assistant professor in the Department of Computer Science at the Whiting School of Engineering, Johns Hopkins University. She is broadly interested in developing principled machine learning algorithms for building more reliable, trustworthy, and human-compatible AI systems in the real world. Her research focuses on enabling the machine learning algorithms to be robust to the changing data and environments, to provide accurate and honest uncertainty estimates, and to consider human preferences and values in AI interactions. She obtained her PhD in computer science from the University of Illinois Chicago. Prior to joining Johns Hopkins, she completed her postdoctoral research in the Department of Computing + Mathematical Sciences at the California Institute of Technology. She is a recipient of the JHU Discovery Award, AI2AI Award, and an Amazon Research Award.

Zoom: https://wse.zoom.us/j/97720056194?pwd=aCNb14fShnXzVWXnOzCDtDKWbi8cNb.1
Meeting ID: 977 2005 6194
Passcode: 159069