# FormulaZero: Distributionally Robust Online Adaptation via Offline Population Synthesis

@inproceedings{Sinha2020FormulaZeroDR, title={FormulaZero: Distributionally Robust Online Adaptation via Offline Population Synthesis}, author={Aman Sinha and Matthew O'Kelly and Hongrui Zheng and Rahul Mangharam and John C. Duchi and Russ Tedrake}, booktitle={ICML}, year={2020} }

Balancing performance and safety is crucial to deploying autonomous vehicles in multi-agent environments. In particular, autonomous racing is a domain that penalizes safe but conservative policies, highlighting the need for robust, adaptive strategies. Current approaches either make simplifying assumptions about other agents or lack robust mechanisms for online adaptation. This work makes algorithmic contributions to both challenges. First, to generate a realistic, diverse set of opponents, we… Expand

#### Figures, Tables, and Topics from this paper

#### 4 Citations

Generating Adversarial Disturbances for Controller Verification

- Computer Science, Engineering
- L4DC
- 2021

An online learning approach that adaptively generates disturbances based on control inputs chosen by the controller is proposed that competes with the best disturbance generator in hindsight and outperforms several baseline approaches, including $H_{\infty}$ disturbance generation and gradient-based methods. Expand

RAT iLQR: A Risk Auto-Tuning Controller to Optimally Account for Stochastic Model Mismatch

- Computer Science
- IEEE Robotics and Automation Letters
- 2021

A novel nonlinear MPC is proposed for distributionally robust control, which plans locally optimal feedback policies against a worst-case distribution within a given KL divergence bound from a Gaussian distribution and provides an algorithm to dynamically adjust the risk-sensitivity level online for risk-sensitive control. Expand

Distributionally Robust Learning

- Computer Science, Mathematics
- Found. Trends Optim.
- 2020

This monograph develops a comprehensive statistical learning framework that is robust to (distributional) perturbations in the data using Distributionally Robust Optimization (DRO) under the… Expand

F1TENTH: An Open-source Evaluation Environment for Continuous Control and Reinforcement Learning

- Computer Science
- NeurIPS
- 2019

The F1TENTH autonomous racing platform is detailed, an open-source evaluation framework for training, testing, and evaluating autonomous systems that enables safe and rapid experimentation of AV algorithms even in laboratory research settings. Expand

#### References

SHOWING 1-10 OF 115 REFERENCES

Variational Inference with Normalizing Flows

- Computer Science, Mathematics
- ICML
- 2015

It is demonstrated that the theoretical advantages of having posteriors that better match the true posterior, combined with the scalability of amortized variational approaches, provides a clear improvement in performance and applicability of variational inference. Expand

Improved Variational Inference with Inverse Autoregressive Flow

- Mathematics, Computer Science
- NIPS 2016
- 2017

A new type of normalizing flow, inverse autoregressive flow (IAF), is proposed that, in contrast to earlier published flows, scales well to high-dimensional latent spaces and significantly improves upon diagonal Gaussian approximate posteriors. Expand

Thermodynamical Approach to the Traveling Salesman Problem : An Efficient Simulation Algorithm

- 2004

We present a Monte Carlo algorithm to find approximate solutions of the traveling salesman problem. The algorithm generates randomly the permutations of the stations of the traveling salesman trip,… Expand

The Nonstochastic Multiarmed Bandit Problem

- Mathematics, Computer Science
- SIAM J. Comput.
- 2002

A solution to the bandit problem in which an adversary, rather than a well-behaved stochastic process, has complete control over the payoffs. Expand

Hit-and-run mixes fast

- Mathematics, Computer Science
- Math. Program.
- 1999

It is shown that the “hit-and-run” algorithm for sampling from a convex body K mixes in time O*(n2R2/r2), where R and r are the radii of the inscribed and circumscribed balls of K and the bound is best possible in terms of R,r and n. Expand

Hit-and-Run Algorithms for Generating Multivariate Distributions

- Mathematics, Computer Science
- Math. Oper. Res.
- 1993

We introduce a general class of Hit-and-Run algorithms for generating essentially arbitrary absolutely continuous distributions on Rd. They include the Hypersphere Directions algorithm and the… Expand

Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions

- Mathematics, Computer Science
- Oper. Res.
- 1984

This work considers the Monte Carlo problem of generating points uniformly distributed within an arbitrary bounded measurable region and considers the class of Markovian methods considered, which are potentially superior to conventional rejection techniques for large dimensional regions. Expand

A Noncooperative Game Approach to Autonomous Racing

- Computer Science, Mathematics
- IEEE Transactions on Control Systems Technology
- 2020

The simulation study shows that the presented games can successfully model different racing behaviors and generate interesting racing situations and two methods for guaranteeing feasibility of the resulting coupled repeated games are studied. Expand

Adversarial Policies: Attacking Deep Reinforcement Learning

- Computer Science, Mathematics
- ICLR
- 2020

The existence of adversarial policies in zero-sum games between simulated humanoid robots with proprioceptive observations, against state-of-the-art victims trained via self-play to be robust to opponents is demonstrated. Expand

AlphaStar: an evolutionary computation perspective

- Computer Science
- GECCO
- 2019

This paper analyzes AlphaStar primarily through the lens of EC, presenting a new look at the system and relating it to many concepts in the field, and hopes to provide a bridge between the wider EC community and one of the most significant AI systems developed in recent times. Expand