Project Page • arXiv • Video • BibTeX
Our novel perceptive Forward Dynamics Model (FDM) enables real-time, learned traversability assessment for safe robot navigation by predicting future states based on environmental geometry and proprioceptive history. Trained in simulation and fine-tuned with real-world data, the model captures the full system dynamics beyond rigid body simulation. Integrated into a zero-shot Model Predictive Path Integral (MPPI) planner, our approach removes the need for tedious cost function tuning, improving safety and generalization. Tested on the ANYmal legged robot, our method significantly boosts navigation success in rough environments, with effective sim-to-real transfer.
⭐ If you find our perceptive FDM useful, star it on GitHub to get notified of new releases! The repository features:
- Implementation of the perceptive FDM training code as extension for IsaacLab
- Integration of the perceptive FDM into a Model Predictive Path Integral (MPPI) planner
- Real-world deployment of the perceptive FDM on the ANYmal robot
A technical introduction to the theory behind our perceptive FDM is provided in our open-access RSS paper, available here. For a quick overview, watch the accompanying 5-minute presentation coming soon. More information about the work is available in the abstract below.
Abstract
Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot's capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable. To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot's future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers, and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation by on average 41% over competitive baselines, which translates into a 27% higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data.
@inproceedings{roth2025fdm,
title={Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation},
author={Roth, Pascal and Frey, Jonas and Cadena, Cesar and Hutter, Marco},
booktitle={Robotics: Science and Systems (RSS 2025)},
year={2025}
}
We are currently working on the code release, more infos will follow soon!