Multi-future environment forecasting and occlusion inference for autonomous driving

Описание: Ph.D. thesis defense of Bernard Lange

Research on self-driving cars is motivated by the need to improve the safety, efficiency, and economic benefits of transporting people and goods. Although today’s autonomous vehicles operate reliably within structured, geo-fenced environments, they struggle to generalize to more complex scenarios that require scene understanding and reasoning about numerous interacting—and often occluded—agents.

This dissertation presents learning-based techniques for representing and predicting the environment surrounding an autonomous vehicle. We use both raw sensor measurements—such as camera images and LiDAR point clouds—and higher-level, structured vectorized representations, generated by upstream perception modules.

Our first contribution introduces a first framework capable of jointly performing trajectory prediction and occlusion inference under partial observability, leading to improved robustness and state-of-the-art performance in occlusion inference. A key limitation of vectorized methods is their reliance on manually labelled data. To overcome this, our second contribution develops a sensor-space approach that predicts LiDAR-based occupancy grid maps while conditioning on complementary modalities, such as RGB cameras, planned trajectories, and maps. This method generates multiple hypotheses of future occupancy sequences in real time, achieving state-of-the-art results with clear improvements due to multimodal conditioning. Our final contribution proposes a unified environment representation that accommodates arbitrary sensor types and configurations. The resulting model generalizes from dash-cam-only setups to complex multi-camera and LiDAR systems, enabling reconstruction of surrounding images and 3-D occupancy maps.

Slides available at: https://web.stanford.edu/group/sisl/p...