SACA: A Scenario-Aware Collision Avoidance Framework for Autonomous Vehicles Integrating LLMs

Shiyue Zhao
Tsinghua University & University of Michigan, Ann Arbor
Junzhi Zhang
Tsinghua University
Neda Masoud
University of Michigan, Ann Arbor
Heye Huang
University of Wisconsin-Madison
Xingpeng Xia
University of Michigan, Ann Arbor
Chengkun He
Tsinghua University

Abstract

Reliable collision avoidance under extreme situations remains a critical challenge for autonomous vehicles. While large language models (LLMs) offer promising reasoning capabilities, their application in safety-critical evasive maneuvers is limited by latency and robustness issues. Even so, LLMs stand out for their ability to weigh emotional, legal, and ethical factors, to support socially responsible and context-aware collision avoidance decision-making. This paper proposes a scenario-aware collision avoidance (SACA) framework for extreme situations that integrates predictive scenario evaluation, data-driven reasoning, and scenario-preview-based deployment to improve collision avoidance decision-making. SACA consists of three key components. First, a predictive scenario analysis module utilizes obstacle reachability analysis and motion intention prediction to construct a comprehensive situational prompt. Second, an online reasoning module refines decision-making by leveraging prior collision avoidance knowledge and fine-tuning with scenario data. Third, an offline evaluation module assesses performance and stores scenarios in a memory bank. Additionally, a precomputed policy method improves deployability by previewing scenarios and retrieving or reasoning policies based on similarity and confidence levels. Real-vehicle tests show that, compared with baseline methods, SACA effectively reduces collision losses in extreme high-risk scenarios and lowers false triggering under complex conditions.

Framework Overview

This study proposes SACA, an LLM-driven collision avoidance policy arbitration and decision-making framework (Fig. 2), built on real-vehicle scenarios and a pre-developed policy library [1, 21–23]. Aiming to minimize collision loss, SACA integrates legal, ethical, and emotional factors to select optimal strategies through scenario analysis, LLM reasoning, and real-time deployment. The chosen policy is executed by the control module, with outcomes evaluated offline and stored for continual self-learning.

Fig. 2. Overview of the LLM-driven collision avoidance framework.

Table I lists candidate collision avoidance policies. Policies 3 and 4 involve coordinated braking and steering without inducing high-slip conditions, enabling lateral movement with quick directional recovery. SACA outputs target velocities for these policies. Policies 5 and 6 involve T-type drift maneuvers, deliberately inducing lateral sliding to rapidly reorient and decelerate the vehicle. These are beneficial in lateral collision scenarios (T-type collision), redirecting impacts to energy-absorbing zones to protect occupants and the battery pack.

Within the LLM-driven decision-making module (Fig. 2c), vector-based scene matching is conducted to compare the current scenario with stored cases in the memory bank, allowing the retrieval of historically effective collision avoidance results to serve as incremental prompts for decision-making. Additionally, expert-annotated avoidance examples and key safety constraints are leveraged to fine-tune the LLM, enhancing its adaptability to collision scenarios.

To address real-time constraints during deployment, a precomputed strategy approach is designed, enabling the previewing of scenarios and retrieving or reasoning policies based on similarity and confidence levels, thereby improving the real-time performance of execution.

Table I. Candidate Collision Avoidance Policies.

Test Results

Real-vehicle experimental validation of the T-type drift collision avoidance maneuver in the intersection scenario demonstrates SACA's ability to mitigate collision impact effectively while ensuring occupant and pedestrian safety.

SACA uniquely integrates ethical and legal factors with collision avoidance expertise, achieving a 70.8%–93.4% reduction in collision loss and an 81.5%–100% reduction in false-trigger loss compared to baselines in tested scenarios, while maintaining real-time performance under optimal API response conditions.

Below, two distinct scenarios are illustrated: (a) a real-vehicle test in an intersection environment, and (b) an extreme collision avoidance reasoning case on a one-way road.

(a) Real-vehicle intersection scenario test. A T-type drift collision avoidance maneuver is executed.

One-way Road Collision Avoidance Reasoning

(b) Single-lane extreme collision avoidance with LLM-based scenario reasoning.

Citation

@article{zhao2024saca,
    title={SACA: A Scenario-Aware Collision Avoidance Framework for Autonomous Vehicles Integrating LLMs-Driven Reasoning},
    author={Shiyue Zhao and Junzhi Zhang and Neda Masoud and Heye Huang and Xingpeng Xia and Chengkun He},
    journal={ArXiv},
    year={2025}
}

Acknowledgements

We thank our colleagues and institutions for their support in data collection, real-vehicle testing, and valuable feedback throughout this research.

SACA: A Scenario-Aware Collision Avoidance Framework for Autonomous Vehicles Integrating LLMs-Driven Reasoning

Paper