LLMs in Autonomous Driving — Part 1
In this series of blog posts, as I work on my PhD topic on multi-agent decision-making in self-driving cars, I will review some papers related to the usage of LLMs in the field of Autonomous Driving. Let’s start with some survey papers.
Here are the other parts of this series:
LLMs in Autonomous Driving — Part 2
GPT-Driver: Learning to Drive with GPT GPT-Driver paper proposes a novel approach to motion planning for autonomous vehicles that leverages the power of large language models (LLMs). The approach works by reformulating motion planning as a language modeling problem, where the planner’s inputs and outputs are represented as language tokens. An LLM, in thi…
LLMs in Autonomous Driving — Part 3
In this part, we will review the DriveGPT4 paper. Let’s get started! DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model Self-driving cars have gone from science fiction to rapidly approaching reality. While much of the focus is on making these vehicles safe and reliable, what about making them understandable and interpretable?…
LLMs in Autonomous Driving — Part 4
Let’s continue with two important papers from Google: PaLM-E: An Embodied Multimodal Language Model The core concept behind PaLM-E is to directly integrate real-time observations (like images, state estimates, and sensor data) into the same representational space used by a pre-trained language model (LLM). These observations are encoded as sequences of ve…
A Survey on Multimodal Large Language Models for Autonomous Driving
The paper “A…
Keep reading with a 7-day free trial
Subscribe to Isaac’s Newsletter to keep reading this post and get 7 days of free access to the full post archives.