Lecture: Self-Driving Cars

Within the last years, driverless cars have emerged as one of the major workhorses in the field of artificial intelligence. Given the large number of traffic fatalities, the limited mobility of elderly and handicapped people as well as the increasing problem of traffic jams and congestion, self-driving cars promise a solution to one of our socities most important problems: the future of mobility. However, making a car drive on its own in largely unconstrained environments requires a set of algorithmic skills that rival human cognition, thus rendering the task very hard. This course we will cover the most dominant paradigms of self-driving cars: modular pipeline-based approaches as well as deep-learning based end-to-end driving techniques. Topics include camera, lidar and radar-based perception, localization, navigation, path planning, vehicle modeling/control, imitation learning and reinfocement learning. The tutorials will deepen the acquired knowledge through the implementation of several deep learning based approaches to perception and sensori-motor control in the context of autonomous driving. Towards this goal, we will build upon existing simulation environments and established deep learning frameworks.

Students develop an understanding of the capabilities and limitations of state-of-the-art autonomous driving solutions. They gain a basic understanding of the entire system comprising perception, learning and vehicle control. In addition, they are able to implement and train simple models for sensori-motor control.

• Basic Computer Science skills: Variables, functions, loops, classes, algorithms
• Basic Python and PyTorch coding skills
• Basic Math skills: Linear algebra, probability and information theory (eg., Math for ML lecture https://www.tml.cs.uni-tuebingen.de/teaching/2020_maths_for_ml/index.php). As a refresher we recommend reading Chapters 1-4 of: http://www.deeplearningbook.org or watching our newly micro tutorials Mathematics for Deep Learning.
• Experience with Deep Learning (eg., through participation in our Deep Learning lecture)

• To participate, you must register via ILIAS and our Quiz Server (see sidebar)
• Information about exam registration can be found here

The exercises play an essential role in understanding the content of the course. There will be 6 assignments in total. The assignments contain pen and paper questions as well as programming problems. Each programming problem involves programming an agent to participate in a challenge against the other agents developed by the students. The winners of each challenge will present their work during the final live session. For programming, the students will use Python and PyTorch, a deep learning framework which features GPU support and auto-differentiation. If you have questions regarding the exercises or the lecture, please ask them during the live sessions, at the zoom helpdesk or in our chat.

• Student's Self-Driving Cars Lecture Notes
• A 30 Minutes Summary of this Lecture on Medium
• Janai, Güney, Behl and Geiger: Computer Vision for Autonomous Vehicles
• Chen et al.: End-to-end Autonomous Driving: Challenges and Frontiers
• Goodfellow, Bengio and Courville: Deep Learning
• Richard Szeliski: Computer Vision: Algorithms and Applications
• Deisenroth, Faisal and Ong: Mathematics for Machine Learning
• Micro Tutorials Mathematics for Deep Learning
• Articles and papers mentioned in the lecture slides