Lecture: Self-Driving Cars

Within the last years, driverless cars have emerged as one of the major workhorses in the field of artificial intelligence. Given the large number of traffic fatalities, the limited mobility of elderly and handicapped people as well as the increasing problem of traffic jams and congestion, self-driving cars promise a solution to one of our socities most important problems: the future of mobility. However, making a car drive on its own in largely unconstrained environments requires a set of algorithmic skills that rival human cognition, thus rendering the task very hard. This course we will cover the most dominant paradigms of self-driving cars: modular pipeline-based approaches as well as deep-learning based end-to-end driving techniques. Topics include camera, lidar and radar-based perception, localization, navigation, path planning, vehicle modeling/control, imitation learning and reinfocement learning. The tutorials will deepen the acquired knowledge through the implementation of several deep learning based approaches to perception and sensori-motor control in the context of autonomous driving. Towards this goal, we will build upon existing simulation environments and established deep learning frameworks.

Students develop an understanding of the capabilities and limitations of state-of-the-art autonomous driving solutions. They gain a basic understanding of the entire system comprising perception, learning and vehicle control. In addition, they are able to implement and train simple models for sensori-motor control.

• Course number: ML-4340
• Credits: 6 ECTS (2h lecture + 2h exercise)
• Total Workload: 180h
• Prerequisites:
  • Basic math and coding skills (in particular Python and PyTorch)
  • Basic knowledge about deep neural nets is beneficial

• All lecture and exercise materials are accessible through ILIAS.
• To participate in this lecture you must enroll through ILIAS at the beginning of the semester.

By continuous and active participation in the weekly exercises, students may obtain a 0.3 bonus on the final grade, when passing the exam. To qualify for this bonus, the student must successfully solve 50% of the assigned homework problems which will be determined by grading the submitted homework solutions.

Homework problems will require coding in Python and PyTorch. Make sure you are familiar with Python. Prior experience with PyTorch or Tensorflow is not required but a plus.

• Current version of the Self-Driving Cars Lecture Notes
• Janai, Güney, Behl and Geiger: Computer Vision for Autonomous Vehicles
• Goodfellow, Bengio and Courville: Deep Learning
• Deisenroth, Faisal and Ong: Mathematics for Machine Learning
• Articles and papers mentioned in the lecture slides (footer)

After the course, Micha Schilling has implemented a conditional imitation learning controller on a Raspberry Pi 3 that was added to the basic Arduino self-driving car kit. Here are links to the project page and video:

• Github: https://github.com/Lucbus/SelfDrivingElegooCar
• YouTube: https://www.youtube.com/watch?v=1-7RTr_nGgs