Lecture: Deep Learning

Within the last decade, deep neural networks have emerged as an indispensable tool in many areas of artificial intelligence including computer vision, computer graphics, natural language processing, speech recognition and robotics. This course will introduce the practical and theoretical principles of deep neural networks. Amongst other topics, we will cover computation graphs, activation functions, loss functions, training, regularization and data augmentation as well as various basic and state-of-the-art deep neural network architectures including convolutional networks and graph neural networks. The course will also address deep generative models such as auto-encoders, variational auto-encoders and generative adversarial networks. In addition, applications from various fields will be presented throughout the course. The tutorials will deepen the understanding of deep neural networks by implementing and applying them in Python and PyTorch.

Students gain an understanding of the practical and theoretical concepts of deep neural networks including, optimization, inference, architectures and applications. After this course, students should be able to develop and train deep neural network architectures for various tasks and understand the potentials and pitfalls when applying deep neural networks in practice.

• Credits: 6 ECTS (2h lecture + 2h exercise)
• Course number: ML-4103 (also credited for INFO-4182 and INF-4182)
• Lectures and exercises will be held asynchronously through YouTube (see sidebar for link). We will provide all lectures and exercise introductions several days before the Q&A sessions for self-study. You must watch these videos before participating in the live Q&A sessions.
• Each Wednesday, we will host a live lecture and exercise Q&A session from 12:00 to 14:00 via Zoom (see sidebar for link) where questions regarding the lecture and exercises are answered. Make sure that you have the latest Zoom client installed.
• All exercise assignments will be handed out and graded through ILIAS (see sidebar for link).
• Students shall watch both the lecture and exercise videos before the Q&A session and take note of questions that they like to have answered during the Q&A sessions.

• Basic computer science skills: Variables, functions, loops, classes, algorithms
• Basic Python coding skills: https://docs.python.org/3/tutorial/
• Basic math skills: Linear algebra, probability and information theory
  As a refresher we recommend reading Chapters 1-4 of: http://www.deeplearningbook.org

• To participate in this lecture or to enroll for our exam, you must enroll via ILIAS (see sidebar for link) until October 28, 2020. Note that the registration deadline has passed and we are not accepting new students do the course. Please do not email us asking about this.
• Information about exam registration and make-up exams can be found here.

The exercises play an essential role in understanding the content of the course. There will be 6 assignments in total (see content table below) which can be conducted in groups of up to 2 students (groups can be formed in the first weeks via the booking pool in ILIAS). The assignments contain pen and paper questions as well as programming problems. In the first half of the course, the students will develop and use the Educational Deep Learning Framework (EDF), a small Python only deep learning framework. This will allow us to understand every aspect of deep learning (computation graphs, backpropagation, optimization, regularization) in detail on small problems (MNIST). In the second half of the course, the students will use PyTorch, a state-of-the-art deep learning framework which features GPU support and auto-differentiation, to address more challenging problems. If you have questions regarding the exercises or the lecture, please ask them during the zoom Q&A sessions or in our ILIAS forum.

The students will collectively write Latex lecture notes to complement the slides, summarizing the content discussed in the lecture videos. In the beginning of the course, every registered student will be assigned one lecture. The lecture notes must be submitted via ILIAS at the latest 7 days after the respective official lecture date (see content table below). Lecture notes must be written individually (not in groups). We will continuously merge and consolidate the lecture notes into a single document. You can edit the lecture notes in Overleaf or a local Latex editor. To get started, copy the Deep Learning Lecture Notes Latex Template.

To qualify for the final exam, students must have:

• registered to the lecture on ILIAS by 28.10.2020
• successfully solved 50% of the exercises - and -
• submitted lecture notes for one lecture
  (slots will be assigned in the first week, notification via email)

To obtain a 0.3 bonus in the final exam, students must:

• successfully solve 75% of the exercises

All students must participate in the main exam that will take place during the official examination period. Only students that failed the main exam and students that were ill during the main exam (doctoral certificate required) are allowed to enroll for a make-up exam.