Within the last decade, deep neural networks have emerged as an indispensable tool in many areas of artificial intelligence including computer vision, computer graphics, natural language processing, speech recognition and robotics. This course will introduce the practical and theoretical principles of deep neural networks. Amongst other topics, we will cover computation graphs, activation functions, loss functions, training, regularization and data augmentation as well as various basic and state-of-the-art deep neural network architectures including convolutional networks and graph neural networks. The course will also address deep generative models such as auto-encoders, variational auto-encoders and generative adversarial networks. In addition, applications from various fields will be presented throughout the course. The tutorials will deepen the understanding of deep neural networks by implementing and applying them in Python and PyTorch.
From November on, live sessions will be held every Wednesday from 14:15 to 16:00 in lecture hall N10 at Hörsaalzentrum Morgenstelle.
Students gain an understanding of the theoretical and practical concepts of deep neural networks including, optimization, inference, architectures and applications. After this course, students should be able to develop and train deep neural networks, reproduce research results and conduct original research in this area.
The exercises play an essential role in understanding the content of the course. There will be 6 assignments in total. The assignments contain pen and paper questions as well as programming problems. In the first half of the course, the students will use the Educational Deep Learning Framework (EDF), a small Python only deep learning framework. This will allow them to understand every aspect of deep learning (computation graphs, backpropagation, optimization, regularization) in detail on small problems. In the second half of the course, the students will use PyTorch, a state-of-the-art deep learning framework which features GPU support and auto-differentiation, to address more challenging problems. If you have questions regarding the exercises or the lecture, please ask them during the live sessions, at the zoom helpdesk or in our chat.
Date | Lecture Slides and Videos | Live Sessions (EDF | PyTorch) | TA Support |
Recap: Math for Deep Learning | |||
16.10. | L01 - Introduction | Slides 1.1 Organization | Video 1.2 History of Deep Learning | Video 1.3 Machine Learning Basics | Video | L01 - Lecture Organization | Christian Reiser |
23.10. | L02 - Computation Graphs | Slides 2.1 Logistic Regression | Video 2.2 Computation Graphs | Video 2.3 Backpropagation | Video 2.4 Educational Framework | Video | L02 - Lecture Q&A E01 - Exercise Q&A | Christian Reiser |
30.10. | L03 - Deep Neural Networks | Slides 3.1 Backpropagation with Tensors | Video 3.2 The XOR Problem | Video 3.3 Multi-Layer Perceptrons | Video 3.4 Universal Approximation | Video | L03 - Lecture Q&A E01 - Exercise Q&A | Christian Reiser Takeru Miyato |
06.11. | L04 - Deep Neural Networks II | Slides 4.1 Output and Loss Functions | Video 4.2 Activation Functions | Video 4.3 Preprocessing and Initialization | Video | L04 - Lecture Q&A E02 - Exercise Q&A | Takeru Miyato |
13.11. | No Lecture | ||
20.11. | L05 - Regularization | Slides 5.1 Parameter Penalties | Video 5.2 Early Stopping | Video 5.3 Ensemble Methods | Video 5.4 Dropout | Video 5.5 Data Augmentation | Video | L05 - Lecture Q&A E02 - Exercise Q&A | Takeru Miyato Bozidar Antic |
27.11. | L06 - Optimization | Slides 6.2 Optimization Algorithms | Video 6.3 Optimization Strategies | Video 6.4 Debugging Strategies | Video | L06 - Lecture Q&A E03 - Exercise Q&A | Bozidar Antic |
04.12. | L07 - Convolutional Neural Networks | Slides 7.1 Convolution | Video 7.2 Downsampling | Video 7.3 Upsampling | Video 7.4 Architectures | Video 7.5 Visualization | Video | L07 - Lecture Q&A E03 - Exercise Q&A | Bozidar Antic Anpei Chen |
11.12. | L08 - Sequence Models | Slides 8.2 Recurrent Network Applications | Video 8.3 Gated Recurrent Networks | Video 8.4 Autoregressive Models | Video | L08 - Lecture Q&A E04 - Exercise Q&A | Anpei Chen |
18.12. | L09 - Natural Language Processing | Slides 9.1 Language Models | Video 9.2 Traditional Language Models | Video 9.3 Neural Language Models | Video 9.4 Neural Machine Translation | Video | L09 - Lecture Q&A E04 - Exercise Q&A | Anpei Chen Daniel Dauner |
08.01. | L10 - Graph Neural Networks | Slides 10.1 Machine Learning on Graphs | Video 10.2 Graph Convolution Filters | Video 10.3 Graph Convolution Networks | Video | L10 - Lecture Q&A E05 - Exercise Q&A | Daniel Dauner |
15.01. | No Lecture | ||
22.01. | L11 - Autoencoders | Slides 11.1 Latent Variable Models | Video | Video 11.2 Principal Component Analysis | Video 11.3 Autoencoders | Video 11.4 Variational Autoencoders | Video | L11 - Lecture Q&A E05 - Exercise Q&A | Daniel Dauner |
29.01. | L12 - Generative Adversarial Networks | Slides 12.1 Generative Adversarial Networks | Video 12.2 GAN Developments | Video 12.3 Research at AVG | Video | L12 - Lecture Q&A | Daniel Dauner |
Research @ AVG 1. Learning Robust Policies for Self-Driving |
Our website uses cookies. Some of them are mandatory, while others allow us to improve your user experience on our website. The settings you have made can be edited at any time.
or
Essential
in2cookiemodal-selection
Required to save the user selection of the cookie settings.
3 months
be_lastLoginProvider
Required for the TYPO3 backend login to determine the time of the last login.
3 months
be_typo_user
This cookie tells the website whether a visitor is logged into the TYPO3 backend and has the rights to manage it.
Browser session
ROUTEID
These cookies are set to always direct the user to the same server.
Browser session
fe_typo_user
Enables frontend login.
Browser session
Videos
iframeswitch
Used to show all third-party contents.
3 months
yt-player-bandaid-host
Is used to display YouTube videos.
Persistent
yt-player-bandwidth
Is used to determine the optimal video quality based on the visitor's device and network settings.
Persistent
yt-remote-connected-devices
Saves the settings of the user's video player using embedded YouTube video.
Persistent
yt-remote-device-id
Saves the settings of the user's video player using embedded YouTube video.
Persistent
yt-player-headers-readable
Collects data about visitors' interaction with the site's video content - This data is used to make the site's video content more relevant to the visitor.
Persistent
yt-player-volume
Is used to save volume preferences for YouTube videos.
Persistent
yt-player-quality
Is used to save the quality settings for YouTube videos.
Persistent
yt-remote-session-name
Saves the settings of the user's video player using embedded YouTube video.
Browser session
yt-remote-session-app
Saves the settings of the user's video player using embedded YouTube video.
Browser session
yt-remote-fast-check-period
Saves the settings of the user's video player using embedded YouTube video.
Browser session
yt-remote-cast-installed
Saves the user settings when retrieving a YouTube video integrated on other web pages
Browser session
yt-remote-cast-available
Saves user settings when retrieving integrated YouTube videos.
Browser session
ANID
Used for targeting purposes to profile the interests of website visitors in order to display relevant and personalized Google advertising.
2 years
SNID
Google Maps - Google uses these cookies to store user preferences and information when you view pages with Google Maps.
1 month
SSID
Used to store information about how you use the site and what advertisements you saw before visiting this site, and to customize advertising on Google resources by remembering your recent searches, your previous interactions with an advertiser's ads or search results, and your visits to an advertiser's site.
6 months
1P_JAR
This cookie is used to support Google's advertising services.
1 month
SAPISID
Used for targeting purposes to profile the interests of website visitors in order to display relevant and personalized Google advertising.
2 years
APISID
Used for targeting purposes to profile the interests of website visitors in order to display relevant and personalized Google advertising.
6 months
HSID
Includes encrypted entries of your Google account and last login time to protect against attacks and data theft from form entries.
2 years
SID
Used for security purposes to store digitally signed and encrypted records of a user's Google Account ID and last login time, enabling Google to authenticate users, prevent fraudulent use of login credentials, and protect user data from unauthorized parties. This may also be used for targeting purposes to display relevant and personalized advertising content.
6 months
SIDCC
This cookie stores information about user settings and information for Google Maps.
3 months
NID
The NID cookie contains a unique ID that Google uses to store your preferences and other information.
6 months
CONSENT
This cookie tracks how you use a website to show you advertisements that may be of interest to you.
18 years
__Secure-3PAPISID
This cookie is used to support Google's advertising services.
2 years
__Secure-3PSID
This cookie is used to support Google's advertising services.
6 months
__Secure-3PSIDCC
This cookie is used to support Google's advertising services.
6 months