Master Theses at the Chair of Cognitive Systems (Prof. Dr. Andreas Zell)

Students who want to take a master thesis should have attended at least one lecture of Prof. Zell and passed it with good or at least satisfactory grades. They might also have obtained the relevant background knowledge for the thesis from other, similar lectures.

Spiking neural network for event-based ball detection

Mentor: Andreas Ziegler

Email: andreas.zieglerspam prevention@uni-tuebingen.de

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of μs), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision.

So far, most learning approaches applied to event data, convert a batch of events into a tensor and then use conventional CNNs as network. While such approaches achieve state-of-the-art performance, they do not make use of the asynchronous nature of the event data. Spiking Neural Networks (SNNs) on the other hand are bio-inspired networks that can process output from event-based directly. SNNs process information conveyed as temporal spikes rather than numeric values. This makes SNNs an ideal counterpart for event-based cameras.

The goal of this thesis is to investigate and evaluate how a SNN can be used together with our event-based cameras to detect and track table tennis balls. The Cognitive Systems groups has a table tennis robot system, where the developed ball tracker can be used and compared to other methods.

Requirements: Familiar with "traditional" Computer Vision, Deep Learning, Python

A Comparison of Robot Arm Motion Planners

Mentor: Mario Laux

Email: mario.lauxspam prevention@uni-tuebingen.de

Description: The aim of this thesis is to review, evaluate and compare different motion planners for robot arms. Suitable metrics have to be developed. The corresponding simulations and real-world experiments have to be analyzed statistically.

Requirements: C++, calculus, statistics, ROS, DNN, MOVEit

Pointcloud Clustering to Segment Unknown Objects

Mentor: Daniel Weber

Email: daniel.weber@uni-tuebingen.de

Description: The recognition and segmentation of objects is often solved by means of neural networks. However, for unknown objects a machine learning approach is not always appropriate. The goal of this thesis is to cluster point clouds in order to segment unknown objects. The point clouds are to be recorded with a Kinect v2 and the code should be implemented in Python or C++ and run on Linux. Different methods (edge based, region based, graph based) can be considered. The developed segmentation should be tested with different objects and environments.

Requirements: Good programming skills

Exploiting Drone Metadata for Multi Object Tracking (MOT)

Mentor: Martin Meßmer

Email: martin.messmerspam prevention@uni-tuebingen.de

Although some deep learning methods like correlation filters and Siamese networks show great promise to tackle the problem of multi object tracking, those approaches are far from working perfectly. Therefore, in specific use cases, it is necessary to impose additional priors or leverage additional data. Luckily, when working with drones, there is free metadata to work with such as height or velocity of the drone. In this thesis, the student should develop some useful ideas on how to exploit this data to increase the performance of a MOT-model and also implement and compare those ideas with other approaches.

Requirements: deep learning knowledge, Python, good English or German

Developing a Meta-Data-Aware Object Detector Running in Real-Time

Mentor: Benjamin Kiefer

Email: benjamin.kiefer@uni-tuebingen.de

Description: UAV imagery differs from other scenarios in that it employs images/videos taken with dynamic altitudes and angles and therefore viewpoints. At the same time, an object detector running in real-time on a UAV requires a lightweight Neural Network. Off-the-shelf object detectors are either too slow or do not consider the environmental factors inherited in Object Detection from UAVs. In this thesis, the student should study and develop a lightweight object detector capable of considering its environment given by, e.g., barometer (altitude) and camera angle measurements that can run in real-time on a small GPU such as the Nvidia Jetson Xavier.

Requirements: Good knowledge in Deep Neural Networks and PyTorch and Tensorflow.

Image fusion with multi-view 3D object detectors

Mentor: Nuri Benbarka

Email: nuri.benbakra@uni-tuebingen.de

Description: For many years, there were mainly two ways to approach 3D object detection in Point clouds; either processing them in the perspective view or the birds-eye view. Each of the approaches has its advantages and disadvantages. Recent work showed that combining features from the two views increases the performance dramatically. An uninvestigated addition to this latest work is to combine image features with the perspective view features of the Point cloud. In this thesis, we will try to implement this idea and see who it affects the performance.

Requirements: Experience with PyTorch.

Self Supervised Learning for Deep Stereo Vision

Mentor: Rafia Rahim

Email: rafia.rahim@uni-tuebingen.de

Description: Self-supervised learning is showing quite a promise for solving various computer vision tasks. The advantage is being able to leverage labelled data for learning a proxy task and then tune the network to perform on the targeted task, here in our case deep stereo vision. The goal here is to explore different self-supervised methods along with geometrical constraints to learn a self-supervised method for deep stereo vision. The advantage of this approach will be that we can make use of a huge quantity of unlabeled data to learn better stereo vision models.

Requirements: Good knowledge of deep neural networks, experience with PyTorch or Tensorflow.

Lidar-based SLAM: Long-Term localization and mapping in outdoor environments

Mentor: Falk Engmann

Email: falk.engmannspam prevention@uni-tuebingen.de

Description: Especially in large outdoor environments, obtaining robust knowledge about the environment is still a challenging task in mobile robotics. Therefore, this work focuses on the review, implementation and evaluation of SLAM approaches in the outdoor sector. Here you have the opportunity to deepen your knowledge in the field of mobile robotics, robust environment perception and long-term SLAM and to implement and evaluate your results on mobile robots in our labs and in real environments.

Requirements: C++, fundamental knowledge of SLAM, ROS

Robust Trajectory Prediction for Autonomous Driving

Mentor: Marcel Hallgarten

Email: marcel.hallgarten@uni-tuebingen.de

Prediction of future behavior of dynamic agents within a scene (given by a lane-graph) is a crucial task for safe (i.e. collision-free) autonomous driving. To this end, current state-of-the-art approaches take the scene context (e.g., lane graph, traffic-light states, position and extent of static objects, etc.) as well as context of agents within the scene (e.g., position, velocity, heading etc. within the last observed timesteps) as an input and predict how the future will unroll by predicting the most likely future trajectories for each agent.

While State-of-the-Art approaches yield impressive results w.r.t displacement errors and off-road rates on test-sets of various large-scale open-source datasets, they have been proven to be vulnerable to realistic adversarial examples These results suggest, that including the agent-history as feature causes the model to perform the prediction by extrapolating the past without taking the lane-graph into account sufficiently

The goal of this thesis is to evaluate different approaches to overcome this. Therefore approaches such as removing the agent history from the features or adding a multitask training objective to enforce a strong correlation of prediction and lane-graph (e.g., self-supervised lane graph completion based on prediction) should be evaluated.

Requirements: Knowledge in Deep Learning, Python (PyTorch)