Mentor: Yuzhi Lai

Email: yuzhi.laispam prevention@uni-tuebingen.de

Effective Human-Robot Interaction (HRI) is crucial for enhancing accessibility and usability in real-world robotics applications. However, existing solutions often rely on gestures or language commands, making interaction inefficient and ambiguous, particularly for users with physical impairments. Gaze, as a natural interaction modality, has great potential in HRI for individuals with severe physical limitations. By integrating gaze with natural language understanding, robot arm can infer the user's intention, reducing ambiguity and enhancing interaction efficiency. This multi-modal approach is particularly beneficial for individuals with severe motor impairments, as it enables hands-free, natural communication without requiring complex gestures or precise verbal articulation.

The student should begin by reading previous related work, including our own publication and the official Aria Research Kit tutorials, to become familiar with the system architecture and capabilities of the glasses. They should then study literature on multimodal human-robot interaction, robotic grasp detection, and large language model prompt engineering. The core tasks include improving the alignment between language and gaze for more accurate intent recognition and integrating a state-of-the-art grasp detection algorithm, such as AnyGrasp, into the robotic system to enable generalizable object manipulation. A critical part of the project is replacing the prior cloud-based LLM module with a local LLaMA model  for privacy-preserving inference. The student is expected to implement at least one existing algorithm for grasp point prediction and propose two prompt for gaze-language fusion and robot action generation. The project will conclude with real world experiments to evaluate system performance and usability.

Mentor: Yuzhi Lai

Email: yuzhi.laispam prevention@uni-tuebingen.de

Effective Human-Robot Interaction (HRI) is crucial for enhancing accessibility and usability in real-world robotics applications. However, existing solutions often rely on gestures or language commands, making interaction inefficient and ambiguous. Gaze, as a natural interaction modality, has great potential in HRI. By integrating gaze and natural language processing (NLP), we can create a multi-modal interaction framework that enables hands-free, efficient, and intuitive HRI.

This project builds on our existing gaze-language fusion framework to develop an efficient and lightweight multi-modal interaction system for robot manipulation. Unlike previous methods that rely on large-scale language models (LLMs) for command interpretation, this project will focus on traditional NLP approaches that offer faster response times, lower memory consumption, and better suitability for edge-device deployment.

To evaluate the effectiveness of this approach, we will conduct a detailed comparison between our traditional NLP-based system and an LLM-based system across key performance metrics such as processing speed, accuracy in interpreting commands, and overall interaction efficiency. This analysis will help determine whether lightweight NLP models can match or exceed the performance of large language models while significantly improving real-time responsiveness and computational efficiency.

Mentor: Rafia Rahim

Email: rafia.rahimspam prevention@uni-tuebingen.de

Implement a leaderboard-style web platform that lists stereo matching methods sorted by runtime and accuracy. Each entry would present detailed runtime statistics, inference speed (FPS), and accuracy metrics on a uniform hardware environment. Users can select two or more methods for side-by-side comparisons, revealing detailed runtime breakdowns through expandable panels, graphs, and visualizations. Include intuitive search/filter functionality to narrow down methods based on criteria (runtime range, accuracy thresholds, GPU/CPU usage).
 

Requirements: good programming skills, deep learning knowledge.

Mentor: Rafia Rahim

Email: rafia.rahimspam prevention@uni-tuebingen.de

This thesis evaluates modern stereo depth estimation models (e.g., MonSter, StereoAnyWhere, FoundationStereo, DEFOM Stereo) that leverage monocular or foundation model priors. It presents a comparative analysis of their zero-shot performance and robustness, alongside an investigation into the accuracy-efficiency trade-offs introduced by model quantization.
 

Requirements: good programming skills, deep learning knowledge.