Cluster of Excellence – Machine Learning for Science

Network Projects

With the format of network projects, we want to specifically target research priorities in which the development and application of machine learning methods to specific problems from different scientific disciplines is advanced. The goal of this format is to deepen the dialog between the different scientific disciplines, that join together in a kind of Mini Graduate School. Our currently 5 funded network projects consist of 4 subprojects (SP) each, in which different research groups work together on the following, overarching topics.

Compositionality in Minds and Machines
Machine Learning in Education
Modeling and Understanding Spatiotemporal Environmental Interactions (MUSTEIN)
 Probabilistic Inference in Mechanistic Models (PIMMS)
Uncovering the inner structure of medical images through generative modeling
► Compositionality in Minds and Machines

Compositionality makes all interesting computation possible. By combining and recombining smaller parts into a meaningful whole, compositionality boosts generalization, reduces sample complexity, and improves interpretability. From convolutional networks to structured probabilistic inference, many frameworks can be modelled as compositional functions for pattern recognition, reasoning, and planning. Indeed, inferring good models from limited training sets — and all training sets are limited — fundamentally requires compositionality.

In this Network Project, across 4 Subprojects (SP) we bridge different disciplines to create models of compositional reasoning in minds and machines: In SP 1, we work towards a model that can perform common-sense compositional zero-shot generalization in open world scenarios. In SP 2, we build models that can reason compositionally to explore their environment and solve abstract intelligence tasks in a human-like fashion. In SP 3, we use the compositional nature of human gaze to improve fine-grained classification. In SP 4, we study which inductive biases allow humans to learn and compositionally reuse latent structures.

Graphic showing illustrations of all subprojects that are related to Graph Neural Networks: Compositional Zero-Shot Learning and Generalization (SP1), Compositional Reasoning in Combinatorial Spaces (SP2), Compositional Representations of Human Gaze (SP3) and Inductive Biases in Compositional Sequence Learning (SP4)
Briding different scientific disciplines in 4 subprojects, all using graph neural networks to build symbolic representations of a problem space over which broad generalizations can be performed.

SP 1
Compositional Zero-Shot Learning, Disentangling Representations and Generalization

  •  Principal Investigators:      Zeynep Akata1 · zeynep.akata@uni-tuebingen.de · eml-unitue.de
                                                    Eric Schulz2 · eric.schulz@tuebingen.mpg.de · cpilab.org
                                                    Matthias Bethge3,4 · matthias@bethgelab.org, bethgelab.org
     
  •  Team member:                    Shyamgopal Karthik1 (PhD student)

                                                                    University of Tübingen, 1Cluster of Excellence Machine Learning, 3Department of Physics, 4Tübingen AI Center
                                                                                       2Max Planck Institute for Biological Cybernetics, Tübingen

 Project duration: October 2021 - February 2025

 

Illustrations comparing the steps of Zero-Shot Learning to Compositional Zero Shot-Learning

SP 2
Compositional reasoning in combinatorial spaces

  •  Principal Investigators:   Eric Schulz1 · eric.schulz@tuebingen.mpg.de · cpilab.org
                                                 Zeynep Akata2 · zeynep.akata@uni-tuebingen.de · eml-unitue.de
                                                 Peter Dayan1 · dayan@tue.mpg.de · webpage
     
  •  Team member:                Tankred Saanum1 (PhD student)

                                              1Max Planck Institut for Biological Cybernetics, Tübingen, 2University of Tübingen, Cluster of Excellence Machine Learning
 

 Project Duration: March 2021 - February 2025

Illustrations showing examples of compositional reasoning in combinatorial spaces in Little Alchemy and Raven's progressive matrices

SP 3
Compositional Representations of Human Gaze

  •  Principal Investigators:  Enkelejda Kasneci1 · enkelejda.kasneci@uni-tuebingen.de · www.hci.uni-tuebingen.de
                                                Zeynep Akata2 · zeynep.akata@uni-tuebingen.de · eml-unitue.de
                                                Felix Wichmann1 · felix.wichmann@uni-tuebingen.de · wichmannlab.org 
     
  •  Team members:              Michael Kirchhoff1 (PhD student)

                                                 University of Tübingen, 1Department of Computer Science, 2Cluster of Excellence Machine Learning
     

 Project duration: July 2021 - December 2024

 

SP 4
Inductive Biases in Compositional Sequence Learning

  •  Principal Investigators:  Charley Wu1 · charley.wu@uni-tuebingen.de · hmc-lab.com
                                               Martin Butz2 · martin.butz@uni-tuebingen.de · cm.inf.uni-tuebingen.de
                                               Eric Schulz3 · eric.schulz@tuebingen.mpg.de · cpilab.org
     
  •  Team Member:              Lee Sharkey1 (PhD student)
                                                  Turan Orjulu (PhD student)

                                              University of Tübingen, 1Cluster of Excellence Machine Learning, 2Department of Computer Science
                                                                               3Max Planck Institut for Biological Cybernetics, Tübingen
 

 Project duration: November 2021 - November 2025

 

Illustrations showing Sequence Tasks, Sequence Structures, Compositional Structure and TGRNN Model in Compositional Sequence Learning
► Machine Learning in Education

Education is designed to foster learning in complex, real-life environments. Learners, however, differ substantially in many respects, including social, cognitive, and motivational factors, and their prior knowledge in the target domain.

Education needs to take this substantial heterogeneity into account. With the increasing use of Intelligent Tutoring Systems (ITS) in real-life education, using machine learning to optimally support individual learning is becoming feasible. Yet, we still lack the ability to
(1) interpret rich learning process data digitally collected in real-life education contexts in a way that supports an understanding of learning outcomes based on the characteristics of individual learners and learning activities. On this basis, optimally supporting individual learners then requires,
(2) taking the structure of the knowledge domain into account, and
(3) adaptively sequencing activities to best match individual skills and challenges. While ML methods are promising for addressing these gaps, we need to consider
(4) how determining learning opportunities based on ML models can be designed to be fair, and how to handle rich learner and learning-process data in a privacy-respecting way.

In this Network Project, we address these four intertwined research gaps, where interdisciplinary collaboration and the advancement of machine learning methods is crucial, within 4 Subprojects (SP1-SP4 below).

SP 1
Advancing Learning Analytics: Understanding learning outcomes

  •  Principal Investigators: Detmar Meurers1 · dm@sfs.uni-tuebingen.de · website
                                               Bob Williamson2 · bob.williamson@uni-tuebingen.de · website
                                               Kou Murayama3 · k.murayama@uni-tuebingen.de · website
     
  •  Team member:              NN1 (PhD student)

                                                               University of Tübingen, 1Seminar for Linguistic Sciences, 2Department of Computer Science
                                                              3Hector Research Institute of Education Sciences and Psychology

SP 2
Modeling learning in structured domains

  •  Principal Investigators: Álvaro Tejero-Cantero1 · alvaro.tejero@uni-tuebingen.de · mlcolab.org
                                              Charley Wu1 · charley.wu@uni-tuebingen.de · hmc-lab.com
     
  •  Co-Advisors:                   Detmar Meurers, Kou Murayama, and Ulf Brefeld
     
  •  Team member:              Hanqi Zhou1 (PhD student)

                                                              1University of Tübingen, Cluster of Excellence Machine Learning, 2Department of Computer Linguistics,
                                                              3Hector Research Institute of Education Sciences and Psychology, 4Leuphana University,
                                                               Institute for Information Systems
 

 Project duration: April 2022 - March 2025

 

Illustration on Modeling Learning in Structured Domains with Spaced Repetition and Structured Spaced Repetition

SP 3
Adaptivity and personalized learning in the flow

  •  Principal Investigators: Enkelejda Kasneci1 · enkelejda.kasneci@uni-tuebingen.de · www.hci.uni-tuebingen.de
                                               Detmar Meurers2 · dm@sfs.uni-tuebingen.de · https://uni-tuebingen.de/en/134140
     
  •  Team member:               Efe Bozkir1 (PhD student)

                                                               University of Tübingen, 1Department for Computer Science, 2Seminar for Linguistic Sciences
 

 Project duration: July 2021 - June 2024

SP 4
Ethics, Privacy, and Fairness in digital education environments

  •  Principal Investigators: Konstantin Genin1 · konstantin.genin@uni-tuebingen.de · kgenin.github.io
                                               Thomas Grote1 · thomas.grote@uni-tuebingen.de · website
                                               Benjamin Nagengast2 · benjamin.nagengast@uni-tuebingen.de · website
                                               Bob Williamson3 · bob.williamson@uni-tuebingen.de · website
  •  
  •  Team member:                Vlasta Sikimic1 (PhD student)

                                                                 University of Tübingen, 1Cluster of Excellence Machine Learning, 2Hector Research Institute of Education  
                                                                 Sciences and Psychology, 3Department of Computer Science
 

 Project duration: March 2022 - June 2023

► Modeling and Understanding Spatiotemporal Environmental Interactions (MUSTEIN)

Our world changes rapidly due to mankind creating the “Anthropocene”. Increasing greenhouse gas concentrations (including CO2 and methane) in the atmosphere lead to climate change. Intensive irrigation, agriculture generally, grubbing, dam building, and impervious surfaces, severely affect our hydrosphere – including our freshwater resources – as well as our biogeosphere – including unprecedented rates of soil erosion. Meanwhile, humans demand more energy in their anthroposphere, while relying on safe and rewarding habitats as well as on sufficient food and water resources.

The Network Project MUSTEIN aims at developing machine learning techniques that reliably learn explainable models of critical aspects of these four highly interacting spheres, focusing in 4 Subprojects (SP) on seasonal weather dynamics (SP1), river water discharge (SP2), soil erosion (SP3), and solar thermal systems (SP4).

SP 1
Seasonal weather forecast

  •  Principal Investigators: Bedartha Goswami1 · bedartha.goswami@uni-tuebingen.de · website
                                              Martin Butz2 · martin.butz@uni-tuebingen.de · https://cm.inf.uni-tuebingen.de/
                                              Hendrik Lensch2 · hendrik.lensch@uni-tuebingen.de · website
                                              Nicole Ludwig3 · nicole.ludwig@uni-tuebingen.de · https://nicoleludwig.github.io/
     
  •  Team members:            Jannik Thuemmel1(PhD student)

                                                               University of Tübingen, 1Cluster of Excellence Machine Learning, 2Department of Computer Science, 3ML in Sustainable
                                                                                    Energy Systems
 

 Project duration: September 2021 - August 2025

SP 2
Water discharge prediction

  •  Principal Investigators: Martin Butz1 · martin.butz@uni-tuebingen.de · website
                                              Christiane Zarfl2 · christiane@zarfl@uni-tuebingen.de · website
     
  •  Team members:            Fedor Scholz1(PhD student)

                                                              University of Tübingen, 1Department of Computer Science, 2Department of Geosciences
 

 Project duration: September 2021 - August 2024

SP 3
Modeling soil erosion

  •  Principal Investigators: Thomas Scholten1 · thomas.scholten@uni-tuebingen.de · website
                                               Martin Butz2 · martin.butz@uni-tuebingen.de · https://uni-tuebingen.de/de/25369
                                               Georg Martius3 · georg.martius@tuebingen.mpg.de · http://al.is.mpg.de
     
  •  Team members:             Manuel Traub1 (PhD student)

                                                                University of Tübingen, 1Department of Geosciences, 2Department of Computer Science
                                                                 3Max Planck Institute for Intelligent Systems, Tübingen
 

 Project duration: April 2021 - March 2025

SP 4
Solar thermal systems

  •  Principal investigators: Volker Franz1 · volker.franz@uni-tuebingen.de · website
                                               Carl-Johann Simon-Gabriel2 · cjsg@ethz.ch · website
                                               Georg Martius3 · georg.martius@tuebingen.mpg.de · http://al.is.mpg.de/
                                               Nicole Ludwig4 · nicole.ludwig@uni-tuebingen.de · https://nicoleludwig.github.io
     
  •  Team members:             Florian Ebmeier1 (PhD student)

                                                              University of Tübingen, 1Department of Computer Science, 4Cluster of Excellence Machine Learning
                                                                                 2Learning & Adaptive Systems, ETH Zürich, 3Max Planck Institute for Intelligent Systems, Tübingen
 

 Project duration: October 2021 - September 2025

► Probabilistic Inference in Mechanistic Models (PIMMS)

Numerical models are at the heart of scientific understanding in many disciplines. In geoscience, numerical models are used to describe geophysical processes; in neuroscience, they provide a testbed of our mechanistic understanding of physiological processes; and in systems cell biology they are used to describe regulatory networks of genes or proteins. Typically such mechanistic models come as systems of differential equations, be it ordinary differential equations (ODEs), simulating a spatial process through time, or partial differential equations (PDEs), simulating a process through both time and space.

In 4 Subprojects (SPs, see below), we want to apply currently available probabilistic numerical tools to difficult problems in these three scientific disciplines where ODE/PDE models are paramount. Through these diverse scientific application fields, our primary objective is the development of a general toolbox for probabilistic inference techniques for mechanistic models that will be applicable across disciplines for decades to come.

SP 1
Inferring landscape development processes

  •  Principal Investigators: Todd Ehlers1 · todd.ehlers@uni-tuebingen.de · esdynamics.net
                                               Philipp Hennig2 · philipp.henig@uni-tuebingen.de · mml.inf.uni-tuebingen.de
     
  •  Team members:             Jonathan Schmidt2 (PhD student)

                                                                University of Tübingen, 1Department of Geosciences, 2Department of Computer Science
 

 Project duration: January 2022 - December 2025

SP 2
Unravelling the ice dynamic, atmospheric and oceanographic history of ice-sheets with Bayesian inference

  •  Principal Investigators:  Reinhard Drews1 · reinhard.drews@uni-tuebingen.de · website
                                                Jakob Macke2 · jakob.macke@uni-tuebingen.de · uni-tuebingen.de/en/196970
     
  •  Team member:               Guy Moss2 (PhD student)

                                                                University of Tübingen, 1Department of Geosciences, 2Cluster of Excellence Machine Learning
 

 Project duration: October 2021 - September 2025

SP 3
Efficient inference and simulation for mechanistic models in neuroscience

  •  Principal Investigators: Philipp Berens1 · philipp.berens@uni-tuebingen.de · berenslab.org
                                               Jakob Macke2 · jakob.macke@uni-tuebingen.de · website
                                               Philipp Hennig3 · philipp.hennig@uni-tuebingen.de · website
     
  •  Team member:               Jonas Beck1 (PhD student)

                                                                1Institute for Ophthalmic Research, Universitätsklinikum Tübingen
                                                                University of Tübingen, 2Cluster of Excellence Machine Learning, 3Department of Computer Science
 

 Project duration: January 2022 - December 2024

SP 4
Inference of predictive genome wide mechanistic models of cellular differentiation from single-cell transcriptomics with gradient information

  •  Principal Investigators: Manfred Claassen1 · manfred.claassen@uni-tuebingen.de · website
                                               Jakob Macke2 · jakob.macke@uni-tuebingen.de · website
     
  •  Team member:              Sebastian Bischoff1 (PhD student)

                                                               1Internal Medicine I, Universitätsklinikum Tübingen, 2Cluster of Excellence Machine Learning, University of Tübingen
 

 Project duration: October 2021 - September 2025

► Uncovering the inner structure of medical images through generative modeling

Automated analysis of medical image data has undergone impressive developments over the past decade mainly due to the application of deep learning methods for image classification, segmentation and image-based regression. Despite these advances, machine learning (ML) methods are still rarely used in clinical practice due to underperformance in real-life settings.

In this Network Project we want to explore how state-of-the art ML methodology can be applied to medical image data in order to (1) better understand the structure and content of medical images and thus the underlying physiological and pathological processes, (2) assess robustness and safety of ML models for medical image analysis, and (3) increase robustness of ML models for medical image analysis.

Within 4 Subprojects (SP) we aim at developing probabilistic generative models as well as incorporating determiniscially known factors such as the image generation process where such information is available. In this manner, we will obtain useful data representations for medical image analysis and simplify the modelling process through prior knowledge of deterministic mechanisms. From a medical point of view, we will target different medical image data covering 2D, 3D and 3D+t scenarios including normal and pathological cases.

Graphic showing the generation process of medical image data to model disease development and to address robustness of image analysis algorithms: causal representation learning (P1), adversarial robustness (P2), time series modeling (P3) and modeling fine image details (P4)
In this network project we aim to investigate the generating process of medical image data in order to model disease development and to address robustness of image analysis algorithms (P = subproject).

SP 1
Domain Generalisation for Medical Image Analysis via Causal Representation Learning

  •  Principal Investigators:  Bernhard Schölkopf1· bernhard.schoelkopf@tuebingen.mpg.de · Website
                                                Sergios Gatidis1,2· sergios.gatidis@med.uni-tuebingen.de · Website
     
  •  Team member:                Sergios Gatidis1

                                                 1Max Planck Institute for Intelligent Systems, Tübingen, 2Department for Radiology, Universitätsklinikum Tübingen
 

 Project duration: January 2022 - September 2023

Illustration showing how Medical Imagining Analysis via Causal Representation Learning is done

SP 2
Assessing the robustness of medical prediction algorithms

  •  Principal investigators:  Christian Baumgartner1· christian.baumgartner@uni-tuebingen.de · Website
                                               Matthias Hein2· matthias.hein@uni-tuebingen.de · Website
     
  •  Team member:               Nikolas Morshuis1 (PhD student)

                                                University of Tübingen, 1Cluster of Excellence Machine Learning, 2Department of Computer Science
 

 Project duration: October 2021 - September 2025

Illustration showing how the medical prediction algorithm works

SP 3
Leveraging anatomical and temporal priors for prediction andunderstanding of disease in high-dimensional medical imaging data

  •  Principal investigators:  Jakob Macke1· jakob.macke@uni-tuebingen.de · Website
                                               Christian Baumgartner1· christian.baumgartner@uni-tuebingen.de · Website
     
  •  Team member:               Jaivardhan Kapoor1 (PhD student)

                                                1University of Tübingen, Cluster of Excellence Machine Learning

 Project duration: September 2021 - August 2025

Illustration of Medical Imaging, showing cognitive decline

SP 4
Understanding the structure of fundus images in ophthalmologyusing hybrid generative models including the physical imaging process

  •  Principal investigators: Philipp Berens1· philipp.berens@uni-tuebingen.de · berenslab.org
                                               Hendrik Lensch2· hendrik.lensch@uni-tuebingen.de · Website
     
  •  Team member:               Sarah Müller1 (PhD student)

                                                1University Hospital Tübingen, Institute for Ophthalmic Research, 2University of Tübingen, Department of Computer Science
 

 Project duration: April 2021 - March 2025

Illustrations showing how medical imaging is used in ophthalmology