Uni-Tübingen

Robust Vision Projects

In the second funding period the CRC 1233 "Robust Vision" is organized in twelve research projects, a infrastructure project and a transfer project, which are supported by the central coordination project (Z-Project).

Project 2: Robust material inference

Project description

The goal of this project is to develop models that recover a physical interpretation of visual scenes. In the first phase, this project focused on developing robust inference methods and generative models for controlled capturing of material reflections. In the second phase we want to broaden the scope towards estimating reflection properties (BRDFs) and shape in complex scenes for unknown geometry and illumination, even allowing for a broader range of material classes. Robust and efficient material estimation in these significantly more complex settings poses several interesting challenges for which we aim to develop rather general inference strategies.

Principle Investigators
  • Hendrik Lensch

Project 4: Causal inference strategies in human vision

Project description

Much of the success of artificial vision systems is typically achieved in highly controlled settings, when the data generating dis-tribution does not change between training and testing. Human vision, on the other hand, is robust to out of distribution changes in sensor noise, illumination, view point, occlusion, or different environments. In this project, we explore the hypothe-sis that the visual system combines causal inference with generative modelling strategies in order to produce robust percep-tion. In an extension to the first proposal, we hereby focus on how temporal gestalt principles facilitate an object-based scene understanding in humans and machines.

Principle Investigators
  • Felix Wichmann
  • Bernhard Schölkopf
  • Matthias Bethge

Project 5: Task–dependent top down modulation of visual processing

Project description

One of the central questions of high-level human vision concerns the functional roles of bottom-up/feed-forward and top-down/feedback connections. In the first funding period, we found that the evidence for purely feed-forward processing in hu-man object recognition is much weaker than currently thought. In the second funding period, we will investigate the question of feed-forward vs. feedback processing across a variety of different tasks, thereby investigating how the human visual system adapts to the different task-demands and whether other tasks than the priming paradigm admit top-down influences more directly.

Principle Investigators
  • Volker Franz
  • Ulrike von Luxburg
  • Peter Dayan

Project 6: Probabilistic inference in early visual cortex

Project description

This subproject will investigate whether generative models can provide a normative account of observed top-down influences on neural activity and behaviour in perceptual decision making tasks.  In the upcoming funding period, we will develop com-putational methods that will help address this empirical challenge: Rather than hand-crafting a normative model and comparing it to behavioural and neural data post-hoc, we will develop a conceptual framework – inverse normative modelling (INM) - to infer models from data.

Principle Investigators
  • Jakob Macke
  • Fabian Sinz

Project 7: Large-scale neuronal interactions during natural vision

Project description

This project investigates large-scale neuronal interactions during natural vision. Natural visual stimuli contain spatiotemporal regularities that are shaped by the physical laws of our world. In the upcoming funding period, we will use dynamic visual stimuli, to test the hypothesis that, during natural vision, the brain exploits these regularities and continuously conveys predic-tions of visual inputs through neuronal feedback projections. 

Principle Investigator
  • Markus Siegel
  • Andreas Bartels

Project 9: Natural dynamic scene processing in the human brain

Project description

This project will provide insights on the role of high-level and low-level scene content on the interpretation of behaviourally relevant features such as scene depth and motion velocity in terms of real-world coordinates, and on the robustness of these representations with regards to low-level feature manipulations. The project will also shed light onto saliency in natural dy-namic scenes. To which extent is saliency driven by low- or high-level visual content, which neural representations reflect low-level and high-level components of saliency, and which biological processing stages share similarities with saliency represen-tations of artificial neural networks.

Principle Investigators
  • Andreas Bartels
  • Michael Black

Project 10: Natural stimuli for mice: environment statistics and neural representations in the early visual system

Project description

From this project, we expect to improve our understanding of which visual features of the environment are sampled by the mouse with combined head and eye movements, to yield robust representations of naturalistic visual stimuli. We also expect to gain insight into how the early visual system uses specialised coding strategies for different aspects of the visual environment, such as the upper vs. lower visual fields. Finally, we expect to learn more about the role of “suppressed-by-contrast” (SbC) neurons, a non-standard, frequent, but poorly-understood functional cell type of the early visual system, in encoding of natural scenes.

Principle Investigators
  • Laura Busse
  • Thomas Euler

Project 11: Impacts of eye movements on visual processing: from retina to perception

Project description

Eye movements result in substantial retinal image shifts. This creates continuous spatio-temporal modulations of neural activi-ty, starting from the photoreceptors and all the way to downstream areas. Our ultimate goal is to understand how such contin-uous “movies” at the input stage of the visual system shape perception. In the second funding period, we will further investi-gate the impacts of eye movements on visual processing, focusing on eye movements of different temporal and spatial scales (slow drifts, microsaccades, and larger saccades) that support different behavioural task requirements.

Principle Investigators
  • Ziad Hafed
  • Katrin Franke

Project 12: Image processing within a locally complete retinal ganglion cell population

Project description

The goal of this project continues to be the development of a convolutional neural network (CNN) model that is able to accu-rately describe how image information is nonlinearly transformed in the retinal network and, hence, can predict the responses of the approx. 40 different retinal output channels to new stimuli. A key challenge of such models is that they require large amounts of training data. Therefore, the project will maintain its dual focus to (1) improve experimental aspects related to max-imising data yield, and (2) to develop analysis and modelling frameworks that can discount heterogeneities both between neurons of the same type and between different experimental sessions.

Principle Investigators
  • Thomas Euler
  • Matthias Bethge

Project 13: Visual processing of feedforward and feedback signals in the dLGN

Project description

The objective of this project was (and is) to study the feedforward and feedback influences on visual information processing and robustness in the dorsolateral geniculate nucleus (dLGN) of the thalamus. For the next funding period, we will determine the role of feedforward and feedback processing in dLGN for processing of more complex visual inputs such as natural mov-ies, using DNN-based models of the early visual system. To elucidate how dLGN representations support multiple tasks in changing environments, we will then probe through experiments and modeling how feedforward and feedback computations differentially contribute to neural processing of visual stimuli in dLGN under various illumination conditions. 

Principle Investigators
  • Laura Busse
  • Philipp Berens

Project 14: Retinal Disease Models as a Tool for Understanding Robust Vision

Project description

The healthy human retina supports robust visual processing, leading to stable contrast and brightness perception over a broad range of conditions. Interestingly, once vision has properly developed, it remains remarkably robust even if certain reti-nal cells start to degenerate due to retinal diseases. The objective of this project is to study the robustness and flexibility of visual task performance in human  RP (retinitis pigmentosa) and CSNB1 (complete congenital stationary night blindness) patients and in corresponding animal disease models by linking visual performance to underlying cellular function.

Principle Investigators
  • Katarina Stingl
  • Günther Zeck
  • Christina Schwarz

Project 17: Learning explainable policies for self-driving cars from little data

Project description

The goal of this project is to learn explainable, robust and generalizable policies for self-driving cars end-to-end from data. Existing approaches to learning self-driving policies end-to-end are limited with respect to two fundamental aspects: generali-zation and interpretability. In this project, we plan to tackle both aspects by combining ideas from modular approaches, repre-sentation learning, recurrent attention and zero-shot learning to yield an introspective model that generalizes to novel driving situations and behaviours.

Principal Investigators
  • Zeynep Akata
  • Andreas Geiger

Project INF: A collaborative data management platform for reproducible neuroscience and machine learning

Project description

In modern neuroscience, machine learning and computer vision research, ensuring consistency and integrity of the results by tracking data dependencies poses a serious challenge to the data management and infrastructure. This project will initiate a collaborative data management platform based in the open source DataJoint framework, which suits for rapid prototyping as well as publication level analyses, ensuring data consistency at all levels. The proposed platform will extend the functionalities of DataJoint to flexibly and consistently share data across labs and with the public and develop new tools to increase the re-producibility of research results.

Principal Investigators
  • Fabian Sinz
  • Philipp Berens

Project T01: Physiologically inspired robust electro-optical autofocals

Project description

After the age of 40, every human being experiences presbyopia, caused by the stiffening of the eye lens, resulting in the inca-pability of focusing on near objects. As a consequence, and due to an aging world population, an estimated number of 1.8 billion people suffers from blurred near vision. The transfer project focuses on autofocals, where a reliable and fast estimator of an object of interest and its distance based on physiological information represents the missing puzzle piece to market suc-cess. Building on the findings of the CRC and the expertise of its members, the transfer project will generate a dynamic spec-tacle prototype changing dioptric power properties at the needs of the wearer’s visual system.

Principal Investigators
  • Siegfried Wahl
  • Katharina Rifai