By combining the mathematical rigor of modern psychophysical theory with current perspectives in cognitive psychology, the present project investigates the uniqueness and the format of the representation(s) underlying crossmodal matching. Several cognitive theories suggest that different modalities draw on a common representation of intensity. This assumption can be put to a strict empirical test by extending the theory of global psychophysics suggested by R. Duncan Luce to crossmodal judgments. The project intends to develop the requested extension of this measurement-theoretic framework, and to conduct experimental tests of its basic axioms. Moreover, it aims at identifying the format of the cognitive representation underlying crossmodal matching, which is often assumed to be of an amodal or spatial type.

We will investigate the representations used by humans to plan and execute motor actions. Many theories assume amodal representations for planning and modal representations for execution. However, the evidence for such a transition from amodal to modal representations is not clear-cut. In four work packages, we will investigate various actions (from button presses to grasping). The first two work packags extend established experimental paradigms to address ambiguities in the literature. The third work package investigates influences of secondary tasks on early and late action components (dual task paradigm). Work Package 4 investigates whether the representations used for action and perception are based on a common scale. Together, the work packages will provide a better understanding of the human perception-action loop and the role of modal/amodal representations.

Evaluative conditioning (EC) differs from typical classical conditioning (CC) paradigms with respect to various factors that may foster abstraction processes during learning, leading to more amodal as compared to modal representations of parings between a conditioned and an unconditioned stimulus. Such paradigmatic differences potentially explain why EC, but not CC, is insensitive to extinction. The present proposal identifies factors that influence abstraction processes in EC to assess their effects on indicators of modal vs. amodal representations. In a second step, we will then investigate the robustness of modal vs. amodal representations. Finally, we will examine the hypothesis that language exposure constitutes a life-long EC procedure that influences learning and judgment in experiments employing verbal material.

A crucial aspect of everyday behavior is our ability to maintain goal-oriented behavior and thereby switching between controlled and automated processing rather effortlessly. A key role in explaining how our cognitive system adapts behavior has been attributed to the experience of conflict, which triggers short-term information processing adjustments. The central question addressed here is whether such short-term adjustments involve modal (i.e., stimulus- and response-specific) representations or act on amodal representations. First, using cross-modal task contexts (stimulus, response), we investigate whether cognitive control operates on amodal representations, modal representations, or both. If amodal representations underlie conflict adjustments, this should result in domain-general cross-task transfer effects. In contrast, if conflict adjustments are the results of domain-specific changes in information processing, these should result in stimulus and response modality-specific adjustments. Second, we investigate the role of conflict-related control adjustments during language processing, specifically whether adjustments following linguistic conflict change the way subsequent linguistic information is represented in the cognitive system.

This project will investigate the interplay of modal and amodal encodings underlying space-metric associations (SMAs) that, for example, elicit the SNARC (Spatial-Numerical Associations of Response Codes) effect. Some accounts of SMAs postulate amodal representations (like the serial order working memory account), while other accounts focus on situated modal modulations. By employing virtual reality setups, we will dissociate different modal influences to understand their contributions and interactions. Moreover, we will examine amodal accounts by contrasting serial order-induced with magnitude-induced SMAs. As a result, we will scrutinize task-dependent interactions between modal and amodal representations underlying the SNARC effect and other SMA effects. Based on these findings, we will develop a computational theory that will model modal and amodal encodings and their interactions.

The present project investigates the relationship between the psychological distance to a target object and the level of amodality of the representations involved. In particular, the project will test the hypothesis that representations of proximal entities (entities close in space and time) are more modal and representations of distal entities are more amodal. This hypothesis of a distance-dependent representation will be examined in three work packages, whereby each package employs a different task, namely language comprehension, object recognition, and spatial navigation. In each case, we will manipulate the spatial distance (within or beyond grasping space, within or beyond pedestrian walking distance) as well as the temporal distance within or beyond some window of immediateness.

Humans have a remarkable ability to process ever-changing, dynamic information. Project A7 investigates how the human mind constructs mental representations of dynamic events from sensory and perceptual input and how this input is abstracted to amodal representations in long-term memory. It also examines the relationship between confidence in understanding events and accurate predictions at event boundaries, shedding light on whether and how the event structure modulates modal and amodal information and influences cognitive processes. In addition, the project investigates how prior knowledge and exposure to various events impact the ability to detect and remember them, providing insights into the role of amodal event schemata in sensory, modal event processing.

The present project investigates representations along the modal-to-amodal continuum from an ontogenetic perspective. The developing cognitive system might initially have only modal representations available with amodal representations emerging via a process of abstraction, or it might have access to rudimentary amodal representations from early on. This project will investigate the evidence for the two trajectories within three domains of cognition – sensorimotor-language interactions, magnitude estimation, and executive control processes. Therewith, this proposal will improve the understanding on how representational formats develop.

Uncontrolled eating is assumed to be determined by both an elevated activation and an insufficient inhibition of food-related stimuli. To date it is unknown which cognitive mechanisms and representations underlie these two behavioral tendencies. In the present proposal we investigate the hypothesis that eating behavior is driven by low-level, modal associations between sensory features of food stimuli and approach tendencies. In particular, we will examine whether elevated activation and insufficient inhibition of food-related stimuli in overweight and obese individuals is particularly strong when stimuli are presented in a pictorial format that presumably gives rise to more modal as compared to more amodal representations. Knowing more about the type of food-related representations underlying uncontrolled eating will also allow the development of effective intervention programs.