Scientific Theory Testing in Randomized Field Studies

Our Goal

LEAD will be developed into a “Center for Randomized Field Studies.” Thereby, interventions are not only tested in practice to generate empirical evidence of their effectiveness but are also explicitly used to contribute to the testing and further development of scientific theories. To achieve this, we do not merely compare outcomes between intervention and control groups. Instead, we utilize a variety of methods (including mixed methods and learning analytics) to investigate the processes that lead to the success (or failure) of interventions.

The Challenge

For many (supposed) innovations in education, there is no evidence of their effectiveness. Moreover, the processes, mechanisms of action, and contextual conditions that determine effectiveness are often insufficiently studied. At the same time, many promising innovations with a scientific foundation do not make their way into classrooms, even when they are derived from psychological and educational theories of learning and teaching.

Our Approaches

The LEAD research area “Scientific Theory Testing in Randomized Field Studies” brings together methodological expertise and best-practice experience required to conduct large-scale randomized field studies and to ensure the robustness of their findings. Researchers in this area, for example, support subject-matter didactics experts in comparing two different fundamental approaches to teaching literature. They also evaluate the added value of professional development for teachers in digitally supported foreign language instruction. In doing so, they not only analyze the effectiveness of an intervention but also examine the role of various enabling conditions (e.g., motivation) in determining intervention success.

In addition to conducting their own randomized field studies, researchers in this area also contribute to methodological advancements and conceptual debates surrounding randomized field studies.

Background: What are randomized field studies?

Randomized field studies are particularly well suited for testing the effectiveness of interventions. Through randomization (i.e., the random assignment), groups are created that are statistically similar and can therefore be compared reliably. If one or more groups receive an intervention while a control group does not, the effect of the intervention(s) can be tested through comparisons between the groups. In this way, strong scientific evidence for the effectiveness of an intervention can be obtained.

Modern scientific approaches (such as learning analytics) make it possible to observe processes that unfold during the course of an intervention. These observations can help identify, for example, drivers of an intervention’s effects or relevant contextual conditions. As a result, randomized field studies not only provide evidence of intervention effects but also help test and further develop theories in real-world settings.

Selected Research Projects

Hector Kinderakademien

The Hector Children’s Academies offer a voluntary, additional program to promote giftedness for elementary school children in Baden-Württemberg. They are scientifically monitored by a team consisting of members of the Hector Research Institute of Education Sciences and Psychology and the Leibniz Institute for Research and Information in Education (DIPF) in Frankfurt.

Among other activities, courses to support gifted primary school children are developed in cooperation with subject-matter didactics experts and systematically examined for their effectiveness and scalability in randomized field studies

SEGEL: Subjectivity and emotionality in conversations about literature in German lessons

Conversations about literary texts are a central component of German lessons. However, there has hardly been any empirical research on this topic so far. The focus of the research project SEGEL therefore is the empirical verification of the widespread assumption in literature didactics that subjectivity and emotionality in conversations about literature are of particular importance for their success.

AI2Teach - Artificial intelligence in foreign language teaching

Digitalization is transforming schools. Digital devices and software licenses are now widely available in most educational settings. However, teachers’ competencies for designing and implementing digital instruction have not always developed at the same pace. While there is evidence that digital tools have the potential to improve learning outcomes, analyses of the effectiveness of complementary teacher professional development are still lacking. AI2Teach aims to provide this analysis and to better understand how and in what ways teacher professional development influences classroom practices and student learning outcomes.

Effectiveness of Eyewitness Interviews in History Education

Eyewitness interviews in history classes are popular because they can make history feel more vivid and engaging for students. However, there is a risk that students may not adequately contextualize or critically interpret the statements made by eyewitnesses. This issue is being investigated in the Eyewitness Testimony Studies conducted at the Hector Research Institute of Education Sciences and Psychology. The studies examine effects on different outcomes (e.g., historical competencies) as well as the underlying processes involved.

KLUG: “Teaching History Inclusively: A Blended Learning Concept for Teacher Professional Development”

The KLUG teacher professional development program was developed in collaboration with experts in history didactics, educational psychology, and special education and has since been continuously optimized, for example with regard to digital learning. A series of studies examines the effectiveness of the program, particularly in terms of enhancing students’ historical competencies.

Contact

Prof. Dr. Benjamin Nagengast

Dr. Florian Berens

Dr. Sven Rieger

Dr. Wolfgang Wagner