03.02.2025

“Screening Smarter, Not Harder”

GEBF Young Publication Prize 2025 awarded for publication by LEAD members

How can AI be used in the screening of educational research literature? Answers to this question were provided by the following publication, which has now won the GEBF Young Researcher Publication Award 2025 in the category ‘PhDs’.

The summary of existing literature on a topic, a research syntheses, helps scientists to gain an overview of the current state of research on a particular topic. In practice and application, they are also an important tool for policymakers to make evidence-based decisions. Creating research syntheses takes a lot of time and is therefore usually associated with high costs: When systematically reviewing primary studies, known as screening, many summaries of studies, the abstracts, must first be viewed and classified. Although programs based on machine learning (ML) can, in principle,support the classification of abstracts, it has not yet been tested whether ML algorithms are up to this task for studies specifically from educational science and educational psychology. The authors of the study tested this. Their approach evaluates the performance of ML screening algorithms, focussing on the criteria of sensitivity, specificity, and timesavings.

The study shows: The use of ML works and can save researchers a lot of time, but the performance and quality for this specific application depends on the algorithm used and the abstracts being analyzed. The results help researchers to use ML in screening for research syntheses and thus to achieve important research results much faster. This is not only practical but can be particularly relevant in situations such as the coronavirus pandemic, for example when decisions need to be made quickly about measures to close schools. In the future, in such situations we can summarize more quickly: What can science tell us about this topic?

The publication was produced as part of the ‘Artificial Intelligence Assisted Systematic Literature Reviews and Meta-Analyses’ project led by Dr Tim Fütterer, which is funded by the LEAD Graduate School & Research Network. The GEBF Young Researchers Publication Award 2025 in the PhD category will be awarded to Diego Campos, University of Oslo. LEAD members involved in the publication are Tim Fütterer, Thomas Gfrörer, Rosa Lavelle-Hill, Steffen Zitzmann and Kou Murayama.

The GEBF, the Society for Empirical Education Science, is an association of scientists who are active in research andteaching in the field of empirical education sciences. Every year, it awards two prizes for young researchers in the categories Postdocs and doctoral students. The prizes are awarded for publications that present the results of empirical educational research in a particularly convincing way and have appeared in a peer-reviewed journal.

Rebecca Beiter

Publication

Campos, D. G., Fütterer, T., Gfrörer, T., Lavelle-Hill, R., Murayama, K., König, L., Hecht, M., Zitzmann, S., & Scherer, R. (2024): Screening Smarter, Not Harder: A Comparative Analysis of Machine Learning Screening Algorithms and Heuristic Stopping Criteria for Systematic Reviews in Educational Research. Educ Psychol Rev 36, 19. https://doi.org/10.1007/s10648-024-09862-5