In the subjects of science and technology, engineering and mathematics – known collectively as STEM subjects – up to 40 percent of students drop out of their studies in the entry phase. A research team from the University of Tübingen’s Methods Center at the Faculty of Economics and Social Sciences has now developed a statistical method that can be used to predict on average eight weeks in advance whether students will terminate their studies.
The new algorithm also represents a general methodological step forward. While making the prediction, the algorithm is able to consider the differences between individual students that already exist at the beginning of the study – such as general cognitive abilities – and to separate these from time-dependent affective states of individual students. In this way, it becomes possible to predict the probability of dropout even for students who are suitable for their course. This separation of different levels of influence could also be useful for many questions from other fields. The researchers have published their study in the journal Psychometrika.
Students in STEM subjects have different preconditions at the outset, which influence the general probability of dropping out. "It is obvious, for example, that mathematics performance in high school and general cognitive ability vary among individual students. Lower performance is initially more likely to lead to dropout in the entry phase," says Professor Augustin Kelava of the Methods Center. "However, we wanted to approach the question of how, among comparably able new students, to identify those who drop out quickly."
Longitudinal study with 122 students
In an initial survey for the study, 122 students at the University of Tübingen in their first semester of mathematics were asked about their prior knowledge of mathematics, their interests, their school career and their financial background; and details of personality variables, including emotional stability, were collected. "The results of the initial assessment gave us a picture of each student's stable characteristics," Kelava says. This was followed by five-minute surveys three times a week, for a total of 50 times over 131 days of the semester, in which students indicated how they were currently feeling and whether they felt they were keeping up in class. “Because we checked back with the students, we were able to verify our results. We knew who had stayed until the end of the semester, as well as the grade of the final exam. We also found that our survey met with a high level of acceptance,” he says.
The research team did not specifically intervene in individual study trajectories. “That is a potential application for the future development of this process," he says. The predictions were calculated using the newly developed statistical method, an algorithm that uses data collected up to a point in time to determine a student’s future behavior and experience with high probability. It’s called a forward-filtering-backward-sampling (FFBS) algorithm. "The levels of influence are complex. They interact, and a multitude of variables play a role in the decision to persist or drop out,” says Kelava.
Early prediction of intent to drop out
As a result, the research team was able to predict dropout intentions on average as early as eight weeks beforehand, at a time when people are still coming to classes. "Often, after starting in the winter semester, students are no longer there after the Christmas break," Kelava says. “In predicting hidden intentions, we've been able to separate the two levels of influence – on the one hand, the students’ stable characteristics, from the affective state changes over time on the other hand. Based on their own disclosures of how they feel and how they're doing, we can tell when they develop a latent, not yet directly observable, intention to drop out."
This statistical method provides an instrument enabling specific approaches to individual students who are in principle qualified for the subject but who are showing tendencies to drop out, Kelava adds. They could be offered coaching or counselling. In general, he says the method is also suitable for certain research questions in other areas, such as the separation of stable influencing variables from situational developments, for example in stock prices or in engineering applications.
Publication
Kelava, A., Kilian, P., Glaesser, J., Merk, S., & Brandt, H. (online first). Forecasting intraindividual changes of affective states taking into account interindividual differences using intensive longitudinal data from a university student drop out study in math. Psychometrika, https://doi.org/10.1007/s11336-022-09858-6
