28.11.2019

Grades before and after school reforms: Like comparing apples and oranges

Upper-school reforms may make it harder to compare grades, say University of Tübingen researchers

High school reforms in the various German states may make it difficult to compare students’ grades from before the reforms with post-reform grades. That is the finding of a study conducted by researchers from the University of Tübingen, St. Gallen University of Teacher Education, and the Leibniz Institute for Research and Information in Education in Frankfurt. They compared the mathematics and English grades of upper-school students with their performance as determined by standardized tests - before and after upper school reforms in Baden-Württemberg and Thuringia. The results show that if the performance-related composition of courses changes in the course of a reform, for example as a result of the introduction or abolition of compulsory core subjects, pupils receive different grades from those obtained in the courses before the reform, even though they have achieved the same results. The results of the study were published in the Journal of Educational Psychology.

In this school year, upper-school students can again choose between basic and advanced courses in most German states. If only two advanced courses have to be taken, these are taught at least five hours per week; if there is a choice of more than two advanced courses, these must be taught for at least four hours each. With this, the states are implementing a resolution of the Standing Conference of the Ministers of Education and Cultural Affairs, aimed at making final-year grades more comparable across Germany. 

For the study, the researchers analyzed the upper-school grades and performance in standardized tests in mathematics and English of a total of around 7,800 pupils – from both before and after the upper-school reforms in Baden-Württemberg (2002) and Thuringia (2010). Prior to the reforms, pupils were able to choose between basic and advanced courses, which were taught for between two to four, five, or six hours a week. After the reforms, German, mathematics and a foreign language had to be taken as core subjects of four hours per week each in both states.  

As expected, students with better grades also scored better in the performance tests. However, it turned out that if a subject was taken as a basic course prior to the upper school reform, the students achieved fewer points in the performance test than those who took the same subject as a core subject after the reform, even though they had received the same grade in everyday school life. Similarly, pupils who had taken a subject as an advanced course before the reform were better in the performance test than those who took the subject as a core subject after the reform, although both received the same marks. The performance differences with the same grade were found in Baden-Württemberg for mathematics and English and could also be proven for mathematics and partly for English in Thuringia.

In summary, the results indicate that reforms that change the performance-related composition of students in upper-school courses may lead to changes in the marks awarded by teachers. This is due in particular to the fact that teachers often compare performance within a class or course. If these are rather high, as in an advanced course, the average pupil will find it harder to get a good grade than in a weaker course. Therefore, it is not only difficult to compare grades between states with different upper-school systems, but also grades within a state, especially before and after upper-school reforms.

This difficulty of comparison has consequences for the selection function of upper-school grades, especially when allocating university places or when applying for a traineeship or a job. This is because individual subject grades, for example in mathematics, are often used as a central criterion for assessing performance potential. “As the study shows, applicants with an identical grade in mathematics do not necessarily have the same knowledge of mathematics. The fact that structural changes in the school system can also systematically influence this relationship has so far been largely ignored,” says Dr. Nicolas Hübner, first author of the study. Those who hire school leavers should be aware of this problem. “Because reforms can change the meaning of grades, there should be routine analysis of these kinds of effects in the future. Furthermore, there should be a discussion about greater standardization of the grading process,” Hübner says. 

Publication:

Hübner, N., Wagner, W., Hochweber, J., Neumann, M., & Nagengast, B. (2019). Comparing apples and oranges: Curricular intensification reforms can change the meaning of students' grades! Journal of Educational Psychology. doi: 10.1037/edu0000351

Contact:

Dr. Nicolas Hübner
University of Tübingen, Hector Institute of Education Science 
+49 7071 29 74385 
nicolas.huebnerspam prevention@uni-tuebingen.de 

Contact for press:

Eberhard Karls Universität Tübingen
Public Relations Department
Dr. Karl Guido Rijkhoek
Director

Antje Karbe
Press Officer
Phone +49 7071 29-76789
Fax +49 7071 29-5566
antje.karbespam prevention@uni-tuebingen.de

www.uni-tuebingen.de/en/university/news-and-publications

Downloads