pt en
Newsletter
glossary

know more

The compulsory shutdown of schools and universities enhances the importance of understanding what is already known about the impact of online classes on student achievement. A very recent study at a Swiss public university reveals that this means of teaching can aggravate the differences between the least and most successful students.

In recent weeks, the forced closure of schools and universities all over the world, to which Portugal was no exception, has created the need to find alternative teaching methods. The different educational institutions have migrated their classes and evaluation moments to online platforms, a medium relatively unused among us.[1] That has required additional effort from schools and teachers in the preparation of materials adapted to this new reality, and there is naturally a lot of uncertainty about its functioning and impact. Thus, any lessons learned during this challenging period are susceptible to biases. Yet, after this adverse moment, is this model desirable for the future? Is the impact the same for all students or does it exacerbate the existing inequalities? In order to answer these questions, we have focused on scientific studies about the impact of online classes in higher education.

In some educational systems, online classes have been progressively introduced.[2] The use of online resources has been presented as an affordable way to facilitate access to education and to give greater flexibility to students in class schedules and content review. However, this means of teaching can also be responsible for lower student participation and commitment, and can lead to a higher dropout rate. Existing studies on the economics of education generally do not mention distance learning programs for primary and secondary education. In higher education, on the other hand, three of the most relevant studies can be highlighted, namely two experiments at US universities and one at a Swiss university. The three investigations compare two similar groups of students, one of whom has face-to-face classes, while the other has distance learning classes. The three analyses conclude that the transition to distance learning has a negative impact on the less successful students and that they should therefore be monitored more closely during this transition.

The impact of online classes is particularly negative for students with a lower average, for boys, and for Hispanic students.

The preliminary study on this topic, conducted by Figlio et al. (2013), took place at an American university during the spring semester of 2007, in the subject of Introduction to Microeconomics, which had previously, in each theoretical class, allowed students to attend the class in person or online. Thus, each classroom was recorded and that recording was made available online to all students throughout the semester. In this case, to assess the impact of attending the class online, it would not be rigorous to directly compare those students who decided to attend the class in person with those who preferred to do so at a distance. That would introduce a bias problem of sample selection. For example, if only the students with the best performance decided to attend the classes in person, at the beginning there would be a difference between the two groups that would make them difficult to compare. In order to overcome this difficulty, the enrolled students were asked if they would be willing to participate in an experience, in which they would be randomly allocated to face-to-face or online classes. Thus, two groups of students exposed to the same content and the same final exam were created – the only difference between them was the adopted teaching means. By observing the results in this same exam, and by controlling a series of other characteristics of the students, it is concluded that the difference between the online classes and the face-to-face classes is not statistically significant. However, it was slightly unfavourable to the online classes. Nevertheless, these results are subject to a high degree of heterogeneity according to certain characteristics of the students: it can be seen that the impact of the online classes is particularly negative for students with a lower average, for boys, and for Hispanic students.

The second, a more recent study by Cacault et al. (2019), focuses on an experience developed during the spring and autumn semesters of 2017 at the University of Geneva. This teaching institution decided that the theoretical classes of the subjects with the greatest number of students enrolled in the Economics and Management courses would be live-streamed at the same time as they were taught. It should be noted that those classes were not recorded, but only streamed simultaneously. In order to isolate the impact of attending classes in person or online, the following methodology was followed: from the total number of students enrolled, 15% were randomly chosen as always having access to the transmission of classes; a different group of 15% would never have access to it, and as such could only attend classes in person; among the remaining 70%, some were provided with the possibility of attending classes online every week and randomly. One of the first results obtained was that, in all the weeks of classes, only 10% of the students who had the opportunity to access classes online actually did so. According to the data collected, the students only used the online system when some exceptional event prevented them from attending the class in person. It is noted that there are no statistically significant effects between the test results of students who had access to the online platform and those who did not. However, these results hide considerable heterogeneity: the students with lower scores, who had access to the platform and used it, saw their results drop by about 20%, while the students with better scores, who had access to the platform and used it, saw their results increase by about 25%.

Students only used the transmission system when some exceptional event prevented them from attending the classroom in person.

Finally, Bettinger et al. (2017) observed the results of more than 230,000 students enrolled in about 168,000 classes and 750 curricular units of different courses, between the spring semester of 2009 and the fall of 2013, at an American university. Each subject always had one face-to-face class, although it could take place in one of the one hundred university centres that the institution had spread throughout the United States. Soon, about two thirds of all students attended the class online. In this case, and unlike the two previous studies, the students were not randomly allocated to one of the two types of classes. However, the fact that the campus where the class was taught was varied throughout the semesters, as well as the distance between the student's residence and the location of the classroom, were two sources of information that allowed for the isolation of the impacts of each type of class. It was anticipated that students who were closer to the location where classes were actually taking place would more often choose face-to-face classes. The impact of online classes was estimated as negative and that effect occurred throughout the entire grade distribution: it reduces the probability of having a very good grade (A) by 12.2 percentage points, of having a good grade (B) by 13.5 percentage points, and of having a medium grade (C) by 10.1 percentage points.[3] Once again, students who had a lower average saw their achievement more negatively affected by attending online classes.

The transition to distance learning now takes place in a very particular context. However, previous studies, especially in higher education, warn us that students with the lowest scores or with a more fragile socioeconomic condition may be the ones who are most harmed at this point. We must also ask ourselves whether these same results are translatable to the reality of primary and secondary education. In this field, one particular fact should alarm us: according to the latest data published by the National Institute of Statistics, in November 2019, about 5.5% of households with children up to the age of 15 claimed not to have access to the Internet at home. This may seem like a residual number, but if we consider that around one million students are enrolled in basic education (between the 1st and 9th grades), we could conclude that a universe of about 50,000 will not have access to online educational resources. In another survey conducted by the Nova SBE Economics of Education Knowledge Center, in March 2020, just at the beginning of the quarantine caused by the coronavirus, 23% of teachers reported that their students did not have access to computers with Internet at home. This figure was higher in the 1st cycle and in public schools. In the present scenario in which a major effort is placed on the feasibility of distance learning, it is therefore necessary to bear in mind that there are students without Internet access and who need alternative solutions for their study.

Notes

1 Referimo-nos à lecionação das aulas exclusivamente online e não a outros recursos que já estão em prática, como, por exemplo, o uso das plataformas Moodle e Escola Virtual.

2 Segundo dados do Departamento de Educação dos Estados Unidos da América, em 2018, 34,7% dos alunos inscritos na universidade já assistiam a pelo menos uma unidade curricular via online.

3 De notar que a percentagem de alunos com nota A é de 40,5%, com uma nota igual ou superior a B é de 69,3%, e igual ou superior a C é de 83,1%.

References

Bettinger, Eric P., Fox, L., Loeb, S., & Taylor, E. S., «Virtual Classrooms: How Online College Courses Affect Student Success?», American Economic Review, 107 (9), 2017,  pp. 2855-2875.

Cacault, M. P., Hildebrand, C., Laurent-Lucchetti, J., & Pellizzari, M., «Distance Learning in Higher Education: Evidence from a Randomized Experiment», IZA Discussion Papers 12298, Institute of Labor Economics (IZA), 2019.

Figlio, D., Rush, M., & Yin, L., «Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning», Journal of Labor Economics, vol. 31 (4), 2013, pp. 763-784.

AUTHORS

Hugo Reis is a research economist in the Economic Studies Department at Bank of Portugal. He is also an adjunct assistant professor at the Catholic University of Portugal and a member of the research unit at UCP and IZA, the Institute of Labor Economics.

He earned his PhD in Economics from University College London and has worked as a consultant for the Development Economics Research Group at the World Bank.

His research topics include the economics of education, program evaluation, labor markets, microeconometrics, and economic development. He has authored various studies published in international peer-reviewed journals, including the International Economic Review.

Pedro Freitas is an Economics PhD candidate at Nova School of Business and Economics (Nova SBE), where he is also a member of the Nova Center for the Economics of Education and earned his BA and MPhil in Economics.

His research focuses on economics of education and human capital, including teacher value-added measures, internal vs. external grade differentials, human capital externalities in the labor market and factors driving PISA test progression. His work utilizes comprehensive microdata from both the labor market and the Portuguese education system.

Pedro Freitas has been involved in impact evaluations of various educational projects in Portugal and was a visiting student at UCL’s Department of Economics and the Research Department of Bank of Portugal. He is also a teaching assistant at Nova SBE for undergraduate and master's courses.

Subscribe to our Newsletter

Keep up with all the news
Subscribe