Each one of us can probably name a teacher who had a key impact on our education, and who might even have influenced our professional path. However, what defines a good teacher? What makes some teachers have a greater impact on their students' learning? These are the questions we address to discuss while presenting measurements and instruments that can be used to evaluate teacher quality and its persistence over time.
The first approaches to measuring teacher quality, using comprehensive databases, sought to find a pattern between easily observed variables of teachers' performance and their impact on students’ results. Several of these studies failed to show a significant relationship between variables such as teachers' professional experience and formal education and students’ results. Using data on teachers from the state of Washington, Goldhaber & Theobald (2011) established a relationship between teachers who were laid off and their ability help students improve their results. It would be expected that these teachers would be the ones who least contributed to students’ success. However, that was not the observed pattern.
Thereby, recent studies on teacher quality revolve less around observable characteristics such as seniority and formal education, which are normally used to structure teachers' careers, and more around the concept of value-added, a method that measures teachers' average and continuing impact on student’s performance. This metric is measured by assessing the contribution of the teacher to the performance of the student in two universal and comparable tests, one taken at the student's starting point, before the first contact between the student and the teacher, and the other one at the end of the school year. This measurement must exclude any other factors that may affect students' performance, such as their family and socio-economic environment. Teacher value-added measurement is a widely spread methodology in education studies, and is used in some school districts in the United States as an accountability instrument for teacher performance. The estimations on the distribution of this value-added not only show there is quality heterogeneity across teachers, but also that hiring a teacher with a higher value-added can have a higher impact on student learning than many other changes in education policies.
Pros and cons of using value-added measurements
There is an ongoing and intense debate on whether value-added modelling is an unbiased and effective method to evaluate teacher quality. There are two main arguments against the use of value-added measurements: 1) the baseline and final standardised tests may not be the best instruments to measure the cognitive level of the students, as they may cover only part of the taught contents and fail to capture the multiple dimensions of the education process (Hanushek et al. 2005, McCaffrey et al. 2009); 2) with greater emphasis, it has been pointed out that even after controlling for the initial cognitive level of the students and a large set of socio-demographic variables, it is not possible to guarantee that the value-added measurements are unbiased, due to the way teachers and students are allocated to classes. The main issue around the discussion of the value-added measurements is the fact that the process of forming classes and allocating teachers to such classes is unknown. If specific teachers are chosen to teach certain type of students, this induces a selection problem which bias the value-added measures. In other words, only when teachers and students are randomly matched, will the value-added measurements be close to the true quality of the teacher. Whereas, if any other factor either restricts or predefines this matching, then the value-added estimates will be biased and may become invalid. Other factors that may add up to the difficulties in measuring teacher value-added are, for example, the mechanisms used by some families to enrol their children in a given school or their weight and influence in the decisions within the school.
In this regard, research on economics of education has taken two distinct positions: the first one states that even if we have baseline test results and a set of variables about the students and their families, it will not be possible to attain a rigorous teacher value-added measure (Cotfelter et al., 2006; Rothestein, 2010 e Horváth, 2015); the second one states that if such set of variables is available, it is possible to attain a precise teacher value-added measure, even if the teacher-student matching is not random (Kane et al., 2013; Chetty et al., 2014a; Chetty et al., 2014b). Following this reasoning, a paper on an experiment carried out in Los Angeles schools (USA) studied cases where students were randomly allocated to teachers, with no external factors influencing this matching. The results of this experiment showed that the value-added measures obtained in the randomised allocation of teachers to classes were aligned with those obtained without such randomisation, thus contributing to the robustness and validity of this method (Kane & Staiger, 2008).
If teacher value-added measurements are indeed able to determine teacher quality, which other instruments can we use to assess that quality? Moreover, are those instruments strongly correlated with teacher value-added measurements? The MET (Measures of effective teachers), a project financed by the Bill and Melinda Gates Foundation between 2009 and 2011, studied more than 3000 teachers across the USA. The study aimed to identify which other measurements would be able to capture the heterogeneity among teachers effectively, and it also confirmed the difficulty of assessing teacher quality. It is unlikely that the quality of a teacher is captured based on a single instrument (Kane et al., 2013), however, if we combine different evaluation instruments, such as classroom observations, peer evaluation, and surveys to members of the educational community, we will be closer to achieving a broad, complete and rigorous evaluation. Another study along the same lines, using data from Chicago, showed that school deans were also able to identify teachers with higher and lower value-added, (Jacob & Lefgren, 2008).
Therefore, we conclude that measuring teacher quality presents considerable difficulties and that an approach based on the combination of multiple criteria increases our capacity to assess teacher quality more rigorously. This view on the multidimensionality of the work developed by teachers is echoed in an experiment conducted in the Equator, where teachers were also randomly allocated to classes. The study concluded that teachers' non-cognitive skills were fundamental determinants to explain teacher value-added, namely the quality of their interaction with students, which varied considerably among teachers (Araujo et al., 2016).
The continued impact of a teacher throughout a student’s life
Another topic that has merited the attention of the economists of education is the duration of the impact teachers have on students. When we measure the teacher value-added, we are considering the specific academic year when the teacher interacts with the student in the classroom. But what impact does the teacher have in the following academic years? When students progress further along their academic paths, will their previous teachers continue to impact their school results? According to several empirical studies, only 20% to 30% of the contribution of previous teachers remains throughout their academic path (Jacob et al. 2010 e Chetty et al. 2014b). However, looking at the "very" long-run, an important paper using data from North Carolina (USA) shows that the impact of teachers gains particular importance when students enter the labour market. When students are exposed to higher value-added teachers in the kindergarten, their future gains in the labour market can be very significant. Other positive effects, however lower, were also identified in students that were exposed to higher value-added teachers between the 4th and 8th grades.
In sum, the economics of education has paid special attention to measuring teacher quality, especially due to the difficulty in determining what defines a good teacher. Given the failure of traditional methods, such as explaining the quality of teachers based on their professional experience, teacher value-added has been considered a comprehensive method to measure the teacher contribution between a baseline and the final performance in an academic year. This measure shows that the investment in good teacher has impact in the short-run but also in the long-run, when students move to the labour market. Such result stresses the invest in good teacher is one of the educational policies with a potential higher impact.
Measuring teacher quality remains the object of several studies in this field, among others, studies that seek to assess how value-added can be transmitted across teachers, meaning, how the teacher quality can be affected by working with other teachers with higher or lower value-added (Jackson & Bruegmann, 2009).
M. Caridad Araujo, P. Carneiro, Y. Cruz-Aguayo, and N. Schady. «Teacher quality and learning outcomes in kindergarten». The Quarterly Journal of Economics, 131 (3): 1415- 1453, 2016.
R. Chetty, J. Friedman, and J. Rockoff. «Measuring the impacts of teachers, I: evaluating bias in teacher value-added estimates». American Economic Review, 104(9): 2593-2632, 2014a.
R. Chetty, J. Friedman, and J. Rockoff. «Measuring the impacts of teachers II: teacher value-added and student outcomes in adulthood». American Economic Review, 104(9): 2633-2679, 2014b.
C. Cotfelter, H. Ladd, and J. Vigdor. «Teacher-student matching and the assessment of teacher effectiveness». Journal of Human Resources, 41(4):778-820, 2006.
Goldhaber D, Theobald R. 2011. «Managing the teacher workforce in austere times: the implications of teacher layoffs». Work. Pap. 2011-1.2, Cent. Educ. Data Res., Seattle
Hanushek, E. S. Rivkin. «Teacher quality». Handbook of the Economics of Education, Vol. 2, ed. EA Hanushek, F Welch, pp. 1051–78. Amsterdam: North Holland.
Hanushek EA, Kain JF, O’Brien DM, Rivkin SG. 2005. «The market for teacher quality». NBERWorking Pap. 11154
HORVÁTH, H., «Classroom assignment policies and implications for teacher value-added estimation», Working Paper, 2015.
KANE, T. e STAIGER, D., «Estimating teacher impacts on student achievement: an experimental evaluation», NBER Working paper nº 14607, 2008.
KANE, T. MCCAFFREY, J., MILLER, R., e STAIGER, D., Have we identified effective teachers? Validating measures of effective teaching using random assignment, MET Project, Bill and Melinda Gates Foundation, 2013.
JACKSON, C. K. e BRUEGMANN, E., «Teaching students and teaching each other: the importance of peer learning for teachers», American Economic Journal, Applied Economics, 2009(1-4), pp. 85-108.
JACOB, B. A e LEFGREN, L., «Can principals identify effective teachers? Evidence on subjective performance evaluation in education», Journal of Labor Economics, 26, 2008, pp. 101-36.
JACOB, B. A, LEFGREN, L., e SIMS, D., «The persistence of teacher-induced learning», Journal of Human Resources, 45, 2010, pp. 915-43.
ROTHESTEIN, J., «Teacher quality in educational production: tracking, decay, and student Achievement», Quarterly Journal of Economics, 125 (1), 2010, pp. 175-214.