How to measure a good teacher

Each one of us can probably name a teacher who had a key impact on our education, and who might even have influenced our professional path. However, what defines a good teacher? What makes some teachers have a greater impact on their students' learning? These are the questions we address to discuss while presenting measurements and instruments that can be used to evaluate teacher quality and its persistence over time.  

The first approaches to measuring teacher quality, using comprehensive databases, sought to find a pattern between easily observed variables of teachers' performance and their impact on students’ results. Several of these studies failed to show a significant relationship between variables such as teachers' professional experience and formal education and students’ results. Using data on teachers from the state of Washington, Goldhaber & Theobald (2011) established a relationship between teachers who were laid off and their ability help students improve their results. It would be expected that these teachers would be the ones who least contributed to students’ success. However, that was not the observed pattern.

Thereby, recent studies on teacher quality revolve less around observable characteristics such as seniority and formal education, which are normally used to structure teachers' careers, and more around the concept of value-added, a method that measures teachers' average and continuing impact on student’s performance. This metric is measured by assessing the contribution of the teacher to the performance of the student in two universal and comparable tests, one taken at the student's starting point, before the first contact between the student and the teacher, and the other one at the end of the school year. This measurement must exclude any other factors that may affect students' performance, such as their family and socio-economic environment. Teacher value-added measurement is a widely spread methodology in education studies, and is used in some school districts in the United States as an accountability instrument for teacher performance. The estimations on the distribution of this value-added not only show there is quality heterogeneity across teachers, but also that hiring a teacher with a higher value-added can have a higher impact on student learning than many other changes in education policies.

Pros and cons of using value-added measurements

There is an ongoing and intense debate on whether value-added modelling is an unbiased and effective method to evaluate teacher quality. There are two main arguments against the use of value-added measurements: 1) the baseline and final standardised tests may not be the best instruments to measure the cognitive level of the students, as they may cover only part of the taught contents and fail to capture the multiple dimensions of the education process (Hanushek et al. 2005, McCaffrey et al. 2009); 2) with greater emphasis, it has been pointed out that even after controlling for the initial cognitive level of the students and a large set of socio-demographic variables, it is not possible to guarantee that the value-added measurements are unbiased, due to the way  teachers and students are allocated to classes. The main issue around the discussion of the value-added measurements is the fact that the process of forming classes and allocating teachers to such classes is unknown. If specific teachers are chosen to teach certain type of students, this induces a selection problem which bias the value-added measures. In other words, only when teachers and students are randomly matched, will the value-added measurements be close to the true quality of the teacher. Whereas, if any other factor either restricts or predefines this matching, then the value-added estimates will be biased and may become invalid. Other factors that may add up to the difficulties in measuring teacher value-added are, for example, the mechanisms used by some families to enrol their children in a given school or their weight and influence in the decisions within the school.

In this regard,  research on economics of education has taken two distinct positions: the first one states that even if we have baseline test results and a set of variables about the students and their families, it will not be possible to attain a rigorous teacher value-added measure (Cotfelter et al., 2006; Rothestein, 2010 e Horváth, 2015); the second one states that if such set of variables is available, it is possible to attain a precise teacher value-added measure, even if the teacher-student matching is not  random (Kane et al., 2013; Chetty et al., 2014a; Chetty et al., 2014b). Following this reasoning, a paper on an experiment carried out in Los Angeles schools (USA) studied cases where students were randomly allocated to teachers, with no external factors influencing this matching. The results of this experiment showed that the value-added measures obtained in the randomised allocation of teachers to classes were aligned with those obtained without such randomisation, thus contributing to the robustness and validity of this method (Kane & Staiger, 2008).

If teacher value-added measurements are indeed able to determine teacher quality, which other instruments can we use to assess that quality? Moreover, are those instruments strongly correlated with teacher value-added measurements? The MET (Measures of effective teachers), a project financed by the Bill and Melinda Gates Foundation between 2009 and 2011, studied more than 3000 teachers across the USA. The study aimed to identify which other measurements would be able to capture the heterogeneity among teachers effectively, and it also confirmed the difficulty of assessing teacher quality. It is unlikely that the quality of a teacher is captured based on a single instrument (Kane et al., 2013), however, if we combine different evaluation instruments, such as classroom observations, peer evaluation, and surveys to members of the educational community, we will be closer to achieving a broad, complete and rigorous evaluation. Another study along the same lines, using data from Chicago, showed that school deans were also able to identify teachers with higher and lower value-added, (Jacob & Lefgren, 2008).
 

The continued impact of a teacher throughout a student’s life

Another topic that has merited the attention of the economists of education is the duration of the impact teachers have on students. When we measure the teacher value-added, we are considering the specific academic year when the teacher interacts with the student in the classroom. But what impact does the teacher have in the following academic years? When students progress further along their academic paths, will their previous teachers continue to impact their school results?  According to several empirical studies, only 20% to 30% of the contribution of previous teachers remains throughout their academic path (Jacob et al. 2010 e Chetty et al. 2014b).  However, looking at the "very" long-run, an important paper using data from North Carolina (USA) shows that the impact of teachers gains particular importance when students enter the labour market. When students are exposed to higher value-added teachers in the kindergarten, their future gains in the labour market can be very significant. Other positive effects, however lower, were also identified in students that were exposed to higher value-added teachers between the 4th and 8th grades.

Measuring teacher quality remains the object of several studies in this field, among others, studies that seek to assess how value-added can be transmitted across teachers, meaning, how the teacher quality can be affected by working with other teachers with higher or lower value-added (Jackson & Bruegmann, 2009).