As we like to say here, you have to understand the limitations of your data. One of the fads that education ‘reformers’ really like is value-added measurement, also known as value-added testing: the basic idea is to use a statistical model to determine how much of a student’s improvement in test scores is due to his or her teacher, then this effect is assessed across all of the students belonging to a given teacher, and presto chango, we can discriminate ‘good’ from ‘bad’ teachers. I’ve discussed the problems with value-added testing before, so I won’t rehash them here.
One assumption underlying this approach is that test scores of subject materials are the outcomes we want to measure. But one could make a compelling argument that eighth grade math scores are a proxy for more important measures, such as finishing high school, staying out of trouble, and adult earnings. Many parents would ultimately be more worried about the latter than a specific test score in junior high. If most reformers are to be believed, our education ‘crisis’ is hurting our economic competitiveness (squishy librul concerns like being a well-rounded citizen don’t really seem to enter into the discussion…). So, if nothing else, education reformers should be interested in determining how teachers affect these long-term outcomes.
C. Kirabo Jackson gets at this in a recent paper (pdf is of non-paywall version; boldface mine):
Using administrative data with students linked to individual teachers, 9th grade English and Algebra teachers have economically meaningful effects on test scores, absences, suspensions, on time 10th grade enrollment, and grades. The results indicate that teachers have larger effects on non-cognitive ability than they do on cognitive ability. A variety of additional tests suggest that these effects can be interpreted causally. Teacher effects on test scores and teacher effects on non-cognitive ability are weakly correlated such that many teachers in the top of test score value-added will also be in the bottom of teachers at improving non-cognitive skills. This means that a large share of teachers thought to be highly effective based on test score performance will be no better than the average teacher at improving college-going or wages. Under reasonable assumptions about the importance of non-cognitive skills for long run outcomes, calculations indicate that test score based measures may understate the importance of teachers by between 50 and 80 percent.
This study highlights that a failure to account for the effect of educational interventions on non-cognitive skills can lead to biased estimates of the effect of such interventions on important long-run outcomes. The two-sample factor analytic framework put forth in this paper can be used in other settings to estimate the effects of educational interventions on both cognitive and non-cognitive ability. Finally, the results illustrate the large potential gains that may be associated with making policy decisions about educational interventions based on estimated effects on both cognitive and non-cognitive outcomes rather than just test scores alone.
Admittedly, this is one study, but this is really important–and probably why it hasn’t been discussed at all in the mainstream media. The idea that value-added testing focused on subject material is telling us very little about long-term outcomes, such as income and high school graduation, isn’t, for obvious reasons, being promoted by education ‘reformers.’ Pretty certain that Pearson, which makes lots of Common Core-associated tests and materials, won’t be discussing this.
But they really are concerned about the children.