Good Teachers, Long-Term Effects and Test Scores: Don't Call the Principal Yet

Not enough Milk Duds or popcorn in the world to watch this one.

Good teachers make a difference.

There’s now more proof of the benefits according to a new study of 2.5 million elementary and middle school students tracked over 20 years. Economists at Harvard and Columbia found evidence that having an effective teacher, a single good teacher, in grades 4 to 8 can impact college matriculation, lifetime earnings and also teen pregnancy. Pretty impressive.

So what is good teaching anyhow? Good test scores. Period.

At least in this study conducted on behalf of the National Bureau of Economic Research. In economic parlance, good teachers are “value-added” – meaning, they raise student test scores. Here’s the formal definition given by the researchers in the paper:

A teacher’s “value-added” is defined as the average test-score gain for his or her students, adjusted for differences across classrooms in student characteristics (such as their previous scores).

In other words it’s a term you may be well-advised to forget before the next parent-teacher conference.

First the economists figured out if differences in test scores gains across teachers could actually be attributed to the teachers and not something hinky about the students like the selection of certain students into classrooms. You know, the-Mrs.-Favorite-Teacher-gets-the-best-students theory. Despite popular opinion about certain teachers getting certain students (mouth shut, sitting on hands) the researchers report that yes, teachers who show test score gains – as determined by their stupefyingly complex mathematical model – can claim bragging rights if not higher pay checks.*

Now all these fresh results are very encouraging if your child should be lucky enough to have a value-added educational experience. Just how lucky? Don’t start imagining Harvard just yet. Turns out the effect of any one teacher remains rather slight**.

All else equal, a student with one excellent teacher for one year between fourth and eighth grade would gain $4,600 in lifetime income, compared to a student of similar demographics who has an average teacher. The student with the excellent teacher would also be 0.5 percent more likely to attend college. Big Study Links Good Teachers to Lasting Gain, New York Times, January 2, 2012

$4,600 dollars, not exactly the jackpot. Obviously this sample didn’t oversample social science grad students otherwise that number might plummet to single digits given my own rough-hewn formula: better teachers = more education = more doctorates = less pay.

Surely over time the effects likely accumulate so the value of multiple effective teachers would be more impressive. Those aggregate effects also matter to school principals, school boards and policy makers, you know, the folks making decisions based on those test scores or rather, teacher ratings based on those test scores.

Cue applause, jeers, sighs, etc. as necessary.

Yes the researchers considered a variety of factors that might influence student test scores and the other outcomes. The results held even after controlling for previous test scores and a range of parental characteristics (income, maternal age at birth, marital status, 401K contributions – hey, love economists – especially if they admit stealing well-established concepts from psychology like motivation and revenge).

This crew has controlled for quite a bit but I still question how anyone could consider all the factors between middle school and college not to mention twenty years of 9 to 5. If good teaching propels a student to take a harder math course, let’s say, that then leads to a stronger interest in math, an invitation to the math club then a math major then an accounting job then a graduate degree and a higher salary then we can say the teacher’s effect so many years in the past had some effect but it surely wasn’t the only factor along the way.

Did your 5th grade teacher really make you $4,600 richer? It’s terribly hard to separate the effects of the math major from the math club from the middle school math department. The team here has tried and figured leaps and falls from year to year into the equations but still. They report the “effects”, ahem, “fade” to about a third after three years and then remain constant.

It’s those tricky intervening variables that are so hard to pin point and assess then figure into the model. The statistical analyses are mind-numbing and I’m not sure to what extent the present research captured them. It’s a 90+ page working draft that’s yet to be down-sized into a peer-reviewed published paper – note that lack of peer-review did not prevent media coverage. Read through a lot and I can tell you that it’s nearly impossible to get at those in a study of this magnitude. Maybe we’ll soon read it in a journal.

I suppose it’s reassuring to have evidence quality teaching matters in the long-term that is if you agree that’s what the test scores measure. We would probably all agree teachers matter and here’s some proof albeit of a mostly financial nature. It would have been cool to see links to a wider range of outcome variables like maybe SATs or critically thinking skills but that’s another study not easily assessed by tax records like the researchers used.

Now I’m off to buy paper products for 3rd Grade Teacher Appreciation Day. I am not kidding. Check your own calendar.

*Because the researchers are economists and thus must like to calculate cost-savings and other practical phenomenon – instead of say prejudice or empathy, ho hum – they report that rewarding highly effective teachers may not be the best or most cost-effective plan. I know, sit down, those of you cheering for merit pay. In part this seemingly contrary conclusion comes becauses in their model it appears cheaper to replace the worst teachers and because the best teachers might not be motivated by more pay. They do what they do because they care, those sappy, sappy people. Hello, Mom, Dad, Aunt Jackie, Aunt Barbara, Aunt Peggy..Grandma…Grandpa..Uncle Mike…

**Note to parents of students with sub-par teachers: Rest assured, in this study the reverse is also true. Crappy teachers also seem to have only modest long-term “effects”.

THE LONG-TERM IMPACTS OF TEACHERS: TEACHER VALUE-ADDED AND STUDENT OUTCOMES IN ADULTHOOD
Raj Chetty, John N. Friedman, Jonah E. Rockoff, Working Paper 17699
http://www.nber.org/papers/w17699