Are Older U.S. Students Learning Less? Why You Have to Understand the Limitations of Your Data

Back when I worked on the microbiome, I noted several times that you have to understand the limitations of your data. You are always measuring something, but you might be pushing the interpretation of those measurements too far.

Recently, I criticized psychologist Laurence Steinberg for claiming that twelfth grade NAEP scores have flat-lined (ZOMG! TEH STUDENTZ ARE STUPID!). Of course, as is usually the case with such claims, once you disaggregate the scores (i.e., break them down by demographic factors), all students have made gains. But here’s something I didn’t know about the twelfth grade NAEP scores (boldface mine):

I responded to Larry Steinberg, whom I knew years ago, and pointed out that the NAEP scores for seniors are meaningless. When I was on the NAEP board in the early 2000s, we devoted a full meeting to discussing the fact that high school seniors don’t even try on NAEP tests. They know the tests don’t count towards high school graduation or college admission; they don’t count for anything, and the kids don’t care about them. They doodle on the answer page, they answer in patterns (like checking off every A), or they leave pages blank. They aren’t dumb. They know what they are doing. They are asked to jump over a meaningless hurdle, and they treat it as a joke. But the adults take their tomfoolery as evidence that they are unmotivated, possibly stupid. I don’t think the kids are stupid. I imagine how I feel when someone calls me on the phone and starts asking questions; usually I hang up, or I say something uncooperative because I don’t like to be interrupted for no reason to fit into someone else’s plan. I expect that the seniors feel the same way.

Interesting. We might want to reconsider the claims that the upper grades in the U.S. are doing poorly. Maybe, instead, they’re doing fine: wisdom might be too strong a word for a seventeen year-old, but they seem to have their heads screwed on better than some would like to credit.

Of course, I’m sure I failed to persuade you if you’re convinced there’s a problem no matter what.

This entry was posted in Education. Bookmark the permalink.

2 Responses to Are Older U.S. Students Learning Less? Why You Have to Understand the Limitations of Your Data

  1. Min says:

    Whenever somebody talks about how dumb kids are today I trot out the Flynn Effect. They’re smarter than we were. 😉

  2. albanaeon says:

    An appreciation for bs is an adult, smart ability.

    What should scare a lot of the reformers is how many of these high schoolers are aware their NCLB/Race to the Top/free market reformed education was bs. I work at a Community College and the number of people that are really mad that they have to relearn basic skills, like critical thinking, arguing positions, etc. is remarkable.

    We’ll probably regret this little experiment quite a bit in the future and we’re relying on this generation, because a lot of them will have proof that older generations sabotaged them and might not be all that forgiving.

Comments are closed.