I’ve often referred to the NAEP data and test scores when writing about education. But what most people don’t know is that the standardized testing component–which is what the NAEP is for all subject areas–was originally only supposed to be a small part of the NAEP evaluation. The original evaluations were much broader:
Keppel appointed a design committee, headed by the Carnegie Foundation’s Gardner, that decided the NAEP should assess any goal area for which schools devote “15-20% of their time…,” the “less tangible areas, as well as the customary areas, in a fashion the public can grasp and understand.” These should include all areas which “thoughtful laymen consider important for American youth to learn;” in art, citizenship, career and occupational development (including vocational education), foreign language, health and physical fitness (including such aspects of emotional health as self-image and self-confidence), literature, mathematics, music, reading, science, social studies, and writing. The citizenship area covered more than political knowledge; it also covered ethical principles and interpersonal behaviors that committee members believed comprised democratic citizenship. Consumer protection was also a subject of assessment; the committee felt that the ability to resist false advertising claims, and budget appropriately, was an important life skill with which youth should emerge from schools.
Following Ralph Tyler’s insistence that educators should assess the behavioral outcomes of education, not only the abstract skills that might lead to such outcomes, the committee designed survey questions about behavior as well as tests of skill. Schools, committee members agreed, don’t teach reading skills as an end in itself. Schools want students to use the skill by reading a newspaper, for example, and effective teaching must lead to such use. An assessment of outcomes should determine not only whether students have basic reading skills, but whether, as they grow older, students actually read newspapers. This behavioral outcome does not stem from direct instruction in language arts classes, but also from classes in other curricular areas, social studies, for example.
In civics education, the NAEP design committee was also interested in assessing behavior as well as factual knowledge. It included whether students showed concern for the welfare and dignity of others, supported rights and freedoms of all, helped maintain law and order, knew the main structure and functions of government, sought community improvement through active democratic participation, understood problems of international relations, took responsibility for their own personal development, and helped and respected their own families.
Attempting to assess each of these goals meant that paper and pencil tests could not be exclusively relied upon. Some outcomes of schools could only be assessed by observation of student behavior, or by survey techniques that verify the activities in which students engage.
Interestingly, there was also attention paid to whether students could work together, as well as their ethical development:
NAEP also assessed the development of attitudes considered essential to our democratic way of life. For example, NAEP attempted to determine whether students understood that individuals should be judged on their own merits and not be held responsible for others’ misdeeds. Interviewers asked 9- and 13-year-olds whether, if the father of a friend was jailed for theft, they would still invite the friend to their homes to play. To assess students’ commitment to free speech principles, 13- and 17-year-olds were asked if they thought that someone should be permitted to say on television that “Russia is better than the United States,” that “Some races of people are better than others,” or that “It is not necessary to believe in God.” NAEP reported that only 3 percent of 13-year-olds and 17 percent of 17-year-olds thought all three statements should be permitted.
The results of these assessments weren’t satisfactory. NAEP’s national report showed, for example, that only 4 percent of 13-year-olds defended the right of another group member to voice a different opinion, and only 6 percent were willing to defend their own viewpoints in the face of opposition. Have we improved since then? We have no way to know.
Budget cutbacks led to the elimination of these surveys. We have defined education very narrowly in the U.S., and that definition has been locked in by our evaluation methods.