“Everything simple is false. Everything complex is unusable.”
As someone who has been publicly critical of learning outcomes assessment for a long time, one of the questions I am often asked is: “If you are so opposed to assessment, what would you replace it with?” By way of an answer I have started resorting to this fable:
Imagine that you live in a Bronze Age village. You and everyone else in the village depend for your livelihood on subsistence farming, so you have a keen interest in the success of your crops. Because of that, you have developed a good sense of when to plant what crop, what types of soil work best with specific crops, when to weed, when to harvest and so on. It’s not scientific knowledge but it works pretty well. Still, you are always on the look out for ways of improving your yields. Continue reading “Stop Sacrificing Children to Moloch”
If you have been waiting for an opportunity to talk about your doubts and concerns about assessment this summer (and who would not want to use valuable summer travel time to talk about assessment), you have several opportunities in the next couple of months.
Friday May 4th, I will a panelist at the San Francisco State University chapter of the California Faculty Association meeting. The theme of the conference is “Resisting the Neoliberal University.” I will be on the morning plenary at 9:00 for the session on the “Mechanics of Managerial Takeover” and then will be in a breakout session called “Tools of Control and Authority.”
Then, in early June, there will be two panels at the Association for the Assessment of Learning in Higher Education (AALHE) meeting in Salt Lake City that should be of interested to assessment doubters. Both are on June 6 and feature me, Bob Shireman, Lynn Priddy, Dave Eubanks and Josie Welsh. The first session at 8:00 will be on “Identifying Problems in Assessment” and the second at 10:30 will about Identifying Solutions to Problems in Assessment.”
The Atlantic just ran an article by Olga Khazan that traces the history of the idea that people have distinctive learning styles. The notion first emerged in the 1990s and rapidly became popular with students and teachers because it seemed to offer both an explanation for why some students failed to do well in school and a solution to that problem. Students failed, the argument went, because their teachers’ instruction did not match their learning styles. The solution was to ensure that auditory learners got auditory instruction, visual learners got visual instruction, and so on. It’s an appealing idea and makes a sort of intuitive sense.
Unfortunately, the research does not support it. Learning styles have been debunked repeatedly, most recently here. Continue reading “The Persistence of Dubious Ideas”
There was an article in Inside Higher Ed yesterday by Sam Wineburg, Joel Breakstone, and Mark Smith, all of Stanford History Education Group. In it they argue that historians don’t like to do assessment. They are right about that. They are also correct that many historians claim that history courses teach critical thinking.
In an effort to assess this claim, they have tested several groups of students on their ability to use historical evidence. They found that few could do it. Their conclusion is that students are not learning the critical thinking skills that historians claim that their courses teach. Continue reading “Assessment and History”
Among other things, the studies described in this article seem to base their evaluation on student “self-reported learning outcomes.” What the heck does that mean? What students think they learned?
It’s not just learning outcomes assessment that suffers from data quality issues. Much of the research on education (and many other fields) suffers from a willingness to use, to make inferences from, and to publish based on dubious data.
By Robert Shireman
Starting in 2000 and repeating every year for most of the decade, a distinguished national committee chaired by former North Carolina Governor James B. Hunt Jr., a Democrat, released an annual report card on higher education. Color-coded maps showed every state’s A-F grade in each of several categories including college affordability, participation, and graduation. In one category, though, every state, every year, got an “Incomplete.” That category for which every state was deficient, was “learning,” because, as the authors complained, there is “no nationwide approach to assessing learning” in college, “no common benchmarks that would permit state comparisons of the knowledge and skills of college students.” Continue reading “Shireman on Jerry Muller’s Tyranny of Metrics”
The National Institute for Learning Outcomes Assessment has published a list of material they see as constituting a response to Molly Worthen and, to a lesser extent, me. To me (confirmation bias alert), it sounds very defensive and I am not sure how carefully they have read the articles that they posted. Some of them, while not agreeing entirely with Worthen, seem to accept much of her argument. Eubanks and McConnell are examples of this and I am not sure why Margaret Spellings’ recent response to Bryan Caplan is included at all.
The American Association of Retired Persons Medicare handbook, deceptively titled “Hello Simplicity,” is typical: endless paragraphs of incomprehensible blather interspersed with pages of charts bearing an uncanny and discomfiting resemblance to outcomes-assessment rubrics.
No one does thoughtful diatribe quite like Fendrich. Read the whole thing here.
When Bad Assessment was in its formative stage, we considered a couple of different names. One was “The Committee for Sane Assessment.” In the end we settled on Bad Assessment, but something recently turned up on one of the assessment list-serves (sp?) that gives one an idea of what a reasonable type of assessment might be. Its author was, no surprise here, Dave Eubanks of Furman. Here it is: Continue reading “What would sane assessment look like?”