IHE blogger responds to IHE’s “Harsh Take” on assessment

John Warner who blogs for IHE, has posted a piece that proposes ways to do better assessment. He has some interesting ideas about measuring things, like student food security, that contribute to or detract from a “learning atmosphere.”   I am not sure about the details of what he proposes, but he makes an important point.  The current highly bureaucratized state of assessment ignores anything that is external to the curriculum and the classroom.  I would argue that most of higher ed’s serious problems lie elsewhere and are far more complex and structural than anything that can be addressed by adding an extra critical thinking exercise or what ever your loop closer of choice is.

I like this passage, which perfectly captures the reality of assessment as it exists at the chalkboard face (or maybe white-board face would now be more appropriate).

As a frontline instructor, my role in the larger assessment regime has been largely pro-forma and somewhat mysterious. I have been asked to randomly collect artifacts that fit the “learning objectives” for the course – learning objectives imposed from somewhere above me[1]– and hand them over to some other body that does something to them, and then I do it again.

Assessment as practiced at the department and institutional level could not have been less relevant to my day-to-day work.

Are students improving at writing? The answer is yes.

How do I know? Because I do…and because students say so.

Take my word for it, except of course, taking my word for it is apparently not enough.

 

[1]The learning objectives are usually vague and unobjectionable and easy enough to attach to something I was planning on doing anyway.

Laurie Fendrich got it right in 2007

Back in 2007 when assessment had just started to really intrude into academic life, Laurie Fendrich wrote the first (and still the best) statement I have read about assessment. Her references to May Day parades,  Mao, and forced self-criticism, presciently captured the Taylorism meets Orwell quality of assessment.  The assessment industry has finally conceded that it has gone down an  “unproductive path.”  They can’t claim that they had not been warned over a decade ago about what they were doing.

This is just the first paragraph.  Read the whole thing here. Unfortunately it is so good that 12 years after its publication, the Chronicle still has it paywalled.

 

Outcomes-assessment practices in higher education are grotesque, unintentional parodies of both social science and “accountability.” No matter how much they purport to be about “standards” or student “needs,” they are in fact scams run by bloodless bureaucrats who, steeped in jargon like “mapping learning goals” and “closing the loop,” do not understand the holistic nature of a good college education. For all the highfalutin pronouncements accompanying the current May Day parade of outcomes assessment, in the end they boil down to a wholesale abandonment of the very idea of higher education. Whatever their purpose, outcomes-assessment practices force-march professors to a Maoist countryside where they are made to dig onions until they are exhausted, and then compelled to spend the rest of their waking hours confessing how much they’ve learned by digging onions. The mentor-protégé model of a college education is gone. We now confront the robot model, in which knowledge is reduced to what Nietzsche called “knowledge stones” — bits of information that administrators can count and students can digest without thinking.

Best Comment on yesterday’s IHE article

Kara Moloney wrote a comment challenging the accuracy of the IHE’s reporting on the events at WASC.  Among other things, she was put out that Bob Shireman was quoted at length despite being an audience member rather than a panelist.  John Ahearn had this response.  If anyone knows him, please buy him a beer for his service to academia.

” Well-intentioned and imperfect people dedicated to students and their learning….”

…who caused many of us to expend massive amounts of time and resources on an ultimately futile task. The magnitude of the failure would warrant a thorough apology, not whitewashing or a passing comment about humility, and perhaps an acknowledgement that the people who expressed skepticism about this costly enterprise were correct.

“One more thing: Who are these ‘many assessment haters in higher education’? How many of them actually exist? I ask because I have yet to encounter a hater in person.”

You are obviously not a regular member of an academic department. In most departments I’ve encountered–including social-science and education departments, from whom you might expect some buy-in–the attitude is that we need to get “assessment” done as quickly and painlessly as possible so that we can get back to the task of educating students as best we can, figuring out what they’ve learned, and thinking about how they can learn more.

Another example: At the beginning of the assessment craze, I happened to be serving as chair of my university’s curriculum committee. The assessment director (I forget her title) sent me an email telling me that I had to require learning objectives to be spelled out in the syllabus of every course, typically with explanations of which assessment methods would be used to measure each learning objective, etc., etc. After a glance at the org chart and the policy manual, I refused, and told her that she was free to show up to the committee meetings if she wished to make that request.

I never heard from her again. While the administration or our accreditors eventually mandated learning outcomes, etc., in the syllabus, I like to think that I held them off for a few years, and saved lots of people’s time in the process.

In IHE: Harsh Take on Assessment… From Assessment Pros

WASC met last week and the theme of the conference was “Provocative Questions…Courageous Answers.”  IHE has this article about what sounds like a pretty thoughtful panel on why assessment is not working.

“It’s not just that faculty members are crabby and hate change … There are good reasons why faculty hate it. It’s real and it’s earned,” Jankowski said. (An Inside Higher Ed survey of faculty members last year, for instance, found that 59 percent of respondents agreed that assessment efforts “seem primarily focused on satisfying outside groups such as accreditors or politicians,” rather than serving students.) Essays like this also reflect faculty disdain.

There is a certain irony to having the head of NILOA lament the state assessment, but it’s a step in the right direction.

College, Learning, and Employment: The Enterprise Car Rental Example

The latest entry in the debate as to whether the value of college lies in what you learn while you are there or in what it signals about you comes from an article in the National Review Online.  James Piereson and Naomi Schaefer Riley build on an article that appeared in the Chronicle: “Why Thousands of College Grads Start their Careers at a Rental-Car Company.” (paywalled)

This is what economists such as Ohio University’s Richard Vedder and George Mason’s Bryan Caplan have been arguing for years: College degrees are simply a signifier — an easy way of telling an employer that you have a basic grasp of the English language, some rudimentary math skills, and the ability to show up on time in clean clothes. On those measures, is a graduate of the University of Michigan any different from a graduate of Michigan State or Northern Michigan University? Not really. Does a 3.8 GPA predict that you will do better or worse at managing a car-rental office than someone with a 2.8 GPA? Probably not. Does majoring in business predict that you will do a better job than an English major or a sociology major or a physics major? It’s unlikely.

The management at Enterprise are saying aloud what many employers know to be true. Bosses who require a college degree are taking advantage of a system that does the sorting for them. They understand that a bachelor’s degree is not really necessary for doing an entry-level job, and that whatever your educational background, you will require significant training to do well in that particular position.

I am not convinced that signaling is the only reason college is valuable, but it does make one wonder whether our efforts at increasing retention and putting programs online in ways that no longer require that students “show up on time in clean clothes” is undermining a central and important aspect of college.

Faculty Complaints from 1953

My favorite academic novel is Kingsley Amis’ Lucky Jim. I reread it every couple of years.  My wife can always tell that I have succumbed to the urge revisit the book because it is the only book that makes me giggle out loud.

Each time I reread it, I notice something new.  This year it was my spring break travel book and it was this conversation between the book’s protagonist Jim Dixon (a first year history lecturer) and his friend Beesley (lecturer in English) that stuck out.

‘It’s the same everywhere you look; not only this place, but all the provincial universities are going the same way…go to most places and try to get someone turfed out merely because he’s too stupid to pass his exams—it’d be easier to sack a prof.  That’s the trouble with having so many people here on Education Authority grants, you see.’

‘How do you mean? The students have to get their money from somewhere.’

‘Well, you know, Jim. You can see the Authorities’ point in a way. “We pay for John Smith to enter College here and now you tell us, after seven years, that he’ll never get a degree. You’re wasting our money.” If we institute an entrance exam to keep out the ones who can’t read or write, the entry goes down by half, and half of us lose our jobs.  And then the other demand: “We want two hundred teachers this year and we mean to have them.”  All right, we’ll lower the pass mark to twenty percent and give you the quantity you want, but for God’s sake don’t start complaining in two years’ time that your schools are full of teachers who couldn’t pass the General Certificate themselves, much less teach anyone to pass it.  It’s a wonderful position, isn’t it?’

One of the conceits of the various reform movements, which of course includes assessment, is that there was once a time when grades were not inflated, when students were serious, engaged and well-prepared, and they learned lots of stuff in College.  Lucky Jim was written right after the war when Amis was a lecturer at provincial red brick much like the unnamed university where Dixon teaches.  Clearly the same types of concerns that surround standards, grades, completion rates, and student preparation were just as prevalent in 1953 as they are now.  No doubt if someone were to look though Peter Abelard’s personal diary medieval scholastics would be found to have had all the same concerns.

 

 

 

Online Program Managers and the Blurring of the Lines between Not-for-Profit and For-Profit Institutions

Huffpost has a long and interesting article on the threat posed to higher education by the proliferation of Online Program Managers. OPMs help colleges develop, manage and market online programs, usually master’s-level programs. In exchange for those services they typically pocket 50 to 60% of the tuition. The article is by Kevin Carey of New America and it includes a section on Bob Shireman’s efforts to regulate for-profits while he was in the Department of Ed.

The short version of the article is that the OPMs take the now discredited for-profit model and use their connections to legitimate non-profits to create a cloak of repeatability around their work.  They started by working with prestigious schools but have shifted to partnerships with financially troubled smaller privates and state schools.  In some these partnerships it’s getting harder and harder to tell whether the OPM or the college is the senior partner.

It’s a good article, although one of the OPMs has disputed some of it’s claims.  Carey is right that the OPMs prefer master’s programs because they are more opaque in their admissions practices than undergraduate programs, so those programs can be more “accessible” than undergraduate programs without affecting a school’s published admissions standards.  But more than that, my sense is that they prefer master’s programs that confer a credential the quality of which is irrelevant.  This is especially the case with master’s in education.  For a teacher to get the pay bump that comes from obtaining a master’s degree it does not matter whether the degree was earned in residence in a face-to-face to program that required lab time and real research or just knocked out online with the cheapest and least rigorous program available.  You still get the same raise whether you earned your degree at Cal Tech or from De Vry.

I wrote about this (Automatic Pay Raises for Teachers Create Perversive Incentives in Graduate Education)a while back for the James G. Martin Center.

 

 

 

 

 

Assessment is on an “unproductive path” and it’s your fault

An article in the March/April issue of Change: the Magazine of Higher Learning begins with this tidbit:

The assessment of student learning in higher education has been headed down an unproductive path for too long.  Not enough faculty and administrators engage in an assessment process that fosters cognitive and affective learning for all their students.

Yes, really.  And this is coming from authors associated with AAC&U, NILOA and AALHE.  People representing the exact organizations that have done the most to impose the bureaucratic form of assessment that is now the norm in higher education have belatedly recognized that assessment is unproductive.  But it’s not them, it’s you.  The good news is that they have a new plan and this time it will work.  Trust them. After all they have only been leading us down an unproductive path for twenty years now, so surely they are just the people to get the bumbling faculty back on track. What’s the new plan? We need to shift our thinking about assessment so that we focus less on “assessment of learning” and more on “assessment for learning.”  I guess now that we have the verbs in our learning outcomes sorted out it’s time to start thinking about prepositions.

How is that anyone still takes this seriously?

 

Letters to the Chronicle

The Chronicle has posted two letters in response to my most recent article.  One is from Dan Sullivan and Kate McConnell who wrote because they felt I had misunderstood an article they had written.  I have previously posted a critique of the article in question.  The second and more recent letter is from Catherine Wehlburg.  She takes the argmentum ad lapidem approach.  I am wrong because my argument is just “silly,”  so there is no need to deploy evidence to counter it.

I also take issue with the statement that “learning isn’t the most important element of a college education.” Now, that’s just silly. Of course learning is the most important part of a college education. That is why students attend college — it isn’t just to collect course credits (though some students might think that it is). The point is to learn — new skills, behaviors, knowledge, and even new perspectives. And there is a great deal of data on what students are learning and how that learning is helping them be successful in the workforce and in life.

This just reinforces my sense that the arguments for assessment are ultimately based on people’s intuition about what college is about or ought to be about rather than thoughtful or critical reflection.

Repost: Stop Sacrificing Children to Moloch

Below is a repost of 2018’s most read entry in the blog.  A highly revised version of it should appear soon in Cultural and Pedagogical Inquiry.

 

As someone who has been publicly critical of learning outcomes assessment for a long time, one of the questions I am often asked is: “If you are so opposed to assessment, what would you replace it with?” By way of an answer I have started resorting to this fable:

 

Imagine that you live in a Bronze Age village. You and everyone else in the village depend for your livelihood on subsistence farming, so you have a keen interest in the success of your crops. Because of that, you have developed a good sense of when to plant what crop, what types of soil work best with specific crops, when to weed, when to harvest and so on. It’s not scientific knowledge but it works pretty well. Still, you are always on the look out for ways of improving your yields. Continue reading “Repost: Stop Sacrificing Children to Moloch”