Student Course Evaluations Get An 'F'

lolakimono · ‎03-29-2012

http://www.npr.org/sections/ed/2014/09/26/345515451/student-course-evaluations-get-an-f

You're next.

At Denny's, diners are asked to fill out comment cards. How was your meal? Were you satisfied with the quality of service? Were the restrooms clean?

In universities around the world, semesters end with students filling out similar surveys about their experience in the class and the quality of the teacher.

Student ratings are high-stakes. They come up when faculty are being considered for tenure or promotions. In fact, they're often the only method a university uses to monitor the quality of teaching.

Recently, a number of faculty members have been publishing research showing that the comment-card approach may not be the best way to measure the central function of higher education.

Philip Stark is the chairman of the statistics department at the University of California, Berkeley. "I've been teaching at Berkeley since 1988, and the reliance on teaching evaluations has always bothered me," he says.

Stark is the co-author of "An Evaluation of Course Evaluations," a new paper that explains some of the reasons why.

For one thing, there's response rate. Fewer than half of students complete these questionnaires in some classes. And, Stark says, there's sampling bias: Very happy or very unhappy students are more motivated to fill out these surveys.

Then there's the problem of averaging the results. Say one professor gets "satisfactory" across the board, while her colleague is polarizing: Perhaps he's really great with high performers and not too good with low performers. Are these two really equivalent?

Finally, there's the simple fact that faculty interactions with students and the student experience in general vary widely across disciplines and types of class. Whether they're in an an upper-division seminar, a studio or lab, or a large lecture course, students are usually asked to fill out the same survey.

Stark says his paper is unlikely to surprise most faculty members: "I think that there's general agreement that student evaluations of teaching don't mean what they claim to mean." But, he says, "there's fear of the unknown and inertia around the current system."

Michele Pellizzari, an economics professor at the University of Geneva in Switzerland, has a more serious claim: that course evaluations may in fact measure, and thus motivate, the opposite of good teaching.

His experiment took place with students at the Bocconi University Department of Economics in Milan, Italy. There, students are given a cognitive test on entry, which establishes their basic aptitude, and they are randomly assigned to professors.

The paper compared the student evaluations of a particular professor to another measure of teacher quality: how those students performed in a subsequent course. In other words, if I have Dr. Muccio in Microeconomics I, what's my grade next year in Macroeconomics II?

Here's what he found. The better the professors were, as measured by their students' grades in later classes, the lower their ratings from students.

"If you make your students do well in their academic career, you get worse evaluations from your students," Pellizzari said. Students, by and large, don't enjoy learning from a taskmaster, even if it does them some good.

There's an intriguing exception to the pattern: Classes full of highly skilled students do give highly skilled teachers high marks. Perhaps the smartest kids do see the benefit of being pushed.

Measuring the teacher by how well the student did in the next course is an important part of this experiment. Previous papers, says Pellizzari, compared student ratings to student grades within that same course. An easy-A prof may earn five stars in return for handing out good grades. But this leniency, his research suggests, does the students no long-term favors.

Both Pellizzari and Stark agree that student surveys should be used in a much more limited way, to capture student satisfaction. And they could perhaps be used to gather information on factual points like whether the professor showed up on time or canceled class more than once or twice.

In addition, they'd like to see other methods of evaluating teachers. As department chairman in statistics, Stark actually has implemented new methods and has seen interest in them spread across several divisions at Berkeley.

One approach is peer evaluation of teaching. They create a rubric and have past winners of department teaching prizes observe classes to gather information on teachers.

Another is to do a review of the materials that professors use to create classes.

"Show me your stuff," Stark says. "Syllabi, handouts, exams, video recordings of class, samples of students' work. Let me know how your students do when they graduate. That seems like a much more holistic appraisal than simply asking students what they think."

Goldengate8361 · ‎01-27-2014

As a former professor, I agree with that article 100%! Student evaluations are a joke....a bad one.

Deb1010again · ‎05-28-2013

good article. My DH is an art prof. I agree that some sort of measurement of how well the student does in subsequent classes should be incorporated into overall evaluations somehow.

THANKS for sharing the article!

Puzzle Piece · ‎06-29-2010

Girlfriend of mine teaches over at CSLB. There are on-line evalutions of instructors by students at the site. She always gets low ratings based on her looks and if they don't like the amount of work or homework that is required. It's not a fair asessment of the knowledge dispersed. Many of the students aren't even college material.

Never Forget the Native American Indian Holocaust

sfnative · ‎03-10-2010

@Goldengate8361 wrote:
As a former professor, I agree with that article 100%! Student evaluations are a joke....a bad one.

@Goldengate8361

I agree to a certain extent.

1. We need to remember that this Italian culture is different from ours. This is a factor which must be taken into consideration.

2. Peer Evaluation: There are dozens ways that could be manipulated. Just let your imagine run wild.

Student evals were a constant, before I made a turn to a clinical career. Though not at the high school at which I taught, this occurred during my 3rd year in college. Students were given a "strong" eval form for each professor. For 8 years running, the professor I had for year 3 of Music History was voted the worst professor on campus. I cannot begin to tell you the horror stories of having to attend his class. "Think" the most benign thing being spittle: no one ever sat in the first two rows and on from there.

pitdakota · ‎03-16-2010

As an asst. professor I agree 100%. They are a joke, plain & simple.

* Freedom has a taste the protected will never know *

skuggles · ‎11-20-2013

There maybe a tendency for students to give poor evaluations to more difficult professors, who may in fact be better than the "easier" ones, especially when compared to the performance of the student in future classes for which they were being prepared by this instructor. However, I think there has to be some value in them, and they should not be canned. I agree that they should fine tune the handling of the results, or modify the evaluations so that the information gleaned can be more particular.