Monday, January 9, 2012

How Republican and Democratic Professors Grade

Talia Bar and Asaf Zussman article explore "Partisan Grading" in the most recent issue (vol. 4, number 1) of the American Economic Journal: Applied Economics. The  article is not freely available on-line, although many can get on-line access to it if your library has a subscription to the publications of the American Economic Association. As they explain in their abstract: "We study grading outcomes associated with professors in an elite university in the United States who were identified—using voter registration records from the county where the university is located—as either Republicans or Democrats. The evidence suggests that student grades are linked to the political orientation of professors."

Their useful starting point is that they have data on SAT scores for students--which are on average a good predictor of college grades. Measured by this standard, they show that the distribution of students headed into the classrooms of Republican and Democrat professors is essentially the same. However, the grading outcomes are not the same.

They have data from "the College of Arts and Sciences of an elite university in the United States between the spring semester of 2000 and the spring semester of 2004." More specifically, they have grades of 17,062 students taking 3,277 undergraduate level courses with 417 professors. They find:

"[T]he variance of grades is higher in courses taught by Republicans than in courses taught by Democrats. Moreover, in additional analysis, we find that relative to their Democratic colleagues, Republican professors tend to assign more very low and very high grades. The share of the lowest grades (F, D−, D, D+, and C−) out of the total is 6.2 percent in courses taught by Republican professors and only 4.0 percent in courses taught by Democratic professors. The share of the highest grade (A+) out of the total is 8.0 percent in courses taught by Republican professors and only 3.5 percent in courses taught by Democratic professors. Both differences are highly statistically significant."

This general  pattern holds up after adjusting for differences in grading across academic departments. Here's a graphical illustration of the same general theme. The horizontal axis shows SAT scored of students; the vertical axis shows the average grade received by students. Notice that students with low SAT scores--say, under 1200--receive an average grade of about 2.4 in classes taught by Republican professors, but 2.9 in classes taught by Democrat professors. At the higher end of the range, students with about 1400 on their SATs up to about 1600 get roughly the same grades on average from Democrat professors, at about 3.4. However, students with SAT scores in the top 1560-1600 range on average get grades of about 3.6 from Republican professors.

The authors are at pains to emphasize that there are multiple possible interpretations of these findings.

"One interpretation is that it reflects a difference in grading practices but not in student performance. In other words, an identical distribution of student performance will translate into different distributions of grades, with Republican professors tending more than their Democratic colleagues to assign low grades to low-ability students and high grades  to high-ability students.

"An alternative way to interpret the finding is that it reflects a difference in student performance but not in grading practices. The difference in student performance could be related to the amount of effort professors are willing to invest in helping students of different abilities or in the extent to which professors encourage students of different abilities. For example, it is possible that Democratic professors would devote more resources (e.g., in office hours time) to helping low-ability students, while Republican professors would devote more resources to nurturing high-ability students. It may also be the case that Republicans have different teaching or testing styles than Democrats (for example, with different needs for memorization or creativity), and that student performance varies across these heterogeneous learning environments. An additional possibility is that Democrats differentially reward something other than pre-existing talent.

"These interpretations are all consistent with our hypothesis. ... The important point from our perspective is that the evidence suggests that Republican professors are associated with less egalitarian grading outcomes."

 While Barr and Zussman are professionally cautious in their interpretation, readers are of course free  to offer their own interpretations. Let the parade of anecdote, speculation, innuendo and overstatement commence!