More on how the PGR is toast

I would genuinely like to know how the Philosophical Gourmet Report evaluators were selected, how many were asked, what percentage they are of the entire philosophy faculty, how representative they are of the faculty overall, and how many have declined to participate this time given all the negative publicity. But I don’t expect much information.  And many others are seeing this too. Another reason to think that the PGR is toast.

Lots more info here.

Documenting the meltdown on Leiter and bad tactics in rankings

If you are a philosopher in the English speaking world, you no doubt know that the old self-appointed emperor has lost his clothes. As of this writing, more than 520 philosophers (including the original signatories at the top) have signed a statement that they will decline to support his Philosophical Gourmet Report so long as he’s running it.  Twenty-four members of his board have asked him to relinquish management. Since I’ve been one of the characters in this tale, I’ve been keeping up with all the talk in the philosophy blogosphere.  For those interested in what’s going on, Leigh M Johnson has been keeping track here.  Also Richard Heck is starting to collect accounts and analyses of what is wrong with the methodology of rankings in general and the Philosophical Gourmet Report in particular on his blog here.

Sign on to the September Statement

The list of philosophers unwilling to take part in the Philosophical Gourmet Report so long as Brian Leiter is editing it keeps growing: https://sites.google.com/site/septemberstatement/ .  Any philosophy professor with an academic appointment is invited to join the list.  You need not be someone who would have been likely to be an evaluator.

If you wish to add your name to those declining for these reasons to volunteer their services to the PGR while under the control of Brian Leiter, please email septemberstatement@gmail.com with your name and affiliation.

Please use your verifiable university email account to avoid confusion.

Is the PGR sexist?

Just to round out my current round of complaints about the rankings of the Philosophical Gourmet Report (and then I really will finish those article revisions!), I want to point out another way in which bias shows up. The “top” 25 programs overall have smaller percentages of tenure-stream women faculty than even the already-dismal percentage in doctoral programs throughout the profession. Only eight of those 25 do better than average.

Julie Van Camp estimates that the national average is 22.7 percent. I’m rounding up, so let’s say that anything above 23% is better than average.

Below is the list. For all the programs listed in the APA’s guide to graduate programs, I’ve used their self-reported numbers.  For those who did not submit their data to the APA (shame, shame, shame), I’ve used Julie Van Camp’s numbers.  The latter I list here with an asterisk. I’ll  italicize those better than average.

  1. NYU 21%
  2. Rutgers 21%
  3. Princeton 17%
  4. Michigan 26%
  5. *Harvard 21%
  6. Pittsburgh 22%
  7. MIT 18%
  8. *Yale 16%
  9. *Stanford 25%
  10. *UNC 17%
  11. *Columbia 35%
  12. UCLA 25%
  13. USC 20%
  14. CUNY 18%
  15. Cornell 33%
  16. Arizona 23%
  17. UC-Berkeley 27%
  18. Notre Dame 16%
  19. *Brown 27%
  20. *Chicage 20%
  21. UT-Austin 17%
  22. UCSD 17%
  23. UW-Madison 22%
  24. Duke 20%
  25. Indiana 22%

Of the next 26 that made the top 51, 16 are better than average.

So, what to make of this?  Is the problem that these “top” schools are not that interested in hiring more women?  Or is it that they are deemed “top” because they are not hiring more women—and hence not doing the kind of “non-philosophical” work those women tend to do? Linda Alcoff writes that the PGR “works to reward convention and punish departments that take the risk of supporting an area of scholarship that is not (yet) widely accepted or respected in the profession. Hiring in the areas of critical race philosophy or feminist philosophy is not going to improve a department’s ranking. As a result, philosophy departments are trying to outdo themselves in conformism and ‘tailism’—tailing the mediocre mainstream rather than leading.”

Additionally, could the fact that 85% of the evaluators were men have anything to do with the problem? There’s not a lot of use in speculating, since the report never pretends to be objective. It is a reputational ranking based on views of those who have, in certain circles, a good reputation.  So it is all circular.

And now I worry that it is also sexist. I am not saying that the group of evaluators are themselves sexist but rather that unconscious biases are bound to slip in to a survey that is shoddily constructed.

Can this survey be saved?  No, dear colleagues, it is time we walked away. I urge anyone who has been involved in this exercise, whether by turning over your list of faculties or serving on the board or as an evaluator, to stop.

PGR participation…

For the 2009 Philosophical Gourmet Report ranking of US doctoral programs, Brian Leiter circulated a list of the faculty at 99 US programs. But for the 2011-12 rankings, the list was of only 60 programs.  That’s a 39% drop, in the space of just two years, of departments willing to participate. No wonder Leiter has not published the list in the usual spot under methodology.  But it can be retrieved as an rtf document from this page.  [Edit: see correction below in my comment replying to Leiter.]

[Nonetheless] I compared the list of 60 faculties [used for the Philosophical Gourmet Report’s 2011 rankings] to Julie Van Camp’s ranking of departments by their percentage of tenure-stream women faculty. From top to bottom of these women-friendly departments (in terms of having above average percentage of women faculty), here is a list of those that do not participate in the PGR rankings:

  • University of Georgia
  • University of Oregon
  • Emory University
  • Villanova University
  • SUNY-Albany
  • University of New Mexico
  • University of South Carolina
  • Arizona State
  • SUNY Binghamton
  • University of Oklahoma
  • Loyola University – Chicago
  • SUNY Stony Brook
  • University of Cincinnati
  • University of Kansas
  • DePaul University
  • Fordham University
  • Marquette University
  • Temple University
  • University of Memphis
  • Duquesne University
  • University of Kentucky
  • Michigan State University

Bravo to all these programs — both for hiring women to the tenure stream and for saying no to the PGR.

[Edit: For background see yesterday’s post on the PGR’s un-women-friendly epistemology.]

The PGR’s un-women-friendly epistemology

Julie Van Camp just updated her Spring 2004 article, “Female-Friendly Departments: A Modest Proposal for Picking Graduate Programs in Philosophy” that pointed out the under-representation of women on the advisory board of Brian Leiter’s Philosophical Gourmet Report. This month Van Camp expanded the postscript with numbers showing that in the past ten years little has changed.

Postscript: November 20, 2004 [updated 2/3/2014]

The 2011 Report:
The list of the Top 51 doctoral programs is included in the 2011 Philosophical Gourmet Report. The 56 members of the  Report’s Advisory Board for 2011 included nine females (16.1%) and was based on the reports of 302 evaluators, including 46 women (15.2%).

The 2009 Report:
The 55 members of the  Report’s Advisory Board for 2009 included eight females (14.5%) and was based on the reports of 294 evaluators, including 37 women (12.6%).

The 2006-08 Report:
The 56 members of the Report’s Advisory Board for 2006-2008 included seven females (12.5%)  and was based on the reports of 269 evaluators, including 26 women (9.67%).

The 2004-06 Report:
The 59 members of the Report’s Advisory Board for 2004-2006 included eight females (13.6%) and was based on the reports of 266 evaluators, including 32 women (12.0%).

The 2002-04 Report:
The 43 members of the Report’s Advisory Board for 2002-2004 included five women (11.6%) and was based on reports from 177 evaluators, including 24 women (13.6%).

Van Camp also notes that the very “top” six programs in the PGR have  a lower percentage of women on the faculty than the national average for doctoral-granting programs. Go HERE to see her helpful chart showing percentages of tenured and tenure-track women faculty in doctoral-granting programs.

On that chart she includes when and how a school was ranked on the PGR since 2002.  Of the top ten on her list, six have no ranking——meaning they have not shown up (since 2002) as one of the PGR’s top 51 programs . That can happen in two ways: (1) the program was ranked at 52d or worse or (2) the program did not turn over its list of faculty, meaning, it chose not to participate at all.

The 2009 PGR was based on a list of faculty from 99 doctoral programs.  How many were on the 2011 list?  Leiter provides previous lists under methodology, but not the 2011 list, at least not as of this writing. I know anecdotally that many of the programs with more women on the faculty choose not to turn over their lists to Leiter.  I think this is because of his explicit bias against self-identified pluralist programs, most of which tend to have more women on the faculty. Regarding some problems with this bias,  see this post on see  on the New APPS blog.

Is there a systematic bias in the PGR methodology that leads it to value more male-dominated departments?  Well, yes.  An unrepresentative and hand-picked advisory board plus unrepresentative and hand-picked evaluators will lead to a slanted take on the value of the work going on in the profession. You don’t have to be a stand-point epistemologist to see this.

 

The favorites’ favorites — another round of PGR rankings of continental philosophy

I see that Brian Leiter has posted a preview of the five 20th century continental programs that his reviewers like best, certainly all fine programs: Columbia University; Georgetown University; University of California, Riverside; University of Chicago; and University of Notre Dame. I wasn’t surprised by the absence of “spep-ish” departments, as the bleiterites are wont to put it, for it is rare that a “spep-ish” continental philosopher serves as an evaluator. (Never mind that the big tent called the Society for Phenomenology and Existentialist Philosophy is the second largest philosophical society in North America, next to the American Philosophical Association, and, I would hazard, the largest continental philosophy society in the world. So to call a program with strengths in continental philosophy “spep-ish” is like calling any program in philosophy “apa-ish”  —  it’s practically trivial.)

(Also, I am not at all surprised by the omission of the Emory University program for we simply do not participate in the rankings.)

I have long argued that the fatal flaw of these reports is that the evaluators do not represent a cross-section of the field.  So, to try to make this point a bit more pointedly, take a look at the names of the 24 evaluators for 20th Century Continental Philosophy programs:

James Bohman, Steven Crowell, Maudemarie Clark, David Dudrick, Gordon Finlayson, Max de Gaynesford, Charles Guignon, Gary Gutting, Beatrice Han-Pile, Scott Jenkins, Pierre Keller, Michelle Kosch, Brian Leiter, Dean Moyar, Stephen Mulhall, Brian O’Connor, Peter Poellner, Bernard Reginster, Michael Rosen, Joseph Schear, Iain Thomson, Georgia Warnke, Mark Wrathall, Julian Young.

This is a great group, including many I personally know and admire.  But let me explain how it does not at all represent a cross-section of philosophers doing work in 20th Century Continental Philosophy.  I took a couple hours this evening to consult the websites and phil papers sites, etc. of members of this group, and made notes of what areas they worked in — in their own words.

Only three-quarters specialize in any area of 20th Century Continental Philosophy. (Unless I am mistaken, Maudemarie Clark, Max de Gaynesford, Scott Jenkins, Michelle Kosch, Brian Leiter, and Dean Moyar have specialties elsewhere, but not here.)

There is a solid group doing work in existentialism, phenomenology, and critical theory, but only four of the 24 specialize in post-1968 French philosophy.  Of those four, only two of the 24 evaluators (Stephen Crowell and Charles Guignon) profess to have any expertise on any of the major thinkers of French poststructuralism after Foucault.

Nietzsche scholars were very well represented (nine of 24), including many who have been published by or with the author of the reports.

So for students interested in the full range of important work in 20th Century Continental Philosophy, especially work post-1968, I encourage a trip to the library, not to the PGR.