Peer-Review as (Self)-Pedagogy

I’ve not read any of the listserv discussion.  I’ve only seen the fall out on twitter.  Here’s a bit of my own life experience that might be of interest to some. 

Minor addition 5/22/18: Some people seem to be reading and even liking this post.  Thank you.  I am also hearing the well worn argument that any affirmative action will undermine decisions based on merit.  As I said in the first version of this paper, the verdict is already in on ‘color blindness’ (it’s racist) and we can extrapolate lessons regarding ‘blind’ peer review from this work.  Similarly, we can extrapolate lessons from the case for affirmative action.  These are well worn and well understood.   See below for suggested links.

In May 2017 I organized with students, former students, and mentees an international interdisciplinary conference called The Cost of Freedom: Debt and Slavery.

In retrospect the most educational part of organization process for me was the peer review process for the abstracts:

The committee consisted of

  • female tenured classicist at Large Public University (me!), European-American
  • male Classics PhD candidate about to defend, European-American
  • female early stage PhD Candidate in Comp Lit at an Ivy League University, Caribbean-American
  • male early stage PhD Candidate in American Studies (trained in Sociology) at a Large Public University, male, Dominican-American
  • female undergraduate in Philosophy (aiming at Law School), African-American

We also represented a wide range of different religious heritages and degrees of present practice.   We started with a blind approach to the review of 60+ abstracts.  All abstracts were read by all committee members.  Only the undergraduate coordinator knew the identities.  Separately we each answered a relatively simple pre-determined questionaire in a spreadsheet including a numerical score and open-ended comments.

At our meeting our first conversation was about the experience of reading and reviewing, where we felt our own fatigue, interests, tastes, stress, scholarly backgrounds coming into play.

Then we constructed a whole program based on our blind scoring.  We worked from the assumption that if even one person on the committee gave the highest possible score to a paper, the paper should be included.  We discussed at length those with great discrepancies in their scoring.  We did not average the scores.

Still in that same meeting we then read the biographical statements submitted separately by each applicant.  My junior collaborators were shocked and horrified (I only a little less so) that the panels we’d constructed were heavily favored towards white male speakers and we’d not managed to attract applicants–let alone select–ANY from a Haitian background, something we felt essential given the theme of our conference.  Many asked, “if this is what WE come back with in our hard-fought attempts at inclusivity, how is there any hope for those from non-traditional backgrounds to succeed in the academy with its belief in blind peer review?!”

We then had a really hard conversation about what we wanted our conference to be about, what work we wanted to support, and foreground.  What made something academically worthy?  Then we read all the bios and abstracts together again.

We decided on a longer, more grueling schedule, to ensure the conference included more scholars from more diverse backgrounds.  We in fact doubled our intended number of speakers.  We re-thought our panel themes to make sure we could showcase different approaches and we went out and actively recruit scholars who hadn’t applied to fill in the intellectual gaps, even providing translations services and offering funding to allow such participation.

What was our biggest take-away?  Regardless of our background, we still found ourselves inclined to defer to the voices of white men.  Why?  I suspect because even in blind peer review those voices sound so much like the familiar voice of authority.   They have been trained to speak with authority and we have been trained to recognize that sound as the sound of authority.  These are speculations, based on a single experience, not tested hypotheses.

Does this threaten the position of white men?  Some, but far from all.  In the second round of review and the reformulation of the program, we in fact found ourselves adding in one senior white male scholar who in the end gave an incredibly moving paper at the conference.   Many thoughtful white male scholars consciously adopt an inclusive open style, preferring themselves to use a voice that may sound less self assured and less authoritative, because it leaves space for others.  In doing so, they engage in richer more rewarding conversations.

One of the things that scares me most is my own tendency to coach mentees to sound more authoritative, more rigorous (more like some one who was acculturated to white male privilege).  To walk the walk, to talk the talk, to fake it till you make it.  To pass.  Enacting privilege has certainly been my own chief survival strategy. 

Blind peer review is not the sole means for achieving ‘fair’ allocation of speaking slots at conferences.  It’s problems are closely tied to the well-critiqued problems of ‘color blindness’ (see below for refs).

We need to understand on a very personal level what sounds like ‘good scholarship’ to our own ears and think about why it sounds so.

Can we be more open in our ideas of what has merits?

Can we save space for ideas and voices that we might not yet readily identify as ‘rigorous’ or ‘disciplined’ or ‘focused’ or ‘trained’?

What’s the worst that can happen?

We hear a bad paper or three?

Doesn’t this happen anyway?!

We hear an idea that makes us angry or makes us question our own assumptions?

Great!  You’d get energized to write more and better!  To think harder.  Isn’t that why we got into the business?

We don’t get to give our own paper?

Yes, you might have to give up a place now and again.  I strongly suspect I was not selected from a recent panel so that a graduate student could speak.  Do I think my paper would have been more ‘relevant’ more ‘informed’ more ‘on topic’ more ‘polished’?  Yes.  Do I fault the committee? not one iota!

This self-examination can be exhausting.  But it is worth it, not just for others, but for myself.

Essays on why ‘color-blindness’ is a problem.

A 2013 editorial from the Washington Post

2011 Psychology Today article

2013 Historical Essay on the idea

Essays on why affirmative action isn’t a bad idea added 22 May 2018

2014 Washington Post article

Critique of 7 common criticisms of Affirmative Action

2017 US News and World Report article

Feeling like you don’t know how to talk about race, but you want to?

Checkout anything by Ijeoma Oluo

She’s on twitter too.

(Please don’t ask your students or friends of color to do the extra work of teaching you personally, or of assuring you you’re getting it right! We all make mistakes and need to be gracious when corrected for them.  Myself included.  I wish I could take back so many things I said in my early career.)

1 thought on “Peer-Review as (Self)-Pedagogy”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s