Reto Gubelmann

A paper that has been on a reviewing journey for close to one year has been outrightly rejected from EMNLP, even though it had an average rating of 3.17, which should usually be findings.

Is this because we filed a meta-review issue report? There, we referred to reviews of modest quality, a complete absence of interaction during the rebuttal period, and primarily a meta-reviewer who took a very minimalistic approach; their only action consisted in writing a rather unspecific meta-review (which made anonymization pretty easy):

The senior Area Chair rewieving our report partly agreed with our objections, but also thought that the paper was in fact only borderline conference ready, downgrading the quality of the paper from the previous assessment by the reviews and the meta-review. This probably lead to the rejection even for the findings track of EMNLP.

Typically, one would take such an event as a reason to submit to a workshop. Ironically, my judgment is that this is one of the better papers that I have contributed to, and others have been accepted to ACL, EMNLP, Coling, JOLLI, etc. Therefore, we might try again at a major conference.

1. To begin with, the review by the Senior Area Chair does indeed contain some helpful suggestions that we will try to implement, so we are grateful for that.
2. As a matter of fact, I would probably proceed in the same way again. I still think that the reviews and the meta-review in particular merit an issue report. This is not how the reviewing process for a major conference should be like.
3. At this point in time, when submitting to ACL Rolling Review (ARR), it is good to prepare for all kinds of surprises. My personal impression is that the number of competent and committed reviewers and the number of submissions to be reviewed are out of balance, leading to a decrease in quality of reviews and sometimes to pretty surprising outcomes.
4. More generally speaking, I think this offers a nice opportunity to let a basic fact of academic existence (and probably of existence in general) sink in: Don’t measure your success by outcomes you can at best partly control; focus on process goals that you can control. As with many important insights, they have to be lived through rather than merely memorized (see Jim Murphy’s Inner Excellence on this).

PS: A scene that illustrates how to focus on process rather than outcome near-perfectly for me is this one from Le Mans 66: Ken Miles wins Le Mans, but, due to a personal feud, a Ford executive ensures that he does not get credit for it. After a moment of shock, Miles takes the salute from Ferrari and continues to do what really makes him feel alive: Working on and improving the car together with Shelby, letting the results be as they may.