| 1 | Jun 10, 2010 7:25 PM | I'm against Phase II, so none of the above options apply. |
| 2 | Jun 10, 2010 7:33 PM | After Phase I, but do give Phase II reviewers access to the feedback immediately after they have submitted their review |
| 3 | Jun 10, 2010 7:48 PM | I see no reason that authors cannot particpate in an
interactive discussion with the reviewers while preserving two-way anonymity. Of course the discussion period should be limited. Right now authors - who put in the maximum effort on the paper and have most to lose - have the least say on the matter. Their response is read or not read and acted upon or not. In most cases, the reviewers are not willing to change their stance. If at all, they only change negatively and they have nothing to lose. The role of the reviewer should not be one-sided - find excuses to reject - but should also be to strongly support papers that they think should be accepted. This is rarely practiced as there are always some weaknesses, and it is harder to defend a paper than to support it. Allowing authors to participate in the discussion would even out the playing field a little more, although the reviewers would still have the final say.
Area chair cannot do this job, since he would not have
read the paper and has responsibility for many papers.
-Prasad Tadepalli |
| 4 | Jun 10, 2010 8:23 PM | I prefer author feedback after both phase I and phase II. Since these are typically factual clarifications, the authors do not need to be given a week to respond (2 days or even one day should be good enough provided these dates are announced well in advance and strictly adhred to). Early abstract submission (like in ECML) can help in saving some more time. |
| 5 | Jun 10, 2010 8:26 PM | Only have one phase, with feedback afterwards. |
| 6 | Jun 10, 2010 10:02 PM | Ask only those authors for feedback for whom you have actual questions. |
| 7 | Jun 10, 2010 11:27 PM | I am not a big fan of two-phase reviewing process the way implemented this year: the number of papers assigned to each reviewer was not decreased because the area chairs were not willing to reject papers in the first phase. In Phase II, I received a ridiculous paper with two rejects in Phase II and it seemed nobody cared about rejecting it in the first phase. Other papers were not much better either.
One idea is two have 3(2) reviews for the first phase, and let authors provide their feedback. If the area chair thinks the paper is marginal, then ask for addition 2(3) reviewers to read the paper AND let them access to the Phase I reviews while reading the paper and before submitting their reviews. This may sound crazy as this is totally biased evaluation. The benefit, however, is that it allows two or three more researchers to see the paper in the light of already mentioned criticism and carefully evaluate those judgments. This may lead to deeper and more focused evaluation of the paper. |
| 8 | Jun 10, 2010 11:37 PM | After Phase I in a model that doe not include Phase II. |
| 9 | Jun 10, 2010 11:46 PM | For practical reasons, I prefer this year's model. A natural issue with the model is that Phase II reviewers may introduce new arguments that the authors will not be able to counter. Therefore, perhaps it makes sense to down-weight the Phase II reviews somewhat. |
| 10 | Jun 11, 2010 1:38 AM | Do not ask for author feedback at all.
But if you do, I feel it *must* be after Phase II. My impression was that the final review was the one that "decided" and it is silly to not be able to respond to that review, if it is indeed the one that matters. |
| 11 | Jun 11, 2010 2:13 AM | I think if you are going to have a rebuttal phase, it needs to be for all reviews. I really did not like the fact that this year, the rebuttal was only for first-round reviews. This caused problems in some of the discussions, in terms of weighing the different reviews. This is a major factor why I would favor the single phase process, or have no rebuttal, or else have the rebuttal after Phase 2 (but PLEASE don't ask for feedback twice, or after only some of the reviews!). |
| 12 | Jun 11, 2010 8:39 AM | Author feedback: The main role of this is not to reach reviewers,
I've honestly never seen this happen. I don't complain about good
reviews, and bad reviewers by def. don't even read the response.
The importance is that the area chair sees the response. |
| 13 | Jun 13, 2010 1:56 PM | I think last year's model was great. I also think we should "pressure" authors to answer clearly to a few questions singled out by reviewers. I had a few lengthy author responses which avoided carefully answering my most important questions. i think that's a waste of time. |
| 14 | Jun 14, 2010 2:09 PM | Only ask for author feedback in cases where it could change the reviewer's decision (i.e., not for all papers). Otherwise don't waste everyone's time. |
| 15 | Jun 14, 2010 7:37 PM | The first option, however, to have a possibility to communicate with authors in a more flexible way, could be in some situations quite interesting. |
| 16 | Jun 16, 2010 7:43 AM | If you are going to have author feedback it would be good if reviewers were forced to respond to it (and the authors could read this reponse). This at least makes one feel it has been read. |
| 17 | Jun 25, 2010 6:48 AM | In my experience both as a reviewer and an author, I rarely found it to be the case that reviewers were willing to change their rating based on author feedback . More often than not, it appeared that the reviewers did not read the author feedback (or for that matter, the reviews other than their own). |