« Back to Summary
What model do you prefer for assigning reviewers to papers? (multiple selections possible)
#Response DateI have the following alternative proposal:
1Jun 10, 2010 8:38 PMReviewers submit sample papers, key words and citations are extracted, and the system does smart matching. But the second phase of manually assigning some reviewers works well.
2Jun 10, 2010 10:42 PMIt is important to be clear on whether first phase is only cursory or whether full reviews are expected.
3Jun 10, 2010 10:46 PMCombination of 'C' and 'D' where the papers are assigned based on the bids, but the area chair can overrule the matching. (AC's should be encouraged not to do it very often.)
4Jun 10, 2010 10:47 PMI don't like the two-phase system, because it means most accept/reject decisions are made without author feedback which at least places some limits on reviewing quality. I believe we should start with a single phase model, where one reviewer is determined by area chair, one by reviewer bidding, and one by either or some other system. Diversity seems important here. Then, instead of having a highly staged author feedback, we should simply have an anonymized email service between authors and reviewers, with an option for cutoff if abuse should occur. An area chair overseeing the process can bring in an additional reviewer as needed. I also think we should differentiate the reviewers. One reviewer should be tasked with checking correctness (only), while others can assess significance.
5Jun 11, 2010 12:59 PMFirst review round with three reviewers assigned through bidding. Second round with manually assigned reviewers only for those papers that could not clearly be accepted or rejected.
6Jun 11, 2010 1:01 PMMixing bidding and manual assignment but not in the sequence as this year. After the system has assigned papers to reviewers based on bidding, area chairs correct the assignment where they consider this necessary.