« Back to Summary
Do you think the ICML2010 reviews were different in quality from the reviews at previous ICMLs?
#Response DateAdditional comments:
1Jun 10, 2010 7:07 PMWasn't in previous ICMLs
2Jun 10, 2010 7:21 PMNot having a response to the final reviews was a terrible choice, as our final reviewer did not know what they were talking about.
3Jun 10, 2010 8:13 PMFor this question as well as for the previous one, I have no idea, because I have no way to compare, I only saw the review I made (I was reviewer, not author)
4Jun 10, 2010 8:36 PMI don't have enough experience to answer this with much confidence.
5Jun 10, 2010 9:06 PMQuality of phase 1 reviews was embarrassingly low.
6Jun 10, 2010 10:01 PMIn the papers I reviewed there was impressive agreement among reviewers.
7Jun 10, 2010 11:17 PMI am not certain about my claim. I do not remember previous reviews accurate enough for any meaningful comparison.
8Jun 11, 2010 8:37 AMAbout double blind: At some point, somebody has to keep track of people submitting the same thing several times. My 1st round reviews were simply awful (for my single author paper), I was seriously shocked. I wrote a harsh reply, and was then positively surprised about the good handling of the area chair (good 2nd round review; of course, the 1st round reviewers didn't give a damn: the main function of the reply is to get the area chair thinking!). I as reviewer was very thorough, which got me branded as somebody with "vested interests". I'll not do it next year for ICML. IMHO: With such poor reviewers today, it is just more work for area chairs. Hire more of them, keep track of who's doing good area chair work, pass it between conferences. There is no simple way around this, look at other serious disciplines, they do the same. And make sure that both authors and reviewers understand that area chairs know who they are.
9Jun 11, 2010 1:26 PMSince I have not participated in past ICMLs as author *or* reviewer, I cannot comment on this.
10Jun 12, 2010 11:01 PMDon't know.
11Jun 17, 2010 7:07 PMquality assurance have to be improved, i.e., reviewer should recheck proofs and related work. to ensure this, reviewers could be divided into two groups specialized these issues (e.g. phd students and professors, respectively). furthermore, reviewers are mentioned in the proceedings related to reviewed papers (only accepted).
12Jun 20, 2010 11:33 AMI don't know about previous ICMLs
13Jun 25, 2010 6:44 AMThere was rather high variability in the quality and depth of the reviews, and in my experience, more than the usual share of low quality, superficial reviews.