« Back to Summary
Do you think the ICML2010 reviews were different in quality from the reviews at previous ICMLs?
#Response DateAdditional comments:
1Jun 10, 2010 7:07 PMWasn't in previous ICMLs
2Jun 10, 2010 7:21 PMNot having a response to the final reviews was a terrible choice, as our final reviewer did not know what they were talking about.
3Jun 10, 2010 8:25 PMBut much better than ICML 2009, which was a disaster in my view.
4Jun 10, 2010 9:06 PMQuality of phase 1 reviews was embarrassingly low.
5Jun 10, 2010 9:26 PMThe trend of ICML is to playing with mathematical notation and following some "hot topics".
6Jun 10, 2010 10:51 PMI've often found ICML2010 reviewers incoherent----making claims that just don't compute. That was true this year also.
7Jun 11, 2010 12:05 AMI do not think some of the area chairs/reviewer are qualified
8Jun 11, 2010 2:30 AMThe main issue seems to be the lack of reviewers with expertise.
9Jun 11, 2010 3:03 AMI thought there was a lot of poor matching -- the process is too complicated, and it depends on area chairs guessing good keywords, and having a pool of reviewers that is "right" for the set of papers they actually get -- which might or might not happen (and often didn't, especially for ACs with broad interests). I also thought that the Phase 2 reviews were no better than the Phase 1 reviews, and the shortened timeline caused by the two-phase reviewing just made it less likely that people would have the time and attention to really do a good job.
10Jun 11, 2010 6:21 AMCan't comment. First submission.
11Jun 11, 2010 8:09 AMIt was not possible to rebut comments from Phase II reviewers. In my case, there were three additional reviews in Phase II, some of which made significant factual errors that could not be addressed.
12Jun 11, 2010 8:20 AMno previous submission...
13Jun 11, 2010 1:03 PMCan't tell, but quality varied a lot!
14Jun 11, 2010 1:19 PMFirst submission to ICML, I can't answer this question.
15Jun 11, 2010 1:20 PMI did not submit to previous ICML
16Jun 11, 2010 1:47 PMwhat improved the reviews was withdrawing a poor review in phase 1 and replacing it by a very competent third review in phase 2
17Jun 11, 2010 1:55 PMThis is my first ICML paper.
18Jun 11, 2010 1:57 PMno experience with previous ICMLs; first time submitter
19Jun 11, 2010 2:03 PMI feel our rejected paper was rejected because of a single poor reviewer who obviously hadn't put in the time to understanding the method. The area chair should have caught this.
20Jun 11, 2010 2:23 PMI can not compare with previous one, as I did not submit
21Jun 11, 2010 2:25 PMThe quality of ICML reviewers should be improved. Trivially wrong reviews appear in my paper.
22Jun 11, 2010 2:31 PMDon't know.
23Jun 11, 2010 2:58 PMdidn't submit to icml2009
24Jun 11, 2010 3:38 PMAbove answer closest to: "I don't know about ahy previous year's ICML".
25Jun 11, 2010 4:19 PMI had a problem with the process on our particular papr: only 2 reviews were given during phase I. Both were positive, leading us to believe that the paper would get in. We responded to the comments, which again were pretty mild. In phase II, though, a third review was added which was more negative, but to which wew were not given the chance to respond. We felt that that was an unfair process. I spoke to another author of a rejected paper who said that his experience was similar and which he also judged to be unfair.
26Jun 11, 2010 5:05 PMAbout the reviewing model: there is evidence that double-blind does remove bias, but my personal preference would be to know the name of reviewers without associating each reviewer to each review.
27Jun 13, 2010 1:15 AMCan't say. This is my first ICML.
28Jun 13, 2010 5:26 AMI didnt submit in the past
29Jun 13, 2010 9:27 AMNo experience
30Jun 13, 2010 12:01 PMcannot say, not much experience
31Jun 13, 2010 5:27 PMOne of our reviewers changed his or her review after our rebuttal. Why? Perhaps this was at the direction of the area chair. If the opinion of a paper changes during phase 2, fine, but the original phase-1 reviews shouldn't get edited to reflect the new thinking about the paper.
32Jun 14, 2010 7:30 PMno idead
33Jun 15, 2010 9:25 AMDo not know
34Jun 16, 2010 8:25 AMDoes not apply for me.
35Jun 19, 2010 5:18 AMIn this year, the quality of many reviews are rather poor. Many reviewers are not loyal to the research itself, while come to a conclusion only based on personal preferences.
36Jun 25, 2010 6:44 AMThere was rather high variability in the quality and depth of the reviews, and in my experience, more than the usual share of low quality, superficial reviews.
37Jun 25, 2010 8:30 AMthis is the first time I send a paper to ICML