Goals as reviewer

Your goals on review should be:

• help editor/panel make good decision
• help authors write a good paper

Titus's approach to reviewing

1. Skim-read the grant or paper; highlight areas that don't make sense.

2. Decide if you are pro-, anti-, or on the fence.

3. If pro, try to write a review that objectively states why. Pay special attention to framing in a larger context, if you can.

4. If anti, identify major flaws clearly and objectively. Behave as if the authors know who you are and where you live.

5. If on the fence, look harder for support one way or another; if you can't find, explain that you're on the fence, and why.

6. I structure my reviews like so:

   Broad overview: why this is important, or not.

   Big-picture problems ("they failed to convince me of X, Y or Z")

   Nits to pick (bad figures; spelling; grammar; etc.)

Note that generally editors want recommendations separate from your text, so that they don't get into arguments with authors about things like "but 2/3 reviewers said accept!"

It's easy to find BS reasons to reject papers; try to find good reasons instead.

Things that I hate to see in grants that will lead me to be grumpy (says Titus):

• misspellings
• bad grammar
• bad science, defined as "stuff that is clearly wrong/badly done", not just stuff I disagree with

Do say when you don't know something/aren't an expert. You will be helping the editor!

Things you should try doing:

• posting the top part of your (positive) review as a blog; not the nits, though.

  see: http://ivory.idyll.org/blog/2013-assemblathon-review-i.html, http://ivory.idyll.org/blog/2013-review-howison.html, and http://ivory.idyll.org/blog/2013-review-troubling-trends.html for three different examples.

  Posting your review can lead to many great things.

  Happy to guest post reviews on my blog.

• posting your papers to arxiv or biorxiv

• signing your reviews

I have posted other peoples' reviews for one of my papers here: http://ivory.idyll.org/blog/khmer-counting-reviews.html

Grants

I post many of my grants on my Web site:

http://ged.msu.edu/research.html

I passed out reviews for three grants; I cannot make the reviews public, but here are the grants.

2011-NSF-CAREER: http://ged.msu.edu/downloads/2011-nsf-career.pdf

• my takeaway: "Good proposal, bad PI, not enough preliminary data"

2012-NSF-CAREER: http://ged.msu.edu/downloads/2012-career-nsf-final.pdf

• my takeaway: "OK proposal, OK PI, but too much preliminary data"

2012-NIH-BIGDATA: http://ged.msu.edu/downloads/2012-bigdata-nsf.pdf

• my takeaway: "Please fund this person."

Note, 2012 grants were written at same time, but got very different reviews. Reviews depend heavily on reviewer!

Entertaining #SixWordPeerReview tweets from Rich:

http://storify.com/ctitusbrown/rich-lenski-and-sixwordpeerreview/

These are absolutely not how you should write reviews. But they are funny.

Tracy's links:

Env. Micro. referee comments:

http://onlinelibrary.wiley.com/doi/10.1111/j.1462-2920.2011.02660.x/full

Some examples:

"It is sad to see so much enthusiasm and effort go into analyzing a dataset that is just not big enough."

"You call the sample fresh water, this is confusing as it is saline water."

"The biggest problem with this manuscript, which has nearly sucked the will to live out of me, is the terrible writing style."

Again, try to avoid this kind of thing in real reviews :)

My blog post on why current peer review is broken: http://ivory.idyll.org/blog/angenmap-discussion-on-peer-review.html

Another recent blog post that I found interesting:

http://svpow.com/2014/01/20/how-to-become-good-at-peer-review-three-points-of-disagreement/

Questions? Comments? Ask!

--titus