How to Write a Useful Scientific Paper Review on Paperstars

Why Reviews Matter

At Paperstars, reviews aren’t just opinions — they’re contributions to a better scientific record. By offering structured, thoughtful feedback, you’re helping others evaluate the quality of a paper based on more than just where it was published or how often it’s cited.

Every review strengthens the platform, promotes accountability, and makes science more transparent for everyone.

✅ What Makes a Good Review?

A useful review on Paperstars is:

• Specific – Focuses on particular elements of the paper (methods, analysis, data, etc.)
• Constructive – Points out both strengths and weaknesses without being dismissive
• Structured – Uses the review prompts to guide your thinking
• Respectful – Even critical reviews can be kind and professional
• Transparent – Reflects your honest assessment, supported by clear reasoning

🧩 Follow the Structured Prompts

Paperstars reviews are guided by seven criteria, each contributing to the final score:

1. Title – Is it appropriate, or does it overstate the findings?
2. Methods – Are they robust, well-described, and suited to the question?
3. Statistical Analysis – Were the right tests used? Any signs of p-hacking?
4. Data Presentation – Are the figures clear and accurately interpreted?
5. Discussion – Are the findings placed in proper context?
6. Limitations – Are the limitations acknowledged?
7. Data Availability – Are the data and code openly accessible?

You’ll score each area using our rubric, and the weighted results will generate a final 1–5 star rating. You can then adjust the final score manually if needed.

💬 How to Write the Written Review

After rating each component, you’ll be asked to write a short review (minimum word count applies to keep things meaningful).

Here’s a simple structure you can follow:

Mention key strengths.

“The methods are robust and clearly described. The preregistration adds credibility.”

Identify areas for improvement.

“The statistical analysis could be more transparent, particularly in how exclusions were handled.”

Conclude with a fair assessment.

“Overall, a valuable contribution, but could be strengthened with more transparent data reporting.”

🚩 What to Avoid

• ❌ Vague reviews: “Interesting paper. Could be better.”
• ❌ Personal attacks or hostile language
• ❌ Overly short, low-effort responses
• ❌ Revealing your identity or others’ in the review

🛠️ Why It’s Worth It

Your review will help:

• Other researchers quickly assess paper quality
• Encourage open science practices (we highlight transparency and data availability!)
• Give authors meaningful feedback they can build on
• Elevate a culture of substance over status

💡 Final Tip

You don’t need to be a PI or have a long publication list to leave a valuable review.

If you’ve read the paper carefully, understood the methods, and have something to say — you belong here.

On Paperstars, your insight matters.