self-review

researchlingzhi2271.5K installs

Automatically review an academic paper using the NeurIPS review form with three reviewer personas, ensemble scoring, and reflection refinement. Extracts text from PDF, runs structured review, and outputs actionable feedback. Use when the user wants to review a paper before submission or get feedback on a draft.

Install

npx skills add https://github.com/lingzhi227/agent-research-skills --skill self-review

SKILL.md

---name: self-reviewdescription: Automatically review an academic paper using the NeurIPS review form with three reviewer personas, ensemble scoring, and reflection refinement. Extracts text from PDF, runs structured review, and outputs actionable feedback. Use when the user wants to review a paper before submission or get feedback on a draft.argument-hint: [pdf-or-tex-file]---

Self-Review

Review an academic paper using a structured review form with multiple reviewer personas.

Input

$ARGUMENTS — Path to PDF file or .tex file

Scripts

Extract text from PDF

python ~/.claude/skills/self-review/scripts/extract_pdf_text.py paper.pdf --output paper_text.txt
python ~/.claude/skills/self-review/scripts/extract_pdf_text.py paper.pdf --format markdown

Tries pymupdf4llm (best) → pymupdf → pypdf. Install: pip install pymupdf4llm pymupdf pypdf

Parse PDF into structured sections

python ~/.claude/skills/self-review/scripts/parse_pdf_sections.py \
  --pdf paper.pdf --output sections.json

Extracts title (via font size), section headings, and section text. Requires: pip install pymupdf Key flags: --format text, --verbose

Workflow

Step 1: Load Paper

If PDF: use extract_pdf_text.py to extract text
If .tex: read the LaTeX source directly

Step 2: Three-Persona Review

Run three independent reviews using different personas (from references/review-form.md):

Harsh but fair reviewer: Expects good experiments that lead to insights
Harsh and critical reviewer: Looking for impactful ideas in the field
Open-minded reviewer: Looking for novel ideas not proposed before

For each persona, generate a review following the NeurIPS review JSON format in references/review-form.md.

Step 3: Reflection Refinement (up to 3 rounds per reviewer)

After each review, apply the reflection prompt: re-evaluate accuracy and soundness, refine if needed. Stop when "I am done".

Step 4: Aggregate

Combine all three reviews
Average numerical scores (round to nearest integer)
Synthesize a meta-review finding consensus
Weight scores using AgentLaboratory weights: Overall (1.0), Contribution (0.4), Presentation (0.2), others (0.1 each)

Step 5: Actionable Report

Output format:

## Review Summary
- **Overall Score**: X/10 (Weighted: Y/10)
- **Decision**: Accept / Reject
- **Confidence**: Z/5

## Strengths (consensus across reviewers)
1. ...
2. ...

## Weaknesses (consensus across reviewers)
1. ...
2. ...

## Questions for Authors
1. ...

## Specific Suggestions for Improvement
1. [Section X, Page Y]: ...
2. [Section Z, Page W]: ...

## Score Breakdown
| Dimension | R1 | R2 | R3 | Avg |
|-----------|----|----|-----|-----|
| Overall | ... | ... | ... | ... |
| Contribution | ... | ... | ... | ... |
| ... | ... | ... | ... | ... |

References

NeurIPS review form, scoring weights, personas, reflection prompts: ~/.claude/skills/self-review/references/review-form.md
PDF text extraction: ~/.claude/skills/self-review/scripts/extract_pdf_text.py

Missing Sections Check

You MUST verify that all required sections are present: Abstract, Introduction, Methods/Approach, Experiments/Results, Discussion/Conclusion. Reduce scores if any are missing.

Related Skills

Upstream: paper-compilation
Downstream: paper-revision, rebuttal-writing
See also: slide-generation

Related skills

researchmattpocock575KInvestigate a question against high-trust primary sources and capture the findings as a Markdown file in the repo. Use when the user wants a topic researched, docs or API facts gathered, or reading legwork delegated to a background agent.paper-context-resolverlllllllama451KRigor Paper Context helper for README-first deep learning repo reproduction. Use only when the README and repository files leave a narrow reproduction-critical gap and the task is to resolve a specific paper detail such as dataset split, preprocessing, evaluation protocol, checkpoint mapping, or runtime assumption from primary paper sources while recording conflicts. Do not use for general paper summary, repo scanning, environment setup, command execution, title-only paper lookup, or replacing R env-and-assets-bootstraplllllllama450KRigor Setup skill for README-first deep learning repo reproduction. Use when the task is specifically to prepare a conservative conda-first environment, checkpoint and dataset path assumptions, cache location hints, and setup notes before any run on a README-documented repository. Do not use for repo scanning, full orchestration, paper interpretation, final run reporting, or generic environment setup that is not tied to a specific reproduction target.ai-research-explorelllllllama311KRigor Explore compatible skill slug for meaningful and potentially novel deep learning research candidates. Use when the researcher has chosen the task family, dataset, benchmark, evaluation method, provided SOTA references, and wants candidate-only exploration on top of `current_research` with auditable repo understanding, idea gating, fair comparison, and governed experiments written to `explore_outputs/`. Do not use for README-first trusted reproduction, open-ended direction finding, narrow c