Revolutionizing scientific research through automated paper generation, comprehensive literature review, and intelligent peer evaluation powered by advanced language models.
Advanced research automation system powered by state-of-the-art language models.
Comprehensive review system implementing multi-stage evaluation framework.
Experience how easy it is to generate groundbreaking research with just a few lines of code.
# Import necessary libraries from ai_researcher import CycleResearcher from ai_researcher.utils import print_paper_summary # Initialize CycleResearcher with default 12B model researcher = CycleResearcher(model_size="12B") from pprint import pprint # Load references from BibTeX file with open('cycleresearcher_references.bib', 'r') as f: references_content = f.read() pprint(references_content.split('@')[:3]) # Generate paper with specific references referenced_paper = researcher.generate_paper( topic="AI Researcher", references=references_content, n=10 ) # Print summary of generated papers for paper in referenced_paper: print_paper_summary(paper)
CycleResearcher employs an iterative preference optimization framework that includes policy model for paper generation, reward model for evaluation, and iterative SimPO for reinforcement learning.
Paper Type | Source | Avg Min Score | Avg Max Score | Avg Score | Accept Rate |
---|---|---|---|---|---|
Conference Accept Papers | Human Expert | 3.91 | 6.98 | 5.69 | 100.00% |
Preprint Papers | Human Expert | 3.24 | 6.62 | 5.24 | 29.63% |
AI Scientist | AI | 2.20 | 5.70 | 4.31 | 0.00% |
CycleResearcher-12B | AI | 3.47 | 6.75 | 5.36 | 35.13% |
CycleResearcher-14B | AI | 3.56 | 6.71 | 5.41 | 37.74% |
CycleResearcher-72B | AI | 3.65 | 6.58 | 5.38 | 33.64% |
CycleResearcher-123B | AI | 3.30 | 6.45 | 5.15 | 24.28% |
CycleReviewer employs a comprehensive evaluation framework to assess research papers, focusing on key metrics including proxy MSE, MAE metrics, decision accuracy, and comparative performance against human experts.
Method | MSE (n-1) | MAE (n-1) | MSE (n) | MAE (n) | Accuracy | Macro F1 |
---|---|---|---|---|---|---|
Expert Individual | 2.34 | 1.16 | - | - | 75.40% | 75.39 |
GPT-4o-mini | 3.44 | 1.53 | 2.98 | 1.40 | 53.06% | 34.72 |
GLM-4 | 4.45 | 1.81 | 3.91 | 1.70 | 49.49% | 33.10 |
Gemini-1.5-pro | 3.02 | 1.34 | 2.56 | 1.23 | 50.98% | 50.75 |
CycleReviewer (123B) | 1.43 | 0.92 | 1.25 | 0.87 | 74.24% | 73.99 |
DeepReviewer implements a structured multi-stage review framework with novelty assessment, multi-dimensional evaluation, and reliability verification to ensure comprehensive paper evaluation.
Model | Constructive | Analytical | Plausibility | Technical | Overall |
---|---|---|---|---|---|
Win Rate vs GPT-o1 | 89.80% | 87.67% | 51.69% | 25.12% | 88.21% |
Win Rate vs Claude-3.5-Sonnet | 96.88% | 97.92% | 80.21% | 77.08% | 95.74% |
Win Rate vs DeepSeek-V3 | 96.04% | 99.01% | 72.28% | 67.33% | 96.22% |
Win Rate vs DeepReviewer-70B | 98.33% | 98.89% | 92.78% | 79.44% | 98.33% |
Win Rate vs DeepSeek-R1 | 89.22% | 74.51% | 45.10% | 26.47% | 80.20% |
We have prepared comprehensive tutorials for both CycleResearcher and CycleReviewer to help users better understand and utilize these models. Our tutorials cover everything you need to get started and make the most of our model suite.
Learn the basics of CycleResearcher and how to set up your first automated research project.
Tutorial 1: CycleResearcherLearn how to leverage AI for comprehensive paper evaluation.
Tutorial 2: CycleReviewerExploring the Integration of Human Thought Processes and Cutting-edge Professional Paper Retrieval Capabilities into LLMs.
Tutorial 3: DeepReviewer