README.md

metadata

title: Research Paper Pitfall Checker
emoji: 🔍📄
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: agpl-3.0

Research Paper Pitfall Checker

A Gradio application that analyzes research papers to identify potential evaluation pitfalls using AI-powered analysis.

Features

• PDF Text Extraction: Extracts text content from uploaded research papers
• AI-Powered Analysis: Uses OpenRouter API with Qwen model for intelligent pitfall detection
• Comprehensive Pitfall Detection: Identifies common evaluation pitfalls including:
  • 🔒 The Lock-In Effect
  • 🍎🍊 Apples-to-Oranges Comparisons
  • 💧 Contamination Leak
  • 🤖❓ Unvalidated Automation
  • 🧐 Vague Scales

How to Use

1. Get an API Key: Sign up at OpenRouter.ai to get your API key
2. Set Environment Variable: Set your API key as an environment variable
3. Upload a PDF: Upload your research paper PDF file
4. Analyze: Click "Analyze Paper for Pitfalls" to get a detailed analysis
5. Review Results: Review the analysis report for potential issues and improvement suggestions

Setup

1. Install dependencies:

   pip install -r requirements.in
   

2. Set your OpenRouter API key as an environment variable:

   export OPENROUTER_API_KEY="your-api-key-here"
   

   Or create a .env file (copy from env.example):

   OPENROUTER_API_KEY=your-api-key-here
   

3. Run the application:

   python app.py
   

Configuration

The pitfalls are defined in pitfalls.json. You can modify this file to add or remove specific pitfalls to check for.

API Requirements

• OpenRouter API key (get one from OpenRouter.ai)
• Uses the qwen/qwen2.5-next-80b-a3b-instruct model via OpenRouter
• API key must be set as the OPENROUTER_API_KEY environment variable

License

AGPL-3.0