File size: 1,969 Bytes
34228a4
9e7cbd8
 
 
 
34228a4
 
 
 
7183f70
34228a4
 
9e7cbd8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
---
title: Research Paper Pitfall Checker
emoji: πŸ”πŸ“„
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: agpl-3.0
---

# Research Paper Pitfall Checker

A Gradio application that analyzes research papers to identify potential evaluation pitfalls using AI-powered analysis.

## Features

- **PDF Text Extraction**: Extracts text content from uploaded research papers
- **AI-Powered Analysis**: Uses OpenRouter API with Qwen model for intelligent pitfall detection
- **Comprehensive Pitfall Detection**: Identifies common evaluation pitfalls including:
  - πŸ”’ The Lock-In Effect
  - 🍎🍊 Apples-to-Oranges Comparisons  
  - πŸ’§ Contamination Leak
  - πŸ€–β“ Unvalidated Automation
  - 🧐 Vague Scales

## How to Use

1. **Get an API Key**: Sign up at [OpenRouter.ai](https://openrouter.ai) to get your API key
2. **Set Environment Variable**: Set your API key as an environment variable
3. **Upload a PDF**: Upload your research paper PDF file
4. **Analyze**: Click "Analyze Paper for Pitfalls" to get a detailed analysis
5. **Review Results**: Review the analysis report for potential issues and improvement suggestions

## Setup

1. Install dependencies:
   ```bash
   pip install -r requirements.in
   ```

2. Set your OpenRouter API key as an environment variable:
   ```bash
   export OPENROUTER_API_KEY="your-api-key-here"
   ```
   
   Or create a `.env` file (copy from `env.example`):
   ```
   OPENROUTER_API_KEY=your-api-key-here
   ```

3. Run the application:
   ```bash
   python app.py
   ```

## Configuration

The pitfalls are defined in `pitfalls.json`. You can modify this file to add or remove specific pitfalls to check for.

## API Requirements

- OpenRouter API key (get one from [OpenRouter.ai](https://openrouter.ai))
- Uses the `qwen/qwen2.5-next-80b-a3b-instruct` model via OpenRouter
- API key must be set as the `OPENROUTER_API_KEY` environment variable

## License

AGPL-3.0