Spaces:
Running
Running
File size: 3,562 Bytes
bad84a3 509e21e bad84a3 509e21e bad84a3 509e21e c417f2d 509e21e c417f2d 509e21e c417f2d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
---
title: AI Evaluation Dashboard
emoji: 📊
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
app_port: 3000
---
# AI Evaluation Dashboard
This repository is a Next.js application for viewing and authoring AI evaluations. It provides a comprehensive platform for documenting and sharing AI system evaluations across multiple dimensions including capabilities and risks.
## Project Goals
The AI Evaluation Dashboard aims to:
- **Standardize AI evaluation reporting** across different AI systems and models
- **Facilitate transparency** by providing detailed evaluation cards for AI systems
- **Enable comparative analysis** of AI capabilities and risks
- **Support research and policy** by consolidating evaluation data in an accessible format
- **Promote responsible AI development** through comprehensive risk assessment
## For External Collaborators
### Making Changes to Evaluation Categories and Schema
All evaluation categories, form fields, and data structures are centrally managed in the `schema/` folder. **This is the primary location for making structural changes to the evaluation framework.**
Key schema files:
- **`schema/evaluation-schema.json`** - Defines all evaluation categories (capabilities and risks)
- **`schema/output-schema.json`** - Defines the complete data structure for evaluation outputs
- **`schema/system-info-schema.json`** - Defines form field options for system information
- **`schema/category-details.json`** - Contains detailed descriptions and criteria for each category
- **`schema/form-hints.json`** - Provides help text and guidance for form fields
### Standards and Frameworks Used
The evaluation framework is based on established standards:
- **Risk categories** are derived from **NIST AI 600-1** (AI Risk Management Framework)
- **Capability categories** are based on the **OECD AI Classification Framework**
This ensures consistency with international AI governance standards and facilitates interoperability with other evaluation systems.
### Contributing Evaluation Data
Evaluation data files are stored in `public/evaluations/` as JSON files. Each file represents a complete evaluation of an AI system and must conform to the schema defined in `schema/output-schema.json`.
To add a new evaluation:
1. Create a new JSON file in `public/evaluations/`
2. Follow the structure defined in `schema/output-schema.json`
3. Ensure all required fields are populated
4. Validate against the schema before submission
### Development Setup
## Run locally
Install dependencies and run the dev server:
```bash
npm ci
npm run dev
```
Build for production and run:
```bash
npm ci
npm run build
NODE_ENV=production PORT=3000 npm run start
```
## Docker (recommended for Hugging Face Spaces)
A `Dockerfile` is included for deploying this app as a dynamic service on Hugging Face Spaces (Docker runtime).
Build the image locally:
```bash
docker build -t ai-eval-dashboard .
```
Run the container (expose port 3000):
```bash
docker run -p 3000:3000 -e HF_TOKEN="$HF_TOKEN" ai-eval-dashboard
```
Visit `http://localhost:3000` to verify.
### Deploy to Hugging Face Spaces
1. Create a new Space at https://huggingface.co/new-space and choose **Docker** as the runtime.
2. Push this repository to the Space Git (or upload files through the UI). The Space will build the Docker image using the included `Dockerfile` and serve your app on port 3000.
Notes:
- If your build needs native dependencies (e.g. `sharp`), the Docker image may require extra apt packages; update the Dockerfile accordingly. |