---
title: Skin Lesion Assistant - Academical tech demo
emoji: 🌍
colorFrom: red
colorTo: red
sdk: gradio
sdk_version: 5.37.0
app_file: app.py
pinned: false

---
## 🧠 Skin Lesion 2.0 — RAG + LLM Explainer

Educational demo that predicts a skin-lesion class from an image and explains the result with cited medical context using RAG + a local LLM. 

⚠️ Not medical advice. Prototype for learning only.

## 💡 Features

- Dual-input CNN (image + metadata) predicts one of 7 HAM10000 classes
- Top-1 label + confidence shown to user
- RAG: retrieves relevant chunks from a small curated knowledge base
- Local LLM (Qwen 2.5–3B Instruct) generates a plain-language explanation with citations
- Simple two-step Gradio UI: Predict → Ask a question (e.g., treatments, risks, follow-up)

## 🚀 Try it

👉 [Live demo on Hugging Face Spaces](https://huggingface.co/spaces/vcabreragreco/skin_lesion_assistant)


## 🛠️ Tech Stack

- TensorFlow/Keras – dual-input CNN (image + tabular metadata)
- sentence-transformers (BioBERT) – text embeddings
- FAISS – fast vector similarity search
- transformers + Qwen/Qwen2.5-3B-Instruct – local LLM for explanations
- Gradio

## 🧩 How it works (pipeline)

- Preprocess: image → (8×8 RGB) tensor; metadata (age/sex/site/dx_type) → scaled/one-hot.
- Predict: CNN outputs class probabilities → top-1 label + confidence.
- Retrieve: build a query from label + question → embed (BioBERT) → FAISS top-k chunks (label-boosted).
- Generate: system prompt + numbered chunks → LLM writes an explanation and cites [1] [2] ….

## 📁 Files

- app.py — Gradio app (predict + RAG explain)
- rgb_8_model.keras — trained Keras model
- age_scaler.joblib, meta_features_list.joblib — preprocessing artifacts
- kb_faiss_index.idx — FAISS index of knowledge base
- kb_chunks_metadata.json — source chunks with tags/sections
- requirements.txt — dependencies

## ⚠️ Limitations

- Not diagnostic. Confidence is not calibrated.
- Small input resolution (8×8) → lower accuracy on fine-grained classes.
- Domain shift: non-HAM images may underperform.

## ❤️ Built by Victoria

© 2025 Victoria Cabrera Greco 
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference