WARP.md

WARP.md

This file provides guidance to WARP (warp.dev) when working with code in this repository.

Project Overview

Wisal QA is an advanced AI assistant specialized in Autism Spectrum Disorders (ASD), developed by Compumacy AI. The system uses sophisticated confidence scoring and relevance filtering to provide accurate, autism-focused responses while recognizing comorbid conditions and indirect relationships.

Key Features

• Multi-tiered relevance assessment (0-100% confidence scoring)
• Comorbidity recognition (depression, anxiety, ADHD, sleep issues in autism context)
• Smart query rewriting with autism-specific context
• Multi-source information combining (web search, RAG, LLM generation)
• Real-time voice chat with transcription and TTS
• Document upload and Q&A functionality

Development Commands

Environment Setup

# Install dependencies
pip install -r requirements.txt

# Set up environment variables (copy from .env.example if available)
cp .env.example .env
# Edit .env with your API keys:
# - SILICONFLOW_API_KEY
# - GEMINI_API_KEY  
# - WEAVIATE_API_KEY
# - QDRANT_API_KEY
# - TAVILY_API_KEY

Running the Application

# Main application (Gradio interface)
python main.py

# Alternative runner with pre-flight checks
python run_main.py

# Test functional components
python test.py

Development and Testing

# Run single test file
python -m pytest test.py -v

# Test specific functionality
python -c "from utils import process_query; print(process_query('What is autism?'))"

# Test query processing pipeline
python -c "from query_utils import process_query_for_rewrite; print(process_query_for_rewrite('My child has depression'))"

# Functional RAG pipeline testing
python -c "from src.pipeline import create_rag_pipeline; pipeline = create_rag_pipeline(); print(pipeline('What is autism?', ['Autism is a developmental disorder...']))"

Configuration Management

# View current configuration
cat config.yaml
cat src/config.yaml

# Test configuration loading
python -c "from src.config import load_config; print(load_config())"

Logging and Debugging

# View recent logs
ls -la logs/
tail -f logs/log_*.txt

# Check logger functionality
python -c "from logger.custom_logger import CustomLoggerTracker; logger = CustomLoggerTracker().get_logger('test'); logger.info('Test message')"

Architecture Overview

High-Level System Design

The codebase follows a multi-paradigm architecture combining:

1. Functional Programming (src/ directory) - Pure functions, immutable data structures
2. Traditional OOP (root directory) - Gradio interface, handlers, utilities

Core Components

1. Confidence Scoring Pipeline (query_utils.py)

• Enhanced relevance checking: 6-tier confidence system (0-100%)
• Smart rewriting: Automatically frames questions in autism context
• Comorbidity awareness: Recognizes depression, anxiety, ADHD as autism-relevant

Key Functions:

• enhanced_autism_relevance_check() - Main confidence scoring
• process_query_for_rewrite() - Complete query processing pipeline
• rewrite_query_for_autism() - Context-aware query rewriting

2. Multi-Source Processing Pipeline

The system combines three information sources:

• Web Search (web_search.py) - Real-time autism information via Tavily API
• RAG Systems (rag.py, rag_domain_know_doc.py) - Domain knowledge retrieval
• LLM Generation (utils.py) - Direct autism expertise via SiliconFlow/Qwen

3. Functional RAG Architecture (src/ directory)

Modern functional programming approach with:

• Immutable data structures (@dataclass(frozen=True))
• Pure functions with consistent interfaces
• Composable pipeline (src/pipeline.py)
• Model factories (src/models.py) for API/local model abstraction

4. Quality Control Layer

Multi-stage validation:

• Pre-processing: Query relevance filtering
• Post-processing: Answer autism-relevance checking
• Hallucination detection: 5-point accuracy scoring
• Translation support: Auto-detect and translate responses

Configuration Architecture

Dual Configuration System:

• config.yaml (root) - Application-level settings, API keys
• src/config.yaml - Functional pipeline configuration (models, chunking, performance)

Model Support:

• API Models: SiliconFlow, Gemini, Weaviate, Qdrant
• Local Models: Hugging Face transformers, sentence-transformers
• Configurable switching between API and local inference

Key Architectural Patterns

1. Confidence-Driven Processing

# Query processing follows confidence scoring
if confidence_score >= 70:
    process_directly()
elif confidence_score >= 25:
    rewrite_for_autism_context()
else:
    politely_reject()

2. Functional Composition

# Pipeline composition in src/
pipeline = compose(
    chunk_documents,
    embed_chunks, 
    retrieve_similar_chunks,
    rerank_documents,
    generate_answer
)

3. Multi-Modal Integration

• Text Input/Output: Primary interface
• Voice Input: Gemini transcription via WebRTC
• Voice Output: Gemini TTS with multiple voice options
• Document Processing: PDF, DOCX, TXT support

Important Implementation Details

Confidence Scoring Thresholds

DIRECT_AUTISM_THRESHOLD = 85      # Accept as-is
HIGH_RELEVANCE_THRESHOLD = 70     # Accept as-is  
SIGNIFICANT_THRESHOLD = 55        # Rewrite for autism
MODERATE_THRESHOLD = 40           # Rewrite for autism
SOMEWHAT_THRESHOLD = 25           # Conditional rewrite
REJECTION_THRESHOLD = 24          # Reject

Comorbidity Recognition Logic

The system specifically boosts scores for:

• Depression in children/teens: +15 points (65-75% final score)
• Anxiety disorders: +15 points
• ADHD symptoms: +15 points
• Sleep disorders: +15 points
• Sensory processing: +20 points

Model Configuration Patterns

# API vs Local model switching
if model_config.type == ModelType.API:
    return create_api_model(config)
else:
    return create_local_model(config)

Error Handling Strategy

• Graceful degradation: Fallback to simpler models/methods
• Comprehensive logging: All failures logged with context
• User-friendly messages: Technical errors translated to helpful responses

Development Guidelines

Working with Confidence Scoring

• Test edge cases: Borderline queries (scores 25-75)
• Validate comorbidity detection: Depression/anxiety in autism context
• Monitor false positives/negatives: Use logging to track decision quality

Adding New Features

1. Functional approach: Add to src/ directory for pipeline components
2. Integration: Use existing confidence scoring for relevance checking
3. Logging: Integrate with CustomLoggerTracker for consistency
4. Configuration: Add settings to appropriate config.yaml

Model Integration

• API models: Add to model factory in src/models.py
• Local models: Ensure HuggingFace compatibility
• Configuration: Update model configs in src/config.yaml

Testing Autism Relevance

# Test confidence scoring
from query_utils import enhanced_autism_relevance_check
result = enhanced_autism_relevance_check("My teenager seems depressed")
# Expected: score=65, action="rewrite_for_autism"

Audio/Voice Features

• WebRTC integration: Real-time voice chat via fastrtc
• Gemini STT/TTS: Voice input/output processing
• VAD (Voice Activity Detection): Automatic speech detection

Common Development Patterns

Adding a New Information Source

1. Create async function in dedicated module
2. Integrate with reranking system in utils.py
3. Add to multi-source processing pipeline
4. Update confidence thresholds if needed

Extending Comorbidity Recognition

1. Update confidence scoring prompts in prompt_template.py
2. Add condition-specific scoring logic in query_utils.py
3. Test with representative queries
4. Update documentation with new thresholds

Document Processing Workflow

1. Upload via Gradio interface
2. Route to appropriate handler (old_docs.py, rag_domain_know_doc.py, user_specific_documents.py)
3. Chunk and embed using functional pipeline
4. Store in vector database (Weaviate/Qdrant)
5. Integrate with RAG retrieval

This codebase represents a sophisticated autism-focused AI system with strong architectural foundations for both traditional and functional programming paradigms. The confidence scoring system and comorbidity awareness are key differentiators that should be preserved and extended carefully.