docs/HEURISTIC_VS_LLM_ANALYSIS.md

🤖 AI Analysis: LLM vs Heuristic

What is "Heuristic Analysis"?

Heuristic = Analysis based on rules and simple calculations, NOT using the AI language model.

Two Types of AI Analysis:

┌─────────────────────────────────────────────────┐
│          AI TACTICAL ANALYSIS                   │
├─────────────────────────────────────────────────┤
│                                                 │
│  🤖 LLM Analysis (Preferred)                    │
│  ├─ Uses: Qwen2.5-Coder-1.5B model            │
│  ├─ Time: ~15 seconds                          │
│  ├─ Quality: ⭐⭐⭐⭐⭐ Natural language      │
│  ├─ Context: Understands game situation        │
│  └─ Example: "Enemy is massing tanks near      │
│              north. Build anti-tank defenses    │
│              and prepare counter-attack."       │
│                                                 │
│  📊 Heuristic Analysis (Fallback)              │
│  ├─ Uses: Simple IF-THEN rules                │
│  ├─ Time: Instant (0.001 seconds)             │
│  ├─ Quality: ⭐⭐⭐ Generic but reliable       │
│  ├─ Context: Counts units/buildings            │
│  └─ Example: "Battlefield is balanced.         │
│              Scout and take map control."      │
│                                                 │
└─────────────────────────────────────────────────┘

📊 How Heuristic Analysis Works

Algorithm (Simple Rules):

def _heuristic_analysis(game_state):
    # 1. COUNT everything
    player_units = count_units(player=0)
    enemy_units = count_units(player=1)
    player_buildings = count_buildings(player=0)
    enemy_buildings = count_buildings(player=1)
    credits = player_credits
    power = player_power
    
    # 2. CALCULATE advantage score
    score = (player_units - enemy_units) + 
            0.5 × (player_buildings - enemy_buildings)
    
    # 3. DETERMINE situation
    if score > 1:
        situation = "AHEAD"  # Winning
    elif score < -1:
        situation = "BEHIND"  # Losing
    else:
        situation = "EVEN"  # Balanced
    
    # 4. SELECT pre-written messages
    if situation == "AHEAD":
        summary = "You hold the initiative. Maintain pressure."
        tips = ["Expand territory", "Scout flanks", "Pressure enemy"]
    
    elif situation == "BEHIND":
        summary = "You're under pressure. Stabilize defenses."
        tips = ["Build defenses", "Protect economy", "Hold position"]
    
    else:  # EVEN
        summary = "Battlefield is balanced. Scout and control map."
        tips = ["Scout enemy", "Build units", "Control resources"]
    
    # 5. ADD specific tips based on resources
    if power < power_consumption:
        tips.add("Build Power Plant")
    
    if credits < 300:
        tips.add("Protect Harvester")
    
    if player_units < enemy_units:
        tips.add("Train more Infantry and Tanks")
    
    # 6. RETURN generic coaching
    coach = "Keep economy safe and strike when you see opening."
    
    return {
        'summary': summary,
        'tips': tips[:4],  # Max 4 tips
        'coach': coach,
        'source': 'heuristic'  # ← Important flag!
    }

🔄 When is Heuristic Used?

3 Scenarios:

1. Model Not Loaded Yet ⏳

Game just started → Model still loading
→ Use heuristic until model ready
→ Switch to LLM after ~15 seconds

2. LLM Busy Processing 🔄

LLM processing NL command → Queue full
→ Return heuristic INSTANTLY
→ User gets immediate feedback
→ LLM analysis appears later when ready

3. LLM Error/Timeout ❌

LLM request failed or timed out
→ Fallback to heuristic (always works!)
→ Better generic advice than nothing

📝 Example Output Comparison

LLM Analysis (15s, Natural):

{
  "summary": "Your tank forces are positioned well but enemy has air superiority. Their helicopters are harassing your harvesters in the northwest quadrant. Consider building SAM sites near your refineries and advancing your tank column while they're distracted.",
  
  "tips": [
    "Build 2-3 SAM sites near western refineries",
    "Push tank column toward enemy war factory",
    "Keep infantry with tanks for anti-air support",
    "Scout their helicopter production rate"
  ],
  
  "coach": "You have material advantage but must counter their air harassment - protect economy and push when helicopters retreat.",
  
  "source": "llm"
}

Characteristics:

• ✅ Specific to current situation
• ✅ Mentions exact threats (helicopters, northwest)
• ✅ Strategic reasoning (push while distracted)
• ✅ Natural, flowing language
• ⏱️ Takes 15 seconds

Heuristic Analysis (Instant, Generic):

{
  "summary": "Battlefield is balanced. Scout and take map control.",
  
  "tips": [
    "Train Infantry and add Tanks for frontline",
    "Scout enemy base and pressure weak flanks",
    "Protect Harvester and secure more ore",
    "Build a Power Plant to restore production speed"
  ],
  
  "coach": "Keep your economy safe and strike when you see an opening.",
  
  "source": "heuristic"
}

Characteristics:

• ✅ Always valid advice
• ✅ Covers basic strategy
• ❌ Not situation-specific
• ❌ Template-based language
• ⚡ Instant (0.001s)

🎯 Quality Comparison

Feature	LLM Analysis	Heuristic Analysis
Speed	15 seconds	Instant
Accuracy	Context-aware	Generic but correct
Language	Natural, flowing	Template-based
Specificity	Exact threats/opportunities	General principles
Reliability	95% (can timeout)	100% (never fails)
CPU Usage	High	Negligible
Memory	1GB model	~1KB rules

💡 Why Both Are Useful

LLM (Qwen2.5-Coder):

• Best for: Detailed tactical advice
• Provides: Specific, context-aware guidance
• Example: "Enemy massing tanks at north bridge - build anti-tank defenses there"

Heuristic:

• Best for: Instant feedback, basic strategy
• Provides: Reliable, always-available advice
• Example: "Build more units and scout enemy position"

Combined Strategy:

User plays game
↓
Every 60s: AI analysis triggered
↓
Is LLM available and not busy?
├─ YES → Use LLM (15s wait, great advice)
└─ NO → Use Heuristic (instant, good advice)
↓
User sees analysis immediately (heuristic)
or after 15s (LLM)

🔍 How to Tell Which One You Got

In the UI:

The analysis object has a source field:

// Frontend can check:
if (analysis.source === 'heuristic') {
  console.log('📊 Using rule-based analysis (instant)');
} else if (analysis.source === 'llm') {
  console.log('🤖 Using AI model analysis (15s)');
}

Visual Indicators (Could Add):

LLM Analysis:

🤖 AI Tactical Analysis
━━━━━━━━━━━━━━━━━━━━
Your tank forces are positioned well...

Heuristic Analysis:

📊 Quick Tactical Overview
━━━━━━━━━━━━━━━━━━━━
Battlefield is balanced. Scout and...

📈 Performance in Production

On HuggingFace (2 vCPU):

Scenario 1: Normal Operation

Game running smoothly
→ LLM available
→ User gets LLM analysis every 60s
→ High quality, natural language advice

Scenario 2: User Spamming Commands

User: "move tanks"
User: "build infantry"  
User: "attack base"
→ LLM queue full (3 requests pending)
→ AI analysis → Use Heuristic (instant!)
→ User still gets advice while LLM busy
→ Later LLM analysis appears when queue clears

Scenario 3: Model Timeout

LLM taking >15s (CPU contention)
→ Request timeout
→ Fallback to Heuristic
→ User gets SOME advice (not nothing!)
→ Next analysis tries LLM again

🎮 User Experience

Player Perspective:

What players see:

• Tactical advice panel updates every 60 seconds
• Sometimes advice is very specific ("Build SAM sites in northwest")
• Sometimes advice is more general ("Scout and control map")
• Advice is ALWAYS useful and valid
• Never waits more than a moment for advice

What players DON'T see:

• Whether LLM or heuristic was used (both look professional)
• Model loading times (heuristic covers gap)
• Queue management (seamless fallback)
• CPU limitations (instant response via heuristic)

Best of Both Worlds:

RELIABILITY (Heuristic) + QUALITY (LLM) = Great UX

• Heuristic ensures advice is ALWAYS available
• LLM makes advice INTERESTING and SPECIFIC when available
• Fallback strategy means no frustration

🔧 Technical Details

Code Flow:

# ai_analysis.py - summarize_combat_situation()

def summarize_combat_situation(game_state, language):
    # 1. Quick check: Is model loaded?
    if not self.model_available:
        return self._heuristic_analysis(game_state, language)
    
    # 2. Try LLM with timeout
    result = self.generate_response(
        prompt=build_prompt(game_state),
        timeout=15.0
    )
    
    # 3. LLM success?
    if result['status'] == 'ok':
        return parse_llm_response(result)  # 🤖 LLM analysis
    
    # 4. LLM failed/busy → Instant fallback
    else:
        return self._heuristic_analysis(game_state, language)  # 📊 Heuristic

Heuristic Implementation:

The heuristic analysis is multi-language and covers:

1. Situation Assessment (ahead/even/behind)
2. Power Management (low power warning)
3. Economy (protect harvesters if low credits)
4. Army Composition (unit balance advice)
5. Defense (fortify if building disadvantage)
6. Generic Coaching (motivational message)

All in English, French, and Chinese! 🌍

🎯 Summary

What is Heuristic Analysis?

Simple rule-based advice that's instant and always works.

When is it used?

• Model not loaded yet
• LLM busy with other requests
• LLM timeout or error

Why is it good?

• Instant (no waiting)
• Reliable (never fails)
• Valid (always correct advice)
• Fallback (ensures users always get help)

Why have LLM too?

• Better quality (context-aware)
• Natural language (more engaging)
• Specific advice (adapts to exact situation)
• Strategic depth (understands complex scenarios)

---

Think of it like:

• Heuristic = Experienced player giving quick tips
• LLM = Professional coach analyzing replay in detail

Both are valuable! Heuristic ensures you always get advice (speed + reliability), while LLM makes the advice amazing when available (quality + depth). 🎯