windows_cuda_fix.md

# Windows CUDA Linking Issues - Troubleshooting Guide

## Issues Identified

### 1. Fixed: torch.cuda.amp Deprecation Warnings ✅
- **Issue**: `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd` deprecation warnings
- **Fix**: Updated `llava/model/qlinear_te.py` to use `device_type='cuda'` parameter
- **Lines changed**: 101 and 153

### 2. Windows CUDA Linking Errors
- **Error**: `LINK : fatal error LNK1181: Eingabedatei "aio.lib" kann nicht geöffnet werden.`
- **Error**: `LINK : fatal error LNK1181: Eingabedatei "cufile.lib" kann nicht geöffnet werden.`

## Root Causes and Solutions

### aio.lib Error
- **Cause**: `aio.lib` (Asynchronous I/O) is POSIX-specific and not available on Windows
- **Solution**: This library should not be linked on Windows builds
- **Action**: The CUDA extension build system should exclude this on Windows

### cufile.lib Error
- **Cause**: Missing NVIDIA GPUDirect Storage (GDS) library or incorrect linking
- **Solutions**:
  1. **Install NVIDIA CUDA Toolkit** with GPUDirect Storage components
  2. **Verify CUDA_PATH** environment variable points to correct CUDA installation
  3. **Check library paths** in `%CUDA_PATH%\lib\x64\`
  4. **Use dynamic linking** instead of static linking for cuFile on Windows

## Recommended Actions

### 1. Verify CUDA Installation
```cmd
echo %CUDA_PATH%
dir "%CUDA_PATH%\lib\x64\cufile*"
nvcc --version
```

### 2. Check PyTorch CUDA Compatibility
```python
import torch
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA version: {torch.version.cuda}")
print(f"CUDA available: {torch.cuda.is_available()}")
```

### 3. Update Build Configuration
The `setup.py` in `llava/model/coat/optimizer/kernels/` may need Windows-specific modifications:
- Exclude `aio.lib` on Windows
- Ensure proper cuFile library linking
- Add Windows-specific compiler flags if needed

### 4. PEFT Version Warning
- **Warning**: PEFT configuration compatibility issue
- **Solution**: Update PEFT library: `pip install -U peft`

## Status
- ✅ **Fixed**: torch.cuda.amp deprecation warnings
- ⚠️ **Needs attention**: Windows CUDA library linking
- ⚠️ **Recommended**: Update PEFT library

The application appears to be running despite the linking warnings, suggesting the core functionality is working but with potential performance or stability impacts.