Add technical conversion reproduction guide

CONVERSION_GUIDE.md

@@ -0,0 +1,125 @@
+# granite-docling ONNX Conversion Guide
+
+## Technical Reproduction Instructions
+
+This document provides complete instructions for reproducing the granite-docling ONNX conversion.
+
+### Prerequisites
+
+- Python 3.10+
+- ~4GB available RAM
+- ~2GB disk space for conversion environment
+
+### Step 1: Environment Setup
+
+```bash
+# Create isolated environment
+python3 -m venv onnx_converter
+source onnx_converter/bin/activate  # Linux/Mac
+# or onnx_converter\Scripts\activate  # Windows
+
+# Install dependencies
+pip install torch torchvision transformers optimum[onnxruntime] safetensors
+```
+
+### Step 2: Download Original Model
+
+```bash
+# Download granite-docling SafeTensors model
+mkdir granite-docling-258m
+cd granite-docling-258m
+
+curl -L "https://huggingface.co/ibm-granite/granite-docling-258M/resolve/main/model.safetensors" -o model.safetensors
+curl -L "https://huggingface.co/ibm-granite/granite-docling-258M/resolve/main/config.json" -o config.json
+curl -L "https://huggingface.co/ibm-granite/granite-docling-258M/resolve/main/tokenizer.json" -o tokenizer.json
+curl -L "https://huggingface.co/ibm-granite/granite-docling-258M/resolve/main/preprocessor_config.json" -o preprocessor_config.json
+```
+
+### Step 3: Install IBM Experimental Fork
+
+```bash
+# Clone IBM experimental optimum-onnx fork
+git clone https://github.com/gabe-l-hart/optimum-onnx.git
+cd optimum-onnx
+git checkout Idefics3Support
+
+# Install experimental fork
+pip install -e . --force-reinstall
+```
+
+### Step 4: Convert to ONNX
+
+```python
+import os
+import torch
+os.environ['CUDA_VISIBLE_DEVICES'] = ''  # Force CPU
+
+from pathlib import Path
+from transformers import Idefics3ForConditionalGeneration
+from optimum.exporters.onnx import export
+from optimum.exporters.onnx.model_configs import Idefics3OnnxConfig
+
+# Load model
+model = Idefics3ForConditionalGeneration.from_pretrained(
+    './granite-docling-258m',
+    trust_remote_code=True,
+    torch_dtype=torch.float32
+).to('cpu')
+
+# Create ONNX config
+onnx_config = Idefics3OnnxConfig(model.config, task='image-to-text')
+
+# Export to ONNX
+output_path = Path('./granite_docling.onnx')
+export(model, onnx_config, output_path, 17)
+
+print(f"ONNX conversion complete: {output_path}")
+```
+
+### Expected Output
+
+```
+Initializing Idefics3ModelPatcher
+Entering Idefics3ModelPatcher context
+Patching Idefics3 model
+Using patched position embedding forward
+Exiting Idefics3ModelPatcher context
+ONNX conversion complete: granite_docling.onnx (1.2GB)
+```
+
+### Validation
+
+```python
+import onnxruntime as ort
+
+# Test ONNX model loading
+session = ort.InferenceSession('granite_docling.onnx')
+print("✅ ONNX model loads successfully")
+
+# Check input/output specifications
+for inp in session.get_inputs():
+    print(f"Input: {inp.name} - {inp.shape}")
+for out in session.get_outputs():
+    print(f"Output: {out.name} - {out.shape}")
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **"Custom architecture" error**: Ensure using IBM experimental fork
+2. **Memory errors**: Use CPU-only conversion (`CUDA_VISIBLE_DEVICES=''`)
+3. **Import errors**: Verify experimental fork installed with `-e .`
+
+### Technical Notes
+
+- **Conversion time**: 5-10 minutes on typical CPU
+- **Memory usage**: ~4GB RAM during conversion
+- **Warnings**: TracerWarnings are expected for complex VLM
+- **File size**: ONNX (~1.2GB) vs SafeTensors (~492MB) due to graph inclusion
+
+## Attribution
+
+Original model: IBM Research granite-docling-258M
+Conversion method: IBM experimental Idefics3Support optimum-onnx fork
+Documentation: lamco-development