add transformers usage of PaddleOCR-VL-0.09B

README.md

@@ -140,6 +140,51 @@ for res in output:
     ```
   
 **For more usage details and parameter explanations, see the [documentation](https://www.paddleocr.ai/latest/en/version3.x/pipeline_usage/PaddleOCR-VL.html).**
+
+## PaddleOCR-VL-0.9B Usage with transformers
+
+
+Currently, we support inference using the PaddleOCR-VL-0.9B model with the `transformers` library, which can recognize texts, formulas, tables, and chart elements. In the future, we plan to support full document parsing inference with `transformers`. Below is a simple script we provide to support inference using the PaddleOCR-VL-0.9B model with `transformers`. We currently recommend using the official method for inference, which is faster and can support page-level document parsing.
+
+
+```python
+from PIL import Image
+import torch
+from transformers import AutoModelForCausalLM, AutoProcessor
+
+DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
+
+CHOSEN_TASK = "ocr"  # Options: 'ocr' | 'table' | 'chart' | 'formula'
+PROMPTS = {
+    "ocr": "OCR:",
+    "table": "Table Recognition:",
+    "formula": "Formula Recognition:",
+    "chart": "Chart Recognition:",
+}
+
+model_path = "PaddlePaddle/PaddleOCR-VL"
+image_path = "test.png"
+image = Image.open(image_path).convert("RGB")
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_path, trust_remote_code=True, torch_dtype=torch.bfloat16
+).to(DEVICE).eval()
+processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
+
+messages = [{"role": "user", "content": PROMPTS[CHOSEN_TASK]}]
+text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+inputs = processor(text=[text], images=[image], return_tensors="pt")
+inputs = {k: (v.to(DEVICE) if isinstance(v, torch.Tensor) else v) for k, v in inputs.items()}
+
+with torch.inference_mode():
+    generated = model.generate(**inputs, max_new_tokens=1024, do_sample=False, use_cache=True)
+
+resp = processor.batch_decode(generated, skip_special_tokens=True)[0]
+answer = resp.split(text)[-1].strip()
+print(answer)
+```
+
 ## Performance
 
 ### Page-Level Document Parsing