|
|
--- |
|
|
base_model: |
|
|
- tencent/SRPO |
|
|
base_model_relation: quantized |
|
|
library_name: diffusers |
|
|
license: other |
|
|
license_name: tencent-hunyuan-community |
|
|
license_link: https://github.com/Tencent-Hunyuan/SRPO/blob/main/LICENSE.txt |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: text-to-image |
|
|
--- |
|
|
|
|
|
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11 |
|
|
|
|
|
Feel free to request for other models for compression as well, although compressing models that do not use the Flux architecture might be tricky for me. |
|
|
|
|
|
This compressed model was made from [rockerBOO/flux.1-dev-SRPO](https://huggingface.co/rockerBOO/flux.1-dev-SRPO)'s BF16 quantization. Thanks to rockerBOO, without which I would not have been able to directly work with. (My PC only has 48 GB of system RAM, too little to work with a 12B model in FP32 precision) |
|
|
|
|
|
### How to Use |
|
|
|
|
|
#### `diffusers` |
|
|
|
|
|
1. Install the DFloat11 pip package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*: |
|
|
|
|
|
```bash |
|
|
pip install dfloat11[cuda12] |
|
|
# or if you have CUDA version 11: |
|
|
# pip install dfloat11[cuda11] |
|
|
``` |
|
|
2. To use the DFloat11 model, run the following example code in Python: |
|
|
```python |
|
|
import torch |
|
|
from diffusers import FluxPipeline, FluxTransformer2DModel |
|
|
from dfloat11 import DFloat11Model |
|
|
with no_init_weights(): |
|
|
transformer = FluxTransformer2DModel.from_config( |
|
|
FluxTransformer2DModel.load_config( |
|
|
"black-forest-labs/FLUX.1-dev", |
|
|
subfolder="transformer" |
|
|
), |
|
|
torch_dtype=torch.bfloat16 |
|
|
).to(torch.bfloat16) |
|
|
|
|
|
pipe = FluxPipeline.from_pretrained( |
|
|
"black-forest-labs/FLUX.1-dev", |
|
|
transformer=transformer, |
|
|
torch_dtype=torch.bfloat16 |
|
|
) |
|
|
DFloat11Model.from_pretrained('mingyi456/SRPO-DF11', device='cpu', bfloat16_model=pipe.transformer) |
|
|
pipe.enable_model_cpu_offload() |
|
|
prompt = "A futuristic cityscape at sunset, with flying cars, neon lights, and reflective water canals" |
|
|
image = pipe( |
|
|
prompt, |
|
|
guidance_scale=3.5, |
|
|
num_inference_steps=30, |
|
|
max_sequence_length=256, |
|
|
generator=torch.Generator("cpu").manual_seed(0) |
|
|
).images[0] |
|
|
image.save("SPRO.png") |
|
|
``` |
|
|
#### ComfyUI |
|
|
Refer to this [model](https://huggingface.co/mingyi456/SRPO-DF11-ComfyUI) page instead, and follow the instructions there. |
