YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
CLIP Transcoder Checkpoint
This model is a Transcoder trained on CLIP's internal representations.
Model Details
Architecture
- Layer: 10
- Layer Type: ln2.hook_normalized
- Model: laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K
- Dictionary Size: 49152
- Input Dimension: 768
- Expansion Factor: 64
- CLS Token Only: False
Training
- Learning Rate: 0.0010281538518292835
- Batch Size: 4096
- Context Size: 50
Sparsity
- L0 (Active Features): 1024
Additional Information
- Wandb Run: https://wandb.ai/perceptual-alignment/openclip-transcoders/runs/9y2vv0lh/
- Random Seed: 42
- Downloads last month
- -
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support