Questions about context length
#11
by
1245244103
- opened
When you tested, was the context length 64K or 128K? if I want to extend it to 128k in sglang, how should I do that? Thanks.
You can add a rope_scaling field to the model's config.json file.
Check out detail information in model card.