John's picture

John

cmp-nct

·

https://github.com/cmp-nct/ggllm.cpp

cmp-nct

AI & ML interests

None yet

Recent Activity

liked a model about 23 hours ago

MiniMaxAI/MiniMax-M2

liked a model 2 days ago

deepseek-ai/DeepSeek-OCR

replied to onekq's post 3 days ago

Context rot is such a catchy phrase, but the problem has been identified 2+ years ago, called attention decay. https://huggingface.co/papers/2307.03172 I spotted the same problem in coding tasks, and documented in my book (https://www.amazon.com/dp/9999331130). Why did this problem become hot again? This is because many of us thought the problem has been solved by long context models, which is not true. Here we were misled by benchmarks. Most long-context benchmarks build around the QA scenario, i.e. "finding needle in haystack". But in agentic scenarios, the model needs to find EVERYTHING in the haystack, and just can't afford enough attention for this challenge.

View all activity

Organizations

Collections 1

models 7

cmp-nct/llava-1.6-gguf

Image-Text-to-Text • 7B • Updated Feb 21, 2024 • 1.17k • 76

cmp-nct/xcomposer2_gguf_for_llama.cpp_development

8B • Updated Jan 31, 2024 • 32 • 2

cmp-nct/Yi-VL-34B-GGUF

34B • Updated Jan 29, 2024 • 81 • 11

cmp-nct/ShareGPT4V-13B-quant-gguf

13B • Updated Jan 26, 2024 • 44 • 7

cmp-nct/Yi-VL-6B-GGUF

6B • Updated Jan 24, 2024 • 53 • 8

cmp-nct/ShareGPT4V-7B-quant-gguf

Updated Jan 10, 2024 • 41 • 4

cmp-nct/Nous-Hermes-2-Vision-Alpha-GGUF

Updated Dec 9, 2023 • 87 • 4

datasets 0

None public yet