Excellent work
Your models make the case for frameworks like LangGraph and CrewAI to do amazing things in parallel. I think models finetuned for Directional Acyclic Graphs (DAG) and RAG combined, plus effective judge models, could help complete this. Are you looking into such specialities?
Thanks for your message @sometimesanotion !
Do you have precise examples for these two use cases? I'd love to explore these ideas.
The tricky part: @sequelbox has published a dataset and some finetunes of larger models to generate the structured DAG. I've been working on abstract dependency management as precalculated reasoning, and this lands quite close. I think you can look at the completion column and get a good idea. It's fascinating work.
Simpler part: I mean a judge model in a general sense. The idea here is to have reasoning blocks for common problems cached from larger model's solutions - in stages.
Thank you for pushing small models further!