15 13 14

Gabriel Mongaras

gmongaras

mshojaei77's profile picture

g-ronimo's profile picture

luosuu's profile picture

gmongaras
gmongaras
gmongaras

AI & ML interests

None yet

Recent Activity

upvoted a paper 22 days ago

Less is More: Recursive Reasoning with Tiny Networks

upvoted a paper 23 days ago

Fast-dLLM v2: Efficient Block-Diffusion LLM

upvoted an article about 1 month ago

There is no such thing as a tokenizer-free lunch

View all activity

Organizations

gmongaras 's collections 7

Stable Diffusion 3 Checkpoints

Collection of checkpoints from the stable diffusion 3 model I am training (https://github.com/gmongaras/Stable-Diffusion-3-From-Scratch)

gmongaras/datav3_attempt5_8GPU_SoftFlash_RoPE2d_2AccSteps_13batchsize_stage3

Updated May 14
gmongaras/datav3_attempt5_8GPU_SoftFlash_RoPE2d_2AccSteps_40batchsize_stage2

Updated Apr 28
gmongaras/datav3_attempt5_8GPU_SoftFlash_RoPE2d_2AccSteps_140batchsize_stage1

Updated Apr 19
gmongaras/datav3_attempt4_8GPU_SoftFlash_RoPE2dV2_2AccSteps_stage2

Updated Apr 11

Cosine Attention (Cottention)

Models for the paper Cottention: Linear Transformers With Cosine Attention https://arxiv.org/abs/2409.18747

gmongaras/Cosine_Attention_GPT_300M

Feature Extraction • Updated Oct 7, 2024 • 2
gmongaras/Softmax_Attention_GPT_1.2B

Feature Extraction • Updated Oct 7, 2024 • 1
gmongaras/Softmax_Attention_GPT_300M

Feature Extraction • Updated Oct 7, 2024
gmongaras/Cosine_Attention_GPT_1.2B

Feature Extraction • Updated Oct 7, 2024

Squad Models

Models trained on squad data

gmongaras/Wizard_7B_Squad

Text Generation • Updated Sep 11, 2023 • 1
gmongaras/Wizard_7B_Squad_8bit

Text Generation • Updated Sep 11, 2023
gmongaras/Wizard_7B_Squad_v2

Text Generation • Updated Sep 15, 2023

Subtitle Data

gmongaras/Anime_Subtitle_data

Viewer • Updated Mar 31, 2024 • 14.6M • 25 • 1
gmongaras/Anime_Subtitle_data2

Viewer • Updated Mar 31, 2024 • 1.91M • 1

datasets

gmongaras/CC12M_and_Imagenet21K_Recap_Highqual_512

Viewer • Updated Apr 24 • 19.8M • 208 • 1
gmongaras/CC12M_and_Imagenet21K_Recap_Highqual_256

Viewer • Updated Apr 21 • 19.8M • 89
gmongaras/CC12M_and_Imagenet21K_Recap_Highqual

Viewer • Updated Feb 21 • 19.8M • 23.2k • 4
gmongaras/CC12M_and_Imagenet21K_Recap

Viewer • Updated Sep 17 • 22.7M • 3.85k • 6

Reddit Models

Some terrible Reddit models I am training just to see what happens. Never again will I hear "As an AI language model"

gmongaras/Wizard_7B_Reddit_Political_2019_13B

Text Generation • Updated Sep 15, 2023
gmongaras/Wizard_7B_Reddit_Political_2019

Text Generation • Updated Sep 11, 2023
gmongaras/Wizard_7B_Reddit_Political_2019_8bit

Text Generation • 7B • Updated Sep 11, 2023
gmongaras/reddit_negative_v1_8B

Text Generation • Updated Sep 15, 2023

BERT_512

gmongaras/BERT_Base_Cased_512_Dataset

Viewer • Updated Nov 28, 2023 • 136M • 66
gmongaras/BERT_Base_Cased_512_Dataset_Mapped

Viewer • Updated Nov 29, 2023 • 136M • 6
gmongaras/BERT_Base_Cased_512_GLUE

Viewer • Updated Dec 11, 2023 • 1.44M • 14
gmongaras/BERT_Base_Cased_512_GLUE_Mapped

Viewer • Updated Dec 11, 2023 • 1.44M • 22

Stable Diffusion 3 Checkpoints

Collection of checkpoints from the stable diffusion 3 model I am training (https://github.com/gmongaras/Stable-Diffusion-3-From-Scratch)

gmongaras/datav3_attempt5_8GPU_SoftFlash_RoPE2d_2AccSteps_13batchsize_stage3

Updated May 14
gmongaras/datav3_attempt5_8GPU_SoftFlash_RoPE2d_2AccSteps_40batchsize_stage2

Updated Apr 28
gmongaras/datav3_attempt5_8GPU_SoftFlash_RoPE2d_2AccSteps_140batchsize_stage1

Updated Apr 19
gmongaras/datav3_attempt4_8GPU_SoftFlash_RoPE2dV2_2AccSteps_stage2

Updated Apr 11

datasets

gmongaras/CC12M_and_Imagenet21K_Recap_Highqual_512

Viewer • Updated Apr 24 • 19.8M • 208 • 1
gmongaras/CC12M_and_Imagenet21K_Recap_Highqual_256

Viewer • Updated Apr 21 • 19.8M • 89
gmongaras/CC12M_and_Imagenet21K_Recap_Highqual

Viewer • Updated Feb 21 • 19.8M • 23.2k • 4
gmongaras/CC12M_and_Imagenet21K_Recap

Viewer • Updated Sep 17 • 22.7M • 3.85k • 6

Cosine Attention (Cottention)

Models for the paper Cottention: Linear Transformers With Cosine Attention https://arxiv.org/abs/2409.18747

gmongaras/Cosine_Attention_GPT_300M

Feature Extraction • Updated Oct 7, 2024 • 2
gmongaras/Softmax_Attention_GPT_1.2B

Feature Extraction • Updated Oct 7, 2024 • 1
gmongaras/Softmax_Attention_GPT_300M

Feature Extraction • Updated Oct 7, 2024
gmongaras/Cosine_Attention_GPT_1.2B

Feature Extraction • Updated Oct 7, 2024

Reddit Models

Some terrible Reddit models I am training just to see what happens. Never again will I hear "As an AI language model"

gmongaras/Wizard_7B_Reddit_Political_2019_13B

Text Generation • Updated Sep 15, 2023
gmongaras/Wizard_7B_Reddit_Political_2019

Text Generation • Updated Sep 11, 2023
gmongaras/Wizard_7B_Reddit_Political_2019_8bit

Text Generation • 7B • Updated Sep 11, 2023
gmongaras/reddit_negative_v1_8B

Text Generation • Updated Sep 15, 2023

Squad Models

Models trained on squad data

gmongaras/Wizard_7B_Squad

Text Generation • Updated Sep 11, 2023 • 1
gmongaras/Wizard_7B_Squad_8bit

Text Generation • Updated Sep 11, 2023
gmongaras/Wizard_7B_Squad_v2

Text Generation • Updated Sep 15, 2023

BERT_512

gmongaras/BERT_Base_Cased_512_Dataset

Viewer • Updated Nov 28, 2023 • 136M • 66
gmongaras/BERT_Base_Cased_512_Dataset_Mapped

Viewer • Updated Nov 29, 2023 • 136M • 6
gmongaras/BERT_Base_Cased_512_GLUE

Viewer • Updated Dec 11, 2023 • 1.44M • 14
gmongaras/BERT_Base_Cased_512_GLUE_Mapped

Viewer • Updated Dec 11, 2023 • 1.44M • 22

Subtitle Data

gmongaras/Anime_Subtitle_data

Viewer • Updated Mar 31, 2024 • 14.6M • 25 • 1
gmongaras/Anime_Subtitle_data2

Viewer • Updated Mar 31, 2024 • 1.91M • 1