qwen7b-lora-odoo

This model is a fine-tuned version of Qwen/Qwen2.5-Coder-7B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5151

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.9868	0.1975	100	0.8838
0.7517	0.3951	200	0.7373
0.6798	0.5926	300	0.6608
0.6531	0.7901	400	0.6275
0.6287	0.9877	500	0.5984
0.61	1.1852	600	0.5712
0.5866	1.3827	700	0.5576
0.5568	1.5802	800	0.5446
0.5625	1.7778	900	0.5390
0.554	1.9753	1000	0.5317
0.5164	2.1728	1100	0.5261
0.5375	2.3704	1200	0.5219
0.5149	2.5679	1300	0.5201
0.5217	2.7654	1400	0.5162
0.5133	2.9630	1500	0.5151

Framework versions

• PEFT 0.13.2
• Transformers 4.45.2
• Pytorch 2.6.0+cu124
• Datasets 4.1.1
• Tokenizers 0.20.3