SequentialLearning
/

SuperLinear

mixture-of-experts

Model card Files Files and versions

lirannoc commited on Jul 14

Commit

4f95ef6

·

verified ·

1 Parent(s): b7195ad

Update config.json

Files changed (1) hide show

config.json +0 -1

config.json CHANGED Viewed

@@ -18,7 +18,6 @@
   "auto_regressive": 1,
   "_comment_moe": "MoE (Mixture of Experts) parameters",
-  "moe": 1,
   "moe_n_experts": 4,
   "top_k_experts": 12,
   "noisy_gating_std": 0.1,

   "auto_regressive": 1,
   "_comment_moe": "MoE (Mixture of Experts) parameters",
   "moe_n_experts": 4,
   "top_k_experts": 12,
   "noisy_gating_std": 0.1,