This seeems limited - who has enough data to train a MoE? Even if you merge, I assume there is limit until you lose quality?
This seeems limited - who has enough data to train a MoE? Even if you merge, I assume there is limit until you lose quality?
No replies