no, there’s no inner vs outer loop. this is literally just a regular MoE where some experts are noop
Replies
the outer loop on the linked paper does that also so i think it is probably an influence. analysis on their approach later showed that this was, in some cases, the important detail
aug 31, 2025, 4:14 pm • 0 0 • view