Is that an implementation of the “models should work with Type 1 and Type 2 brains” paper? bsky.app/profile/segy...
Replies
no, there’s no inner vs outer loop. this is literally just a regular MoE where some experts are noop
aug 31, 2025, 3:33 pm • 1 0 • view
the outer loop on the linked paper does that also so i think it is probably an influence. analysis on their approach later showed that this was, in some cases, the important detail
aug 31, 2025, 4:14 pm • 0 0 • view