•1 min read•from Machine Learning
Does this idea sound fun? [R]
It's about inference-time learning by inserting some experts specialized for updating sibling expert weights in MoE. All the components needed were already there, but no one tried it inside MoE, so I did a small PoC. It kinda worked. I'd love to hear what you think.
[link] [comments]
Want to read more?
Check out the full article on the original site