#8 Vision MoE — sparse to soft
· One min read
A working notebook on Vision MoE — from V-MoE's hard top- routing to Soft MoE's continuous slot-based formulation.
:::note Work in progress This post is being written. The references below are the load-bearing reading list. :::
References
- Carlos Riquelme et al. "Scaling Vision with Sparse Mixture of Experts" arXiv:2106.05974, 2021. (V-MoE)
- Joan Puigcerver et al. "From Sparse to Soft Mixtures of Experts" arXiv:2308.00951, 2023. (Soft MoE)
Created: 2026-05-18 / Updated: 2026-05-18