Google have released the models and code for the Switch Transformer from Fedus e... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		javidlakha on Oct 9, 2023 \| parent \| context \| favorite \| on: Mixtures of Experts Google have released the models and code for the Switch Transformer from Fedus et al. (2021) under the Apache 2.0 licence. [0] There's also OpenMoE - an open-source effort to train a mixture of experts model. Currently they've released a model with 8 billion parameters. [1] [0] https://github.com/google-research/t5x/blob/main/docs/models... [1] https://github.com/XueFuzhao/OpenMoE

Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact