Skip to content
#

mup

Here are 17 public repositories matching this topic...

From-scratch GPT and BPE tokenizer in raw PyTorch, plus µP and mixture-of-experts scaling experiments on cloud GPUs. BPE beats character tokens by 31% in bits/char; µP keeps the best learning rate fixed across widths; MoE beats dense per FLOP but not per GPU-second.

  • Updated Nov 17, 2026
  • Python

Add this topic to your repo

To associate your repository with the mup topic, visit your repo's landing page and select "manage topics."

Learn more