Nous Research releases Token Superposition Training, cutting LLM pretraining time up to 2.5x

Nous Research published Token Superposition Training (TST), a two-phase pretraining method that averages contiguous token embeddings into 'bags' in phase 1 then reverts to standard next-token prediction. Cuts wall-clock training time up to 2.5x at matched FLOPs across 270M-10B models.