DeepMind introduces Decoupled DiLoCo: 88% goodput under high failure rates

Google DeepMind unveiled Decoupled DiLoCo, an asynchronous distributed training architecture that maintains 88% goodput even under elevated hardware failure rates. It addresses the fragility of synchronized gradient updates at hundreds-of-billions-of-parameter scale.