MoE Long-Context Training Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-long-context/card.yaml What Changes At Long Context Once sequence length moves well past the 4K-class regime, attention memory and activation residency become the dominant constraints. For MoE models, that usually means you need some combination of: - context parallelism - selective recompute - lower precision - CPU offload for optimizer state - a dispatcher and PP layout that do not waste the smaller remaining DP budget Rounded Scaling Patterns DSV3 on H100 The DSV3 long-context runs…