AWS Compute Selection Guide for Distributed Training (Part 2: Ultra-Scale Scaling and Instance Availability) | Amazon Web Services

AWS outlined an UltraClusters architecture and GPU capacity strategy for large-scale distributed training.

The key idea is to place all nodes on the same non-blocking network fabric to avoid bandwidth variance and bottlenecks.

GPU capacity can be reserved through ODCR or Capacity Block, while EFA, SRD, GPUDirect RDMA, and NCCL tuning are critical.

AWS also recommended integrating FSx for Lustre with S3, SageMaker HyperPod, PCS, EKS, and checkpointing workflows.