Announcing Amazon SageMaker Inference for custom Amazon Nova models | Amazon Web Services

Amazon SageMaker Inference adds general availability support for custom Amazon Nova models, amazon SageMaker Inference now supports custom Amazon Nova models in general availability.

Instance types includes g5.12xlarge, g5.24xlarge, g5.48xlarge, g6.12xlarge, g6.24xlarge, g6.48xlarge, and p5.48xlarge for Nova Micro; g5.48xlarge, g6.48xlarge, and p5.48xlarge for Nova Lite; and p5.48xlarge for Nova 2 Lite, supported instance types at GA for Nova Micro, Nova Lite, and Nova 2 Lite are listed.

Deployment workflow allows deploying trained Nova model artifacts via SageMaker Studio or SageMaker AI SDK, deployments can be created from SageMaker Studio or the SageMaker AI SDK using a trained Nova model artifact.

Cost optimization supports optimized GPU utilization using Amazon EC2 G5 and G6 instances to reduce inference cost compared with P5 instances, amazon EC2 G5 and G6 instances are presented as options to reduce inference cost compared with P5 instances.

Endpoint creation requires time to provision infrastructure, download model artifacts, and initialize the inference container, endpoints require provisioning, model download, and container initialization before showing InService status.