SageMaker SOCI-Indexed Images: Training Containers Start in Seconds, Not Minutes

AWS announced on October 27, 2026 that SageMaker AI training and processing jobs can now use Seekable OCI (SOCI) indexed container images. Instead of downloading a whole 10 GB-plus image before your code runs, SageMaker lazy-loads it: it fetches only what is needed to start the container and streams the rest while your job runs. In AWS's benchmarks, download time falls from minutes to seconds. This guide covers the announcement, the numbers, how to switch and where it does not help.

It sits alongside other SageMaker changes we covered recently, including training session APIs for your own RL loop and a per-camera edge vision pipeline on SDK v3.

Quick reference

The problem it targets

SageMaker jobs run in container images stored in Amazon ECR. Before your code executes, the service downloads, decompresses and writes every layer of the image to the instance. Deep learning images routinely exceed 10 GB, which adds minutes to every job, and AWS notes you pay for that as job time. The pain peaks in hyperparameter sweeps with hundreds of jobs, quick experiments and short processing jobs where the pull can dominate runtime.

How SOCI works

A SOCI index is metadata mapping every file in your image, stored in ECR as an OCI-compliant artifact linked to the original. Because the index is a separate artifact, your image digest and signatures stay valid. When a job is created, SageMaker checks for an index. If present, it lazy-loads; if absent, it downloads normally; if the index is unreadable or lazy loading fails, it falls back to a standard download. In AWS's words, adding an index never prevents a job from running.

What AWS measured

AWS pulled eight images (1.8 to 35.4 GB compressed) on instance types from ml.g4dn through ml.p5 and ml.m7i, in us-west-2 and us-east-1, reporting medians of repeated runs.

Across 13 instance type and size combinations with the 13.2 GB image, standard downloads took 3m16s to 4m20s while lazy loading held at 6 to 8 seconds, so the saving does not depend on instance size.

Training speed is not harmed beyond a one-time cost. AWS ran ResNet-50 on Imagenette (PyTorch and TensorFlow images, seven instance types) and full-parameter SFT of Qwen2.5-0.5B on alpaca-cleaned (three instance types). From the second epoch on, time per epoch matched the standard image; the first epoch took 3 to 16 seconds longer while files were fetched on first touch. Net result: each job finished 1 to 6 minutes sooner.

When the gain shrinks

AWS is upfront about the limits:

• If your entry point reads most of the image before training (scanning, checksumming, preloading), the first epoch waits for roughly the bytes a full pull would fetch, and the improvement is small.
• Front-loaded starts still came out ahead in tests: torch.compile and a JIT-built CUDA extension added 7 to 15 seconds yet still saved about 3m25s to 3m56s; a 6 GB model baked into the image added 2 to 3.5 minutes at load time but the job still finished 2 to 2.5 minutes sooner.
• Images smaller than roughly 4 GB see relatively small improvements.
• Only Amazon ECR private repositories qualify. Images from other registries, or pulled through a private Docker registry (TrainingRepositoryAccessMode set to Vpc), are not eligible.
• SOCI images run on a containerd-based runtime, so check scripts that call the Docker CLI, read /var/run/docker.sock or derive identifiers from cgroup paths. SageMaker mounts a /.dockerenv marker for compatibility, but AWS advises against adding new dependencies on it.

These benchmarks cover two workloads; test your own, since first-epoch cost depends on what your code touches at startup.

Option 1: use a pre-indexed Deep Learning Container

AWS publishes SOCI-indexed variants of several Deep Learning Containers, marked with a -soci tag suffix, for PyTorch Training, TensorFlow Training, vLLM and SGLang, with arm64 variants for some frameworks. Change the tag in your job definition, for example:

763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.9-gpu-py312-cu130-ubuntu22.04-sagemaker-v1-soci

If you build your own image FROM a DLC, you must index your final image; the base image's index does not carry forward.

Option 2: index your own image

You need a Linux host (AWS suggests Amazon Linux 2023 on EC2) with containerd, the SOCI snapshotter and Finch or nerdctl. The core steps from AWS's post:

sudo dnf install -y containerd fuse soci-snapshotter runfinch-finch
sudo systemctl start soci-snapshotter
sudo systemctl restart containerd
sudo nerdctl pull "$REGISTRY/$REPOSITORY_NAME:$ORIGINAL_IMAGE_TAG"
sudo nerdctl image convert --soci "$REGISTRY/$REPOSITORY_NAME:$ORIGINAL_IMAGE_TAG" "$REGISTRY/$REPOSITORY_NAME:$SOCI_IMAGE_TAG"
sudo nerdctl push --platform linux/amd64 "$REGISTRY/$REPOSITORY_NAME:$SOCI_IMAGE_TAG"

This creates the SOCI index and an OCI image index manifest linking your image to it, leaving the original image and digest unchanged. Authenticate to ECR first with aws ecr get-login-password. To automate on every push, AWS points to the CFN ECR SOCI Index Builder, which uses an EventBridge rule on ECR push events.

Then reference the tag as usual, for example with the SageMaker Python SDK's ModelTrainer(training_image=".../my-training-image:v1.0-soci", ...). To verify the tag is indexed, query it with aws ecr batch-get-image and look for a child manifest whose artifactType begins with application/vnd.amazon.soci.index..

Lazy loading, explained without the jargon

A container image is a stack of compressed layers. A normal pull downloads every layer, unpacks it and writes it to disk before the first line of your code runs. Most of that data is never touched in the first seconds: documentation, unused libraries, alternative CUDA components. Lazy loading flips the order. The open-source SOCI snapshotter uses the index to know where each file lives inside the compressed layers, so it can fetch just the files the container asks for, when it asks for them.

That is why the start time stays between 3 and 11 seconds whether the image is 1.8 GB or 35.4 GB: the work needed to start does not grow with image size, only the background streaming does. It is also why the first epoch pays a small tax. The first time your framework imports a library or opens a file, that file may still be on the network, and AWS measured that at 3 to 16 seconds.

The design choice that makes this safe to adopt is that the index is separate metadata. Your image digest, signatures and tags are unchanged, so supply-chain checks that rely on digests still work, and removing the index simply returns you to ordinary downloads.

What it does to your bill

AWS frames the benefit in two ways: time and money. Training jobs are billed for job time, which includes the image download. Cutting a multi-minute pull to seconds cuts billable time per job by the same amount, minus the first-epoch cost. AWS reports each tested job finishing 1 to 6 minutes sooner.

For one job that is small. For a tuning sweep of 300 jobs, a three-minute saving on each is 15 hours of instance time that no longer idles on GPU hardware. We have not priced that for you, because instance rates differ by type and Region and AWS does not publish a dollar figure for the benchmark. The practical advice is to multiply your own measured saving by your own job count and instance price before deciding whether the migration is worth an afternoon.

Why this matters for teams training models

Startup time is a hidden tax on experimentation. A team running hundreds of short jobs in a tuning sweep can lose a large share of billed time to downloads, and GPU instances sit idle while images pull. The change is low-risk because of the automatic fallback and unchanged job definitions. For teams already fine-tuning open models on SageMaker, such as the Exowatt pipeline that trained Qwen3 models in minutes to hours, shorter starts compound across many experiments. Serverless model customization hides this layer entirely, but custom-container users now have an equivalent speedup.

A rollout plan

1. Pick one frequent job. Choose a workload that runs often with an image over 10 GB.
2. Switch the tag or index the image. Use the -soci DLC or build an index.
3. Compare startup and end-to-end time. Measure image download, first-epoch time and total runtime for both images.
4. Look for Docker assumptions. Search entry points for Docker CLI and socket usage.
5. Automate the index. Add the EventBridge-based builder so new pushes are indexed.
6. Roll out wider once first-epoch cost is acceptable.

Related reading

• SageMaker training session APIs
• SageMaker multi-turn RL walkthrough
• One model per camera on SageMaker SDK v3
• Exowatt fine-tuned Qwen3 on SageMaker
• Property Finder and evaluation bias on Bedrock
• AI chip architectures: GPU, TPU, Trainium

Details are accurate as of October 27, 2026; figures are AWS's own benchmarks.