Amazon SageMaker AI drops the mandatory S3 upload for asynchronous inference

Amazon has updated SageMaker AI: asynchronous inference now accepts input data directly in the body of the API request, with no mandatory prior upload to Amazon S3.

How asynchronous inference works

SageMaker Async Inference is a mode for ML tasks that cannot be completed within a few seconds. Audio transcription, PDF processing, long-form text generation and batch image classification are all scenarios where waiting for the model's response can take anywhere from tens of seconds to several minutes. Synchronous inference is a poor fit here: a client cannot keep an HTTP connection open that long.

Async mode solves this with a queue. The request is queued for processing, the model works in the background, the result is saved to an S3 bucket, and the client either receives a notification or polls periodically to check whether it is ready. This is a standard pattern for production ML systems with variable response times.

Before this update, the standard workflow required two steps: first upload the input data to S3, then pass `InvokeEndpointAsync` a reference to that object. Only then did SageMaker start work. Every call meant an extra write operation to storage and an additional network request.

What has changed

Amazon has added support for inline payloads: data can now be passed directly in the body of an `InvokeEndpointAsync` request, as in an ordinary REST API. Uploading input data to S3 beforehand is no longer needed.

Practical changes:

• Less code: no need to write logic for creating S3 objects, passing the key and later cleaning up temporary files
• Lower operating costs: every PUT operation in S3 costs money, and with thousands of small calls a day it adds up to a noticeable sum
• Easier debugging: input data is visible right in the request, so there is no need to open the S3 Console to see what was actually sent
• Fewer points of failure: one network call instead of two lowers the chance of failure at the data upload stage
• Faster processing start: SageMaker begins inference immediately, without waiting to fetch data from storage

Which tasks it suits

The update is especially useful where input data is small or medium in size and generated dynamically.

Embedded ML requests from applications: when a user submits text for analysis or an image for classification, storing an intermediate object in S3 just to hand it to the model is wasted work. Now the data goes directly.

Post-deployment testing: in CI/CD pipelines, checking a model after rollout can involve dozens of test requests. Managing temporary S3 objects for each of them is no longer necessary.

For heavy data such as long video files, large audio recordings and document corpora, the S3 route remains the optimal option and is still available. Both approaches work side by side, and the developer picks the one that fits the task.

What this means

Amazon is steadily removing infrastructure barriers in SageMaker. Inline payload support is a small but concrete improvement: fewer steps, fewer dependencies, a cleaner pipeline architecture. For teams running high-frequency asynchronous inference, it means real savings in development time and operating costs.