AWS walks through reinforcement fine-tuning in Amazon Bedrock using OpenAI-compatible APIs

AWS has published a detailed walkthrough of how to run reinforcement fine-tuning in Amazon Bedrock through OpenAI-compatible APIs. In essence, the company is offering a route developers already know: the same OpenAI SDK, but with Bedrock as the platform for training, evaluation and inference.

How the process works

Reinforcement fine-tuning (RFT) is for cases where simply showing the model correct answers, as in classic supervised fine-tuning, is not enough. Here the model generates several candidate answers to the same prompt, and a separate reward function then assigns each a numerical score. Amazon Bedrock takes that signal and runs the optimization loop itself using the GRPO algorithm. For a team, this means there is no need to build heavy infrastructure for reinforcement learning: the service handles orchestration, parallelism, checkpoints and metrics.

In the practical walkthrough, AWS shows that it made the entry point as similar as possible to the familiar OpenAI stack. A developer only needs to point `OPENAI_BASE_URL` at the regional Bedrock Mantle endpoint and pass the `OPENAI_API_KEY` generated for Bedrock. From there, the same calls work: `client.files.create()`, `client.fine_tuning.jobs.create()` and `client.chat.completions.create()`. In other words, the barrier is not a new SDK but how well you have formalized the criterion for answer quality.

Data and the reward function

In its example, AWS uses the GSM8K dataset of grade-school math problems. The data is uploaded through the Files API in JSONL format: each line contains a `messages` block, and verifiable tasks add a `reference_answer`. This format lets you not only send the question to the model but also store a reference answer or a checking rule. The walkthrough also shows that the prompt can be structured in advance so the final answer is easy to extract automatically, for example in a special format such as `\boxed{}` or after a `####` marker.

The key component of the whole setup is the reward function in AWS Lambda. In the demo, it receives the trajectories, finds the assistant's last answer, extracts the correct answer from `reference_answer` and returns a score from 0 to 1. For math this is a simple binary check, but the logic is not limited to such cases. AWS stresses that you can embed your own rules in Lambda, and for tasks that are harder to formalize you can use a model-as-a-judge approach. There is also a point that matters for enterprises: during the process, data does not leave the protected AWS environment and is not used to train Bedrock models.

Training and launch

Launching training is fairly compact: `fine_tuning.jobs.create()` takes the base model, the training file, the `reinforcement` method type, the ARN of the Lambda grader and a set of hyperparameters. The example uses `openai.gpt-oss-20b`, one epoch, `batch_size=4` and `learning_rate_multiplier=1.0`, although the documentation recommends starting below one for stability. Bedrock then creates the job itself, counts the steps and saves intermediate checkpoints, which can be used to assess quality before training ends.

During training, AWS suggests watching not only the job status but also the events with metrics. In the example, the job on a GSM8K subset runs 67 steps, and the reward curve rises from roughly 0.56 to a range of 0.85–0.97 by the middle of training. At the same time, answers get shorter, which the authors read as a sign that the model learned to solve problems more accurately and without needless chatter.

• `critic_rewards_mean`: the main signal; if it rises, the model is learning
• `actor_entropy`: shows whether answer diversity is collapsing into mode collapse
• `actor_grad_norm`: helps spot instability if gradients start jumping sharply
• `response_length_mean`: useful against reward hacking, when the model starts padding answers to score higher

After the job finishes, the model does not need to be deployed separately. You just take `fine_tuned_model` from the job details and call it immediately through the Chat Completions API or Responses API, including streaming. This is the main practical advantage of the whole approach: training and inference stay within a single API landscape. The Bedrock documentation separately notes that the OpenAI-compatible path for fine-tuning is currently available for `openai.gpt-oss-20b` and `qwen.qwen3-32b` in the `us-west-2` region.

"No separate endpoint or hosting."

What this means

AWS is clearly trying to turn reinforcement fine-tuning from a research exotic into an ordinary engineering tool. If a team already has code written for the OpenAI SDK and clear reward logic, getting into RFT becomes noticeably easier: you can start with 100–200 examples, check the metrics, compare checkpoints and find out whether tuning will yield a cheaper, faster model for a specific task. This is especially interesting for math, code and other scenarios where answer quality can be checked automatically.