BerriAI releases LiteLLM Agent Platform for running agents in production

Running an AI agent in a local script is easy. Running it reliably in production, keeping its state across restarts and providing isolated environments for different contexts is another matter entirely.

The production problem

When an AI agent runs only on a developer's machine, everything looks simple. But the move to production quickly brings problems: there is no session storage across OS restarts, no context isolation between users, teams are hard to manage, and built-in monitoring is missing. Every company that has tried to run an agent in production ended up writing its own infrastructure.

BerriAI, the company behind the popular LiteLLM AI Gateway, has released a solution to this problem: LiteLLM Agent Platform. It is a Kubernetes-based, self-hosted infrastructure layer that automates everything required.

What LiteLLM Agent Platform includes

The platform provides the key components for production-ready agents:

• Isolated sandboxes for each agent or context
• Persistent session management: state is preserved across restarts
• Team management and access control
• Built-in monitoring, logging and tracing
• Self-hosted deployment on Kubernetes
• An API for managing the agent lifecycle

You define the agent specification; the platform creates the environment for it and manages everything else.

How it differs from LiteLLM Gateway

It is important not to confuse LiteLLM Agent Platform with LiteLLM AI Gateway (from the same developer). The Gateway manages requests to different LLMs (OpenAI, Anthropic, Cohere, local models). It handles request routing and caching. Agent Platform sits a level above: it is infrastructure for the full lifecycle of an agent. The Gateway can be part of the stack, but the Platform takes on all the complexity of the environment.

What this means

In practice, this removes a whole class of problems for teams that want to run AI agents in production. They no longer need to write session persistence, context isolation and monitoring themselves. Because the code is open source, companies can integrate the platform into their own stack without vendor lock-in. This should speed up the move of agents from proof of concept to production.