NVIDIA Confidential Computing: protecting data during AI inference without sacrificing speed

NVIDIA published in February 2025 a technical overview of Confidential Computing (CC) — a hardware solution for protecting corporate data directly in the process of AI computations, developed with an eye toward the era of agent systems.

Why security became the main barrier for AI

Wide adoption of AI in organizations is hindered by three interconnected problems: data confidentiality, sovereignty — that is, control over where and how data is processed — and security at the moment of direct computations, data-in-use. Traditional encryption mechanisms cover data at rest and in transit, but at the moment of inference — when the GPU actually processes the request — data becomes available inside memory. This is precisely the "open" moment that is the key vulnerability in corporate AI deployments. For industries with strict regulatory requirements — finance, medicine, jurisprudence, public sector — this gap historically has blocked or significantly limited the deployment of AI models from third-party providers.

What is NVIDIA Confidential Computing

The solution implements data protection at the hardware level: even during GPU computations, data remains encrypted and isolated inside a trusted execution environment (Trusted Execution Environment, TEE). Neither the operating system, nor the hypervisor, nor other processes gain access to the information being processed. Key characteristics of the technology:

• Hardware data isolation throughout the entire lifecycle: storage, transmission, and direct computations
• Protection is implemented at the GPU architecture level, not in the software layer on top of it
• Addresses confidentiality, data sovereignty, and security during inference simultaneously
• NVIDIA claims an absence of significant performance losses compared to unprotected computations

The last claim is a principle point of differentiation: traditional hardware security mechanisms created substantial overhead, forcing teams to sacrifice either speed or protection.

Why this is needed for agent systems

The architecture of agent AI fundamentally changes the threat model. An agent that independently formulates chains of actions — calls tools, works with databases, accesses corporate documents — processes sensitive data not once, but throughout the entire multi-step pipeline. In such architecture, protection only at "input" and "output" is insufficient: data must be guarded at every step of the chain. It is precisely for this scenario that NVIDIA positions Confidential Computing as a foundational infrastructure component. For organizations that feared transferring corporate data to AI models from third-party providers, hardware isolation guarantees potentially remove the key objection — without the need to build isolated on-premise infrastructure.

What does this mean

As agent AI systems transition from laboratories to production, hardware security during inference turns from an option into a basic requirement. NVIDIA's approach — protection without performance compromises — may become the standard for corporate AI deployment in regulated industries.