Microsoft brings confidential AI security to NVIDIA Vera Rubin NVL72

Share

- Advertisement -

Microsoft is taking another step towards making large scale artificial intelligence more secure with its latest technology preview of Azure Confidential Computing on NVIDIA Vera Rubin NVL72. Announced on November 17, 2026, the initiative brings together Microsoft’s cloud security expertise and NVIDIA’s next generation AI infrastructure to protect sensitive information and proprietary models while they are being processed.

The technology preview introduces Confidential Bare Metal Instances (C BMI), giving customers greater control over their software environments while using hardware based security to protect data and AI models during execution. The announcement marks an important development in Microsoft’s broader confidential AI strategy, particularly as organisations increasingly want to run advanced AI workloads on information that cannot be exposed to external systems or infrastructure operators.

AI systems are becoming more useful across industries because they can work with highly sensitive information. Hospitals can use patient records to support clinical decisions, financial institutions can analyse confidential transactions, and government agencies can build assistants that understand individual citizen records. However, these applications also introduce security challenges.

Protecting information when it is stored or transferred is no longer enough. Organisations also need confidence that their information remains protected while AI models actively process it.

Microsoft and NVIDIA target a new level of AI security

Confidential computing is designed to address this challenge by protecting data and models during execution. It uses hardware based security mechanisms to reduce the amount of trust customers must place in cloud infrastructure operators.

This is particularly important for organisations working with proprietary information. A financial institution, for example, may want to use an advanced AI model to identify suspicious transactions without exposing customer records or its internal risk models to infrastructure administrators.

- Advertisement -

Microsoft says confidential computing can provide stronger isolation between customer workloads and privileged infrastructure operators. However, this protection depends on how the complete system is designed. Hardware security alone does not automatically prevent access by authorised applications, tenant administrators or other trusted software components.

Another important part of the architecture is attestation. This process provides cryptographically signed evidence about the identity and security configuration of the environment running a workload.

Organisations can use this information to decide whether a particular environment should receive access to encryption keys or sensitive information. In practice, this means data and model owners can establish policies that allow workloads to run only on approved hardware and software configurations.

These capabilities could also support sovereign AI deployments. Governments and public sector organisations need more than local data storage. They also require control over encryption keys, processing environments, jurisdiction and the organisations responsible for operating their infrastructure.

The collaboration between Microsoft and NVIDIA aims to bring these security protections to increasingly powerful AI systems.

Vera Rubin NVL72 provides the hardware foundation

The NVIDIA Vera Rubin NVL72 is designed as a rack scale computing platform that combines 72 NVIDIA Rubin GPUs, 36 NVIDIA Vera CPUs and high speed interconnect technologies.

- Advertisement -

Its architecture includes NVIDIA NVLink and NVLink C2C, allowing processors and accelerators to communicate across a tightly integrated system.

This matters because large AI models rarely operate on a single GPU. Instead, their workloads are distributed across multiple accelerators, making the connections between processors an important part of the overall security architecture.

NVIDIA’s confidential computing approach is designed to extend hardware rooted protection across the platform, including processors, GPU memory and interconnects.

Microsoft’s Confidential Bare Metal Instances build on this foundation by placing the customer’s host operating system, runtime and workload inside the trusted computing base. In other words, customers retain responsibility for the software stack running on the machine, while the infrastructure operator remains outside the defined trust boundary.

The distinction is important. The host operating system is not automatically treated as untrusted. Instead, its measurements and security configuration become part of the evidence used to establish whether the environment can be trusted.

The architecture combines CPU and GPU memory protection with encrypted physical connections, including C2C, PCIe IDE and NVLink.

- Advertisement -

It also uses Dynamic Root of Trust for Measurement (DRTM), which reduces the trusted computing base by keeping UEFI firmware outside the tenant’s trust boundary. The host operating system and workload remain part of that boundary and must be secured accordingly.

A 397 billion parameter AI model demonstrates the technology

Microsoft’s Ignite demonstration provides a practical look at how the technology works. Rather than showcasing the entire rack scale architecture, the demonstration uses a single Vera Rubin NVL72 node equipped with four Rubin GPUs.

The process begins with collecting attestation evidence from the CPU and GPUs. This includes platform firmware measurements, host kernel measurements and individual GPU firmware and configuration measurements.

Each of the four GPUs provides its own security evidence, allowing the verification process to establish whether the accelerators are operating in Confidential Bare Metal mode.

The collected information is checked using a local verifier. This component validates digital signatures, challenges and the relationship between platform and tenant tokens. It also compares GPU measurements against NVIDIA’s known good values.

Once the verification process is complete, the demonstration moves to the AI workload itself.

Microsoft plans to run a 397 billion parameter model using vLLM with four way tensor parallelism. Each GPU handles one worker, allowing the model to distribute its computation across the four accelerators.

The demonstration also checks the health of the model endpoint, displays the four GPU workers and their memory consumption, and submits an inference request to generate a response.

This is significant because the demonstration connects hardware security verification with an actual large language model workload. It shows how organisations could establish confidence in the underlying computing environment before allowing sensitive AI operations to begin.

However, the demonstration should not be confused with full rack wide deployment. The broader Vera Rubin NVL72 architecture is designed around a rack scale confidential domain, while Microsoft’s Ignite technology preview focuses on a single node with four GPUs. The distinction is important when assessing the current capabilities.

Microsoft plans to expand confidential AI across Azure

Confidential Bare Metal Instances represent the first stage of Microsoft’s wider plans for confidential AI on Vera Rubin NVL72.

The company is also exploring additional deployment models, including confidential virtual machines, confidential containers through Azure Kubernetes Service and Kata Containers, and integration with Microsoft Foundry.

These capabilities are still part of future development rather than the current technology preview.

The broader objective is to give organisations more flexibility in deciding how they deploy sensitive AI workloads. Some businesses may prefer bare metal infrastructure because it provides greater control over the software stack. Others may want virtual machines or container based environments that fit into existing cloud and application management systems.

For Microsoft, the challenge is to extend protection beyond encrypted storage and network connections to the actual execution of AI workloads.

Follow TechBSB For More Updates

- Advertisement -
Emily Parker
Emily Parker
Emily Parker is a seasoned tech consultant with a proven track record of delivering innovative solutions to clients across various industries. With a deep understanding of emerging technologies and their practical applications, Emily excels in guiding businesses through digital transformation initiatives. Her expertise lies in leveraging data analytics, cloud computing, and cybersecurity to optimize processes, drive efficiency, and enhance overall business performance. Known for her strategic vision and collaborative approach, Emily works closely with stakeholders to identify opportunities and implement tailored solutions that meet the unique needs of each organization. As a trusted advisor, she is committed to staying ahead of industry trends and empowering clients to embrace technological advancements for sustainable growth.

Read More

Trending Now