AWS Nitro System: Hypervisor Architecture and Offload Hardware

Overview

The Nitro System is AWS's custom virtualization platform that replaced the Xen-based hypervisor that AWS ran from 2006 through the mid-2010s. It is one of the most consequential pieces of infrastructure engineering in cloud computing history. By disaggregating hypervisor functions — networking, storage, security — onto dedicated hardware ASICs and a minimal software hypervisor, AWS achieved near bare-metal performance for virtualized instances, reduced hypervisor overhead from 10-15% to under 2%, and eliminated an entire category of supply-chain and insider-threat attack by removing the privileged host OS from the trust boundary.

Understanding the Nitro System is essential for anyone reasoning about AWS instance performance, security guarantees, or the future trajectory of cloud hardware.

Prerequisites

Understanding of x86 virtualization (VMX/SVM, EPT/NPT, IOMMU)
Familiarity with KVM and how it works with QEMU
Basic understanding of PCI Express device enumeration
Knowledge of NVMe protocol and RDMA concepts
Familiarity with hardware root of trust and TPM concepts

Historical Context: The Xen Problem

When Amazon launched EC2 in 2006, Xen was the most mature open-source hypervisor available. Xen's architecture uses a privileged management domain called Dom0 that runs a full Linux OS. All device I/O for guest VMs (DomU) passes through Dom0's device drivers via a software data path (split driver model: frontend driver in guest, backend driver in Dom0).

This created two compounding problems:

Performance: Every disk and network I/O operation crossed the Dom0 software path, adding latency and consuming CPU. At scale (100Gbps networking, NVMe storage latency targets), Dom0 CPU overhead was 10-15% of each host's compute capacity — CPU that could otherwise be rented to customers.

Security: Dom0 is a full Linux userspace running on the most privileged software layer of the machine. Any compromise of Dom0 — kernel vulnerability, driver bug, insider access — grants complete control over all tenant VMs on that host. Sharing a hypervisor management OS with thousands of tenants is a fundamental security boundary problem. The 2018 Xen hypervisor CVEs (Spectre/Meltdown variant mitigations) demonstrated how fragile this boundary is.

AWS's solution was architectural: move every non-essential function out of software running on the host CPU and onto dedicated hardware.

Nitro Architecture Overview

+------------------------------------------------------------------+
|                          Host Server                            |
|                                                                  |
|  +-------------------+    +-------------------+                 |
|  |   Guest VM (EC2)  |    |   Guest VM (EC2)  |                 |
|  |                   |    |                   |                 |
|  |  App | OS | VirtIO|    |  App | OS | VirtIO|                 |
|  +--------+----------+    +--------+----------+                 |
|           |                        |                            |
|  +-------------------------------------------+                  |
|  |         Nitro Hypervisor (KVM-based)       |                  |
|  |  <2% CPU overhead  |  written in Rust+C    |                  |
|  |  manages: vCPU scheduling, memory mapping  |                  |
|  +-------------------------------------------+                  |
|                                                                  |
|  +----------------+  +-----------+  +-------------------+       |
|  | Nitro Card VPC |  | Nitro Card|  | Nitro Card for    |       |
|  | (networking)   |  |  for EBS  |  | Instance Storage  |       |
|  | - Security Grp |  | - NVMe/IP |  | - NVMe passthrough|       |
|  | - VPC routing  |  | - encrypt |  | - local SSD       |       |
|  | - ENA driver   |  | - throttle|  |                   |       |
|  +----------------+  +-----------+  +-------------------+       |
|                                                                  |
|  +----------------------------------------------+               |
|  |          Nitro Security Chip                 |               |
|  |  - Hardware Root of Trust                    |               |
|  |  - Firmware signing and verification         |               |
|  |  - Cryptographic attestation                 |               |
|  |  - Physical tamper protection                |               |
|  +----------------------------------------------+               |
+------------------------------------------------------------------+

Nitro Hypervisor

The Nitro Hypervisor is a minimal KVM-based hypervisor. Critically, it does not run a general-purpose host OS. There is no Dom0, no userspace Linux, no shell. The hypervisor exposes hardware to guests and does nothing else.

Key properties: - Written primarily in Rust (memory safety) with C for performance-critical paths - Manages only CPU scheduling (vCPU to pCPU mapping) and memory isolation (EPT page tables) - Does not handle any I/O — all device I/O goes directly to Nitro Cards via PCIe - Overhead: less than 2% of host CPU, compared to 10-15% for Xen Dom0 - Measured attack surface: the entire hypervisor codebase is orders of magnitude smaller than a Linux kernel

Guests see VirtIO devices, which talk directly to Nitro Cards over PCIe MMIO. The hypervisor is not in the data path for I/O at all.

Nitro Cards

Nitro Cards are dedicated ASICs on the PCIe bus of each host. There are multiple specialized cards for different subsystems.

Nitro Card for VPC Networking

Implements the entire AWS networking stack in hardware:

Elastic Network Adapter (ENA) driver interface presented to guest
VPC routing, security groups, and network ACLs enforced in hardware — not iptables, not software
Security Groups are stateful connection tracking implemented on the card
Encapsulation/decapsulation of packets in the VPC overlay network (VXLAN-like)
Bandwidth limiting and burst bucket token enforcement per-instance

Because Security Groups run on the Nitro NIC hardware, they cannot be bypassed by a compromised guest OS. A VM that tries to send packets violating its Security Group rules has those packets dropped in hardware before they exit the server.

Performance: up to 100Gbps on current-generation instances (c5n, p4d, trn1). Packet processing at line rate with microsecond-level latency additions.

Nitro Card for EBS

Implements the EBS data path in hardware:

Guest sees an NVMe device (fast, low-latency protocol vs legacy virtio-blk)
NVMe commands translated to EBS network protocol (proprietary RDMA-like protocol to EBS storage servers)
Hardware encryption: AES-256 encryption/decryption of all EBS data at line rate, no CPU cost
I/O throttling (IOPS and bandwidth limits) enforced in hardware per-volume
All EBS I/O is off-CPU — the host CPU is not involved in the EBS data path

Performance comparison (Xen para-virtualized EBS vs Nitro EBS): - Latency: ~1ms p99 (Xen) → ~100μs p99 (Nitro) — approximately 10x improvement - IOPS: capped at ~80K (Xen) → 256K+ (Nitro, io2 Block Express) - Bandwidth: limited by Dom0 CPU → limited by PCIe/network bandwidth

Nitro Card for Instance Storage (NVMe SSD)

For instances with local NVMe SSDs (i3, i4i, d3 families):

Direct PCIe pass-through of physical NVMe SSDs to guest
No virtualization layer — guest NVMe driver talks directly to physical SSD
Achieves raw hardware performance: i4i.32xlarge: 120GB/s sequential, 3.3M IOPS
Data is not encrypted by default (unlike EBS) — customer responsibility if needed
Data is erased on instance termination (hardware-level secure erase)

Nitro Security Chip

The Nitro Security Chip is a hardware root of trust (RoT) embedded on the server motherboard. It fulfills several security functions:

Firmware integrity: The Security Chip holds cryptographic keys and verifies signatures of all firmware (BIOS/UEFI, Nitro Hypervisor, Nitro Card firmware) before the host boots. Unsigned or tampered firmware will not boot.

Verifiable attestation: The Security Chip generates a cryptographically signed attestation of the current firmware state. This allows AWS to verify that a host is running exactly the expected software stack. It also underpins Nitro Enclaves attestation.

Physical access protection: AWS explicitly claims that even AWS employees with physical access to the host hardware cannot read tenant memory or storage. The Security Chip enforces this by controlling access to encryption keys that are never exposed in plaintext outside the chip. This is a meaningful security claim for regulated industries that historically could not use public cloud due to insider threat concerns.

Supply chain security: Hardware is verified to be unmodified during shipping. The Security Chip detects hardware tampering.

Nitro Enclaves

Nitro Enclaves are isolated execution environments created from a portion of an EC2 instance's CPU and memory, running a separate minimal VM with no persistent storage and no network access (only a local vsock connection to the parent instance).

Architecture:

+------------------------------------------+
|            EC2 Instance (parent)         |
|                                          |
|  +-----------------------------------+   |
|  |         Nitro Enclave             |   |
|  |  - isolated vCPUs (dedicated)     |   |
|  |  - isolated memory (separate EPT) |   |
|  |  - no disk, no external network   |   |
|  |  - vsock: local IPC only          |   |
|  |  - Nitro Security Chip attestation|   |
|  +-----------------------------------+   |
|                                          |
|  [parent instance sees enclave as        |
|   a vsock endpoint only]                 |
+------------------------------------------+

Use cases: - Cryptographic operations: private key operations (signing, decryption) where the key never leaves the enclave. AWS KMS uses enclaves for key operations. - ML inference on sensitive data: run inference on PII or medical data without exposing data to the host OS or application code - Multi-party computation: each party's data enters enclave via attestation-verified channel; no party's raw data is exposed to any other

The parent instance cannot inspect enclave memory, cannot attach a debugger, cannot read its filesystem (there is none). The enclave can request AWS KMS to decrypt data using its attestation document — KMS only decrypts if the enclave's PCR values match a policy. This creates a cryptographic proof that only a specific, verified piece of code ran on the enclave.

Nitro Bare Metal

Bare metal instances (i3.metal, m5.metal, i4i.metal, c5.metal) expose the full physical server to a single customer with no hypervisor layer at all. The Nitro Cards are still present for networking and storage, but there is no virtualization of CPU or memory.

Use cases: - Running your own hypervisor (VMware Cloud on AWS, Nutanix) - Workloads requiring direct hardware access (DPDK networking, GPU without SR-IOV overhead) - Licensing requirements (some enterprise software licenses are per-physical-core, not per-vCPU) - Benchmarking and performance-sensitive applications that need reproducible results

Bare metal instances have identical Nitro Card capabilities to their virtualized counterparts — you still get hardware-accelerated Security Groups, EBS, and ENA networking.

Performance Gains Summary

Metric                  | Xen (pre-Nitro)    | Nitro              | Improvement
------------------------+--------------------+--------------------+-------------
Hypervisor CPU overhead | 10-15%             | <2%                | ~7-8x
EBS p99 latency         | ~1ms               | ~100μs             | ~10x
Max EBS IOPS (single)   | ~80K               | 256K+ (io2 BE)     | ~3x
Max network bandwidth   | 25Gbps             | 100Gbps+           | ~4x
Network packet rate     | limited by Dom0    | line rate          | significant
Security Group impl.    | software (iptables)| hardware ASIC      | cannot bypass
Local NVMe path         | paravirt driver    | PCIe passthrough   | near-native

Influence on Industry

The Nitro System directly influenced competing cloud providers:

Microsoft Azure Maia ASIC: Custom silicon for Azure VMs, offloads networking and security functions. Azure's SmartNIC program uses FPGAs (Project Catapult) and ASICs for similar hardware offload.

Google Titanium: Google's SmartNIC/offload system that implements VPC networking in hardware, analogous to Nitro Card for VPC. Google also uses custom ASICs for their network fabric (Orion) and TPUs for ML workloads.

AMD/Intel: Both CPU vendors have accelerated integrated I/O virtualization (AMD IOMMU improvements, Intel VT-d) and network offload (Intel ACC100/ACC200 for 5G workloads) inspired by hyperscaler requirements.

The industry has moved broadly toward a model where the hypervisor manages only CPU and memory, and all I/O is offloaded to purpose-built hardware. This is now considered best practice for any new hyperscale deployment.

Debugging Notes

If EC2 instance shows degraded EBS performance, check aws cloudwatch metrics for VolumeQueueLength (indicates I/O saturation) and BurstBalance (gp2 volumes deplete burst credits).
ENA network driver issues: check /proc/net/dev for dropped packets. Run ethtool -S eth0 | grep -i err for hardware-level error counters exposed by ENA driver.
Nitro Enclave debugging is limited by design. Use nitro-cli console to read enclave stdout. PCR value mismatches in KMS attestation policies are the most common enclave access failure.
Bare metal instances do not support live migration. If the underlying hardware fails, the instance is terminated (not migrated). Use this in your architecture planning.
Instance store NVMe devices are enumerated as /dev/nvme[0-N]n1 on Linux. On bare metal, these are the raw physical drives — write ahead logs before using.

Security Implications

The Nitro Security Chip's attestation eliminates the class of attacks where a compromised AWS employee with physical data center access could read tenant data. Validated by third-party security firms (NCC Group, CrowdStrike).
Security Groups in hardware mean they are enforced even if the guest OS is fully compromised and trying to send arbitrary packets. This is a meaningful security improvement over software firewall implementations.
Nitro Enclaves enable regulated industries (healthcare, finance) to process sensitive data in cloud without trusting the cloud operator with plaintext data.
Side-channel attacks (Spectre, Meltdown, L1TF) between tenants are mitigated by hardware isolation (separate EPT page tables, SMT disabled for sensitive instances). AWS disables hyperthreading on C5/M5/R5 bare metal instances by default for security.

Performance Implications

The ~2% hypervisor overhead is now paid almost entirely in memory management (EPT walk on TLB miss) rather than I/O. TLB-intensive workloads (large working sets with poor locality) see proportionally higher virtualization overhead.
For NVMe-intensive workloads (databases with local NVMe), choosing instance store (i4i, i3en) over EBS provides significantly lower and more consistent latency. EBS, even with Nitro, has network RTT in the path (~100μs vs ~20μs local NVMe).
100Gbps ENA networking requires placement groups (cluster placement group for low-latency, 10μs latency between instances in same placement group vs 100μs across AZ).

Failure Modes

Nitro Card hardware failure: AWS handles transparently via live migration (host-level fault detection triggers migration to healthy host, typically completes in <1 second with minimal packet loss).
Nitro Security Chip failure: host taken out of service. Customers see instance reboot/migration.
EBS multi-attach + Nitro: io1/io2 volumes support multi-attach to up to 16 instances. Corruption is possible if multiple instances write without cluster-aware filesystem (GFS2, OCFS2). The Nitro hardware does not arbitrate concurrent writes.
Nitro Enclave vsock disconnect: if the parent instance reboots, the enclave is destroyed. Enclaves have no persistence.

Modern Usage

As of 2024, virtually all new AWS instance families run on Nitro. Xen instances (older T2, M3, C3 families) are being phased out. AWS recommends migrating any remaining Xen instances to Nitro equivalents for performance and security.

Graviton3 (ARM-based) instances combine Nitro with custom AWS CPU design, delivering better price-performance than x86 for most workloads except those with x86-specific optimizations (certain database workloads, legacy compiled binaries).

Future Directions

Nitro v5 (speculative, based on patent filings and re:Invent hints): increased offload of hypervisor memory management functions to hardware, targeting sub-1% overhead.
Confidential computing expansion: Nitro Enclaves expanding to support multi-party computation protocols at hardware speed.
CXL (Compute Express Link): next-generation PCIe-based interconnect enabling shared memory pools between hosts — likely the next substrate for Nitro Card evolution.
Network performance: 400Gbps (4x current) instances in development for HPC and ML training workloads.

Exercises

Create a Nitro Enclave on an m5.xlarge instance. Write a simple Python application that runs inside the enclave and uses KMS to decrypt a secret, accessible only when the enclave's PCR values match a policy you define.
Benchmark EBS vs instance store latency using fio on an i4i instance. Document the difference in p50, p99, and p999 latency.
Use ethtool -S eth0 on a Nitro instance to observe ENA hardware counters. Simulate network saturation and observe queue depth counters.
Review AWS's public security whitepaper on the Nitro System. Identify which threat model claims are verifiable by a third party and which require trusting AWS's assertion.
Compare the Nitro Security Chip's trust model to Google's Titan chip and Microsoft's Pluton. What guarantees are shared and what differs?

References

AWS re:Invent 2017: "AWS Nitro System: Next-Generation EC2 Infrastructure" (CMP332)
AWS Security Whitepaper: "The Security Design of the AWS Nitro System" (2022)
Brendan Gregg: "AWS EC2 Virtualization 2017: Introducing Nitro" (brendangregg.com)
NCC Group: Third-party security assessment of the Nitro System (2022)
AWS re:Invent 2019: "Nitro Enclaves: Isolated VMs for Processing Confidential Data" (CMP331)
Intel VT-d Specification: https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf
QEMU/KVM vs Nitro comparison: lwn.net coverage of KVM Summit 2019