Where Memory and Optics Intersect
Semiconductor experts have been saying “memory is the bottleneck” for long enough that most people following AI infrastructure already know it. Memory is the most expensive and most critical resource in the stack. But here is a question worth sitting with: how much of that expensive memory is actually being used?
According to Microsoft Azure research, up to 30% of DRAM installed in servers is stranded. It has been allocated but sits idle, and it cannot be handed off to another server. The single most expensive component in a server’s cost structure, wasted at roughly one in three units the moment it gets installed.
There is a technology designed to fix this.
CXL, Compute Express Link
CXL is an open interconnect standard that lets CPUs, GPUs, and accelerators connect to memory in a cache-coherent way. It takes memory out of its fixed socket and makes it something that can be dynamically allocated and reclaimed wherever it is needed.
In 2024, the prevailing view was that CXL was dead in the AI era. Then the second half of 2025 hit and the picture shifted fast. Microsoft Azure deployed CXL memory in production. A DRAM shortage drove both Google and NVIDIA to start seriously evaluating CXL as an alternative. The explosion of KV cache demand from LLM inference created a concrete use case for offloading to CXL memory.
By early 2026, the market started moving in earnest. For CXL to expand beyond a single server and scale across an entire datacenter, it has to break through the physical limits of copper. Optical interconnect technology has been advancing fast enough that a path through that wall is now opening up. Run CXL protocol over optics, and memory that was previously trapped inside a server can become a shared resource across the entire datacenter. Companies have already started placing billion-dollar bets on exactly that combination.
You alread know that memory is the bottleneck in AI infrastructure. You probably already know that optics is central to the next generation of interconnects. If you have not yet seen how CXL connects those two things, this article is where to start.
1. What CXL Actually Is
CXL is an interconnect standard that allows CPUs, GPUs, and accelerators to connect to memory in a cache-coherent way. It started as a CPU-centric spec, but from CXL 3.0 onward, GPUs and accelerators can access memory directly without going through the CPU, and devices can share memory with each other.
To understand why this matters, you need to look at how servers have traditionally been built.
In a conventional server, each CPU manages only the DRAM physically attached to its socket. You populate the DDR5 DIMM slots, and that is the server’s total memory. The GPU has its own HBM, the NIC has its own buffer, the SSD has its own controller. Each device’s memory is physically bound to that device.
The problem is the waste this creates. According to Microsoft Azure research, up to 30% of installed server DRAM is stranded. Memory that has been allocated but is not actually in use, and cannot be reassigned to another server. For a hyperscaler, this translates to billions of dollars of DRAM sitting in racks doing nothing.
CXL offers the most direct solution to this problem. There are still real challenges around scheduling, NUMA architecture, and software stack maturity, but at the protocol level, CXL is the only open standard that can take memory out of the server and make it shareable. It operates across three sub-protocols.
CXL.io handles device discovery, initialization, and management over PCIe. Think of it as the “new device detected” mechanism for CXL devices entering the system.
CXL.mem lets a CPU read and write to memory attached to an external device as if it were local DRAM. The key word is load/store: the CPU issues standard memory instructions and gets data back directly, without constructing network packets or waiting for acknowledgment. Latency is higher than local DRAM (tens to hundreds of nanoseconds), but the access model is the same.
CXL.cache lets external devices access CPU-side memory with cache coherency maintained. When a GPU or accelerator caches data from CPU memory, the synchronization with the CPU happens automatically.
For a non-engineer, the simplest way to put it:
CXL is a system where all the chips inside a server share one large memory space and stay in sync in real time about what each of them has modified.
That is what cache coherency means.
CXL devices fall into three types.
Type 1 accesses CPU memory coherently but has no memory of its own. SmartNICs are the classic example.
Type 2 has its own memory (HBM, GDDR) while also accessing CPU memory. GPUs and FPGAs belong here.
Type 3 provides memory for the CPU to map into its address space. This is the hottest category in the market right now. Memory expanders and memory poolers are all Type 3.
Why does Type 3 matter so much? Because in the AI era, memory is the most expensive, most scarce, and most wasted resource in the system.
2. The CXL Spec Timeline: 1.0 Through 4.0
Here is how the spec has evolved.
CXL 1.0 (2019): Intel-driven first release. Built the coherency protocol on top of PCIe 5.0 physical layer. Single-device connections only.
CXL 2.0 (November 2020): The first real game-changer. Switching was added. A single CXL switch could now connect multiple hosts to multiple devices. This is where memory pooling became possible; multiple CPUs sharing one memory pool.
CXL 3.0 (August 2022): PCIe 6.0 doubled bandwidth to 64 GT/s. The critical addition was fabric support. Where earlier CXL was a tree structure (root CPU, branch devices), 3.0 introduced Port-Based Routing (PBR). PBR assigns each device a unique address, and switches forward data to the right destination based on that address, similar to how a postal system works. This enabled flexible topologies like mesh and ring, support for up to 4,096 nodes, and peer-to-peer communication between devices without going through the CPU. Memory sharing was also added: where pooling means one user at a time, sharing means multiple hosts can view the same memory region simultaneously with real-time synchronization.
CXL 3.1 (November 2023): Extension of 3.0. Global Integrated Memory (GIM) enabled direct memory sharing between different servers. The Fabric Manager API was formalized for centralized PBR switch management. Security was strengthened with TSP, providing encrypted isolation for each tenant’s data in multi-tenant environments. The intent was clear: turn the entire datacenter into one unified memory system.
CXL 4.0 (November 18, 2025): The latest spec. PCIe 7.0-based at 128 GT/s, doubling bandwidth again. Three new features were introduced.
First, bundled ports. Multiple physical CXL ports are combined into a single logical connection. Think of it as widening a four-lane highway to sixteen lanes. This enables up to 1.5 TB/s bandwidth over a single connection.
Second, native x2 width. CXL connections use physical lanes; x16 means sixteen lanes, x4 means four. Previously, x2 was a fallback mode used only when errors occurred. Now it is a first-class operating mode. This matters because it improves fan-out, the number of devices a single switch can connect simultaneously. Instead of one sixteen-lane road, you can lay eight two-lane roads. More devices per switch.
Third, four retimer support, up from two. A retimer is a signal repeater. Electrical signals degrade as they travel through copper, and retimers read and re-transmit the signal cleanly. Four retimers substantially increases physical reach. This points directly toward multi-rack CXL configurations.
And there is one line in the spec document: “Optical support with PCIe 7.0.”
Official recognition of optical connectivity. The implications of that one line are covered in detail later.
3. Where CXL Actually Stands Today: A Reality Check
The vision is ambitious. But is CXL a core AI datacenter interconnect right now?
No, not yet.
In March 2024, SemiAnalysis published a piece titled “CXL is Dead in the AI Era.” The core argument: AI shifted hyperscaler priorities. Scale-up GPU interconnect is owned by NVLink and Google ICI. Scale-out networking belongs to Ethernet and InfiniBand. CXL has nowhere to fit.
There is a technical basis for this. CXL/PCIe and Ethernet are built on fundamentally different design philosophies. CXL/PCIe is a load/store memory access protocol. Because it deals directly with memory data, any error halts the system. The PHY layer therefore demands extremely low error rates, and Forward Error Correction (FEC) is used minimally to avoid adding latency. Ethernet is a packet-based network protocol that uses FEC aggressively to correct errors, extracting higher bandwidth efficiency from the same physical channel. The result is that the same number of pins delivers more data over Ethernet. SemiAnalysis pointed out this tradeoff: chip I/O pins are a finite resource, and allocating them to CXL/PCIe at lower bandwidth efficiency is a meaningful cost.
So is CXL dead? Also no. The situation has shifted since 2024.
First, Microsoft Azure has started real deployments. In November 2025, Microsoft announced a CXL memory preview deployment in Azure M-series VMs using Astera Labs’ Leo CXL Smart Memory Controller. The first CSP to do so. Not yet GA, but the world’s largest cloud operator putting real money behind CXL is a signal.
Second, the DRAM shortage brought CXL back into the conversation. The race to build AI infrastructure has made DRAM supply extremely tight. The Information reported that both Google and NVIDIA are actively evaluating CXL as an alternative memory technology. When DRAM efficiency directly affects AI infrastructure economics, CXL’s memory pooling becomes a practical solution rather than a theoretical one.
Third, the KV cache problem blew up. KV cache is the data AI models store in memory to remember what has been said in a conversation. As conversations get longer (as context windows grow), this data grows explosively. A 70B parameter model with 128K context and batch size 32 requires more than 150GB of KV cache, exceeding the 80GB VRAM on an H100. This is exactly where CXL memory creates an opening. Offloading KV cache from GPU VRAM to CXL pooled memory can achieve 4-5x cost reduction versus VRAM and 200-500x lower latency versus SSD, according to various analyses. Marvell’s own testing showed a 4.8x improvement in inference throughput and 82.7% reduction in time to first token with CXL pooling. These numbers come from Marvell’s optimized test environment, and real-world production results will vary by workload.
The positioning is this. CXL is not a primary scale-up interconnect for AI clusters. It is not competing with NVLink. CXL’s role is as a supplementary tool for memory tier expansion and efficiency. But the economic value of that supplementary role runs into the billions, and that makes it impossible to ignore. The CXL memory expansion market is projected to grow from roughly $1.3B in 2025 to $11.8B by 2034 (MarketIntelo, 28.7% CAGR).
Estimates vary across research firms, but the direction is consistent.
4. Hyperscaler CXL Strategy: Google and Microsoft
Google is a founding member of the CXL Consortium and one of the most pragmatic adopters of the technology. Industry observers note that Google has been working with Marvell on custom CXL solutions. At Google’s scale, datacenters hold large volumes of retired DDR4 memory from server refresh cycles. The scenario of using CXL memory expanders to repurpose that DDR4 as cold tier memory (a slower, cheaper memory layer for data that is not frequently accessed) is reportedly under discussion at both Google and Oracle.
Marvell’s Structera X 2404 targets exactly this scenario. It accepts up to twelve DDR4 128GB modules pulled from retired servers, delivering up to 1.5TB of physical capacity. Apply LZ4 compression at line rate, and effective capacity rises to roughly 3TB. Expanding memory capacity without buying new DDR5 translates to hundreds of millions of dollars in annual DRAM procurement savings at hyperscaler scale.
The intersection of Google’s TPU ecosystem and CXL is also worth watching. Google’s seventh-generation TPU, Ironwood, has been announced, and Anthropic has a contract giving it access to up to one million TPUs through its Google Cloud partnership. At that scale, memory efficiency is a central challenge, and CXL memory pooling is a natural part of the solution.
Microsoft has already been covered with the Azure M-series VM deployment of Astera Labs Leo. The goal is to push past the DIMM slot limits on CPU sockets for memory-intensive workloads like in-memory databases. A server with sixteen DDR5 slots maxes out at 16TB with 1TB RDIMMs. CXL memory expanders break through that ceiling.
This is where CXL stands today. The technology is well-defined, the market is opening, and the hyperscalers have started committing capital. But there is one fundamental physical constraint CXL has not yet solved. Copper. And optical solutions are emerging as the next step beyond that constraint.