NVIDIA disclosed NVHBM on August 26, 2026. It said it would put a custom memory controller into the HBM base die and let multiple memory providers supply the same NVHBM specification. Versus standard HBM4E it cited up to 30% higher stack bandwidth, up to 15% lower HBM power and up to 25% more XPU compute die area.

People outside the semiconductor industry sometimes read the name and assume NVHBM is a new form of HBM.

It is not.

The industry calls the HBM we normally refer to standard HBM, or sHBM, and calls the products whose base die is tailored to a customer’s requirements cHBM, or custom HBM. Research and development on cHBM has been under way for years. NVHBM is simply the cHBM NVIDIA uses.

The reason NVHBM deserves attention is not that it is technically special.

It is that NVIDIA defines its own cHBM specification, namely the controller and interface that sit in the base die, and intends to have multiple memory suppliers build to that same structure. In other words, the domain memory vendors used to design together with their customers moves wholesale into NVIDIA’s platform layer. That is where its significance lies. Section 4 onward covers this in detail. First, though, we look at why cHBM emerged and what it means for memory vendors.

Subscribe now

2. The Faster HBM Gets, the More the XPU Pays

HBM has raised bandwidth in two ways: widening the interface between XPU and HBM, and running each signal faster. The interface went from 1,024-bit in HBM3E to 2,048-bit in HBM4, and pin speed keeps climbing.

The problem is that HBM does not bear the cost of higher bandwidth alone. A wider interface requires more PHY and I/O circuitry on the XPU as well, and consumes more die edge and more interposer routing. Higher signal rates add power and signal-integrity burden on top.

In an AI accelerator this cost matters more. The XPU’s leading-edge silicon is an expensive resource that should go to compute and cache wherever possible. Yet the more HBM bandwidth rises, the larger the share of that area and power budget the memory interface takes.

So the problem after HBM4 is not solved by making DRAM itself faster.

What is needed is

an architecture that delivers the same bandwidth using less XPU area and power.

That is where the value of an advanced-logic base die and a custom interface comes from.

Subscribe now

3. cHBM Reduces That Cost While Creating a New Source of Revenue and Profit

The main disclosed direction for cHBM is to move part of the memory-related logic out of the XPU and into an advanced-logic base die, and to redefine the XPU-to-HBM interface to fit the system. That frees some of the area the HBM PHY and controller occupied on the XPU, and lets the recovered silicon and power budget go back to compute or cache.

For memory vendors this creates a new value pool. In standard HBM, DRAM die, stacking, test and qualification accounted for most of the product value. In cHBM, base-die design, controller and PHY IP, verification, NRE and customer-specific optimization can be added on top.

Micron’s stated expectation that a customized HBM4E base logic die will carry a higher gross margin than standard HBM4E points the same way. cHBM is not simply faster HBM. It is a product through which memory vendors can sell more engineering value.

Subscribe now

4. NVHBM Can Redistribute cHBM’s Value Capture Toward NVIDIA

Throughout this article, custom HBM whose architecture is designed anew for each customer is called bespoke cHBM, and the model in which a platform owner such as NVIDIA sets the spec and multiple suppliers build the same structure is called platform-defined cHBM.

In bespoke cHBM, base-die architecture and qualification can differ by supplier. Once controller, PHY and base die have been designed with one memory vendor, switching to another requires redesign and revalidation. That switching cost is an important reason memory vendors can defend custom design premium and customer relationships.

NVHBM changes this structure. Instead of memory vendors building different cHBM architectures with each customer, NVIDIA defines the controller and interface and multiple memory providers supply the same NVIDIA implementation.

That also changes where the new logic value in an HBM stack accrues. In bespoke cHBM, a memory vendor can go deeper into base-die design, controller, PHY and customer-specific logic and capture proprietary design value. Under NVHBM, NVIDIA defines much of the architecture, so memory vendor differentiation is likely to shift toward DRAM performance, stacking, yield, power and manufacturing execution.

Multi-sourcing reinforces the shift. When several suppliers meet the same NVHBM specification, NVIDIA can qualify suppliers against more common criteria and allocate volume accordingly.

That does not mean memory vendors become fully interchangeable or that prices automatically fall. HBM still differs meaningfully by supplier in capacity, yield, speed, power and thermal behavior, and supply itself remains constrained.

Even so, a common architecture works in the direction of lowering supplier-specific switching barriers. The net effect is that NVHBM increases NVIDIA’s influence over custom memory architecture and supplier selection, and can move part of the design value memory vendors could capture in bespoke cHBM toward the platform owner.

Subscribe now

5. What NVIDIA Is Optimizing Is Not HBM but the Whole AI System

Taken separately, density despec and NVHBM look like HBM optimization. The scope NVIDIA is drawing is wider than that: binding compute, memory and interconnect into a single AI factory architecture rather than treating them as separate components.

Density despec lowers memory capacity per accelerator but lets a constrained DRAM supply produce more HBM stacks. All else equal, moving from 12-Hi to 8-Hi raises the number of stacks buildable from the same DRAM die output by a theoretical factor of 1.5. At the same time, demand for base-die units, one per stack, rises.

In NVHBM, NVIDIA directly defines the architecture between XPU and HBM. Outside that, NVLink and NVSwitch handle Scale-Up while Spectrum-X and InfiniBand handle Scale-Out. NVIDIA has added Scale-Across, Context Memory and BlueField-4-based Scale-In to frame the AI factory as five infrastructure pillars. As rack and data center scale grow, optical connectivity grows in importance alongside them.

Add NVLink Fusion and NVIDIA does not need to build every piece of compute silicon itself. Custom XPUs and CPUs are allowed, while memory, the scale-up fabric and the MGX rack architecture still connect into the NVIDIA platform. Amazon Annapurna Labs being named the first NVHBM collaborator, with NVLink Fusion support signaled from Trainium4, shows this strategy is aimed at custom XPUs beyond NVIDIA’s own GPUs.

What NVIDIA is expanding is not just GPU share but the scope of the AI system whose architecture it defines. From the GPU out to HBM, rack-scale interconnect, networking, context memory and infrastructure access, that boundary keeps widening. NVHBM is one step in bringing the memory layer inside the NVIDIA platform.

Subscribe now

That is how the market reads NVHBM today: NVIDIA takes the spec and keeps widening the scope it defines. Most commentary stops here.

So are memory vendors simply losing ground? No.

This is not the last round. Memory vendors are already preparing for the next one, and in that round there is one position left where what NVIDIA has taken can be won back. It is a domain that cannot be pinned down in a specification, which is exactly why it resists commoditization. Look at where Samsung and SK hynix are putting people and money right now and the direction becomes visible.

The same shift also does not land on all three memory vendors the same way. Set a few conditions, run the numbers per accelerator, and a variable emerges that moves profit far more than design ownership itself. How sensitive that variable is, and how far down the value chain the change carries, is what the rest of this analysis quantifies.

The era of reading HBM through bit growth is ending. What to watch and what to verify in the next cycle starts here.

6. Memory Vendors Respond by Splitting Volume and Design Value

Read more

This material is general research opinion based on public information and is not investment advice tailored to any individual. Future contracts, capacity targets, ARR and secured power are conditional figures, and actual outcomes may differ depending on financing, permitting, utility schedules, supply chains, customer acceptance and demand.

A neocloud, put simply, is a company that rents out GPUs for AI.

AI companies need large scale compute to build models, but securing the latest GPUs and building data centers takes enormous capital and time. Neoclouds build the power, the data center, the NVIDIA GPUs, the high speed network and the operating software first, then rent that stack to customers. Instead of building their own AI factory, customers buy compute capacity that is ready to use.

The reason to look at this industry now is that neoclouds are growing from a stopgap for temporary GPU shortages into an independent supply layer of AI infrastructure.

On its FY27 second quarter call, NVIDIA projected that installed capacity at its neocloud partners would grow from roughly 3 GW at the end of 2025 to 8 GW at the end of 2026. Beyond AI startups and enterprises, hyperscalers that operate their own data centers are also using AI clouds to supplement capacity they lack.

This layer may also persist after GPU supply normalizes. GPU generations turn over quickly, but grid connections, substations, cooling and data center sites can be reused by the next generation of equipment. Even when customers change, a standardized NVIDIA cluster can be redeployed to other AI workloads.

This article is a technology and investment guide for anyone considering the three listed neocloud names, CoreWeave, Nebius and IREN.

The three sit in the same industry, but how they make money and what risks they carry are different.

How much of the power and data center they own directly, what kind of capital they use to secure GPUs, and whether the numbers they announce are still secured megawatts or capacity that has already passed customer acceptance all change the investment call.

The article first explains what a neocloud is and what a megawatt means, then compares the technical strengths, economics, capital structure and execution risk of the three companies, and finally sets out how to approach each name at current prices.

Subscribe now

A neocloud rents out AI compute, not just GPUs

What the customer buys is not the GPU itself but an AI computing environment that is ready to use. A neocloud bundles power, the data center, thousands of NVIDIA GPUs, a high speed network and operating software, and rents that bundle out.

Capability is not determined by GPU count alone.

Thousands of GPUs have to behave as a single computer. If the network congests, if cooling wavers, or if the software that splits the work stalls, even expensive GPUs fail to earn properly. What matters is less how many GPUs a company holds than how reliably and how quickly it finishes customer jobs.

Where general clouds such as AWS, Azure and Google Cloud sell storage, databases and business software as well, neoclouds concentrate on AI compute. They can install the latest GPUs faster, but capital intensity is high and dependence on a small number of large customers is easy to fall into.

Subscribe now

A megawatt is the size of the power a data center can draw

MW (a megawatt) is the amount of power a data center can use at a point in time. One MW is one million watts. Used continuously for a year, that consumes 8,760 MWh. GPU models keep changing, but grid connections and cooling capacity do not expand easily. That is why neoclouds describe the scale of their business in megawatts.

The same megawatt figure can be measured on different bases.

Facility MW is the power the whole site draws, while IT MW is the power that actually reaches GPUs, CPUs, storage and the network. Cooling and power conversion take the rest, which creates the gap.

Note that if a company does not state which basis it used, two companies cannot be placed side by side. In other words, what matters is less the megawatt figure itself than the basis and the method used to compare.

MW 101

Moving from secured power to a completed data center, to IT power delivered to servers, to capacity that has passed customer acceptance, the megawatts that can actually be used shrink at every step. What remains then meets a contract and becomes operating ARR, and passes through utilization and costs to become shareholder cash flow. The further along the chain, the smaller the number but the higher the certainty that it turns into revenue.

CoreWeave leads on installation and bring-up speed and on large scale production operating experience.

Nebius aims to raise revenue per MW by selling software alongside compute.

IREN controls its own power and land, but still has to complete facilities on schedule and pass customer acceptance.

Subscribe now

Secured megawatts and earning megawatts are not the same thing

The megawatt figures companies announce are best read in three stages. The first is power secured so that it can be developed at some point. The second is IT power that will actually be delivered to the data center and the GPUs. The third is operating power that has passed customer acceptance and started earning. The earlier the stage, the larger the number and the greater the uncertainty.

The megawatts the three companies announce also point to different stages.

Placing the three companies’ megawatts on one line therefore risks a wrong read of how far each business has actually progressed.

When secured power actually arrives differs by site and by utility contract. Rather than relying on a target year, three things are worth checking.

First, when the utility contract was signed and what delivery start date it specifies.

Second, whether the site is already receiving electricity or is still building its substation.

Third, whether the quarterly increase in active power a company reports is running at a pace consistent with the target schedule.

Dividing the quarterly increment by the remaining period, as with CoreWeave adding roughly 500 MW in a single quarter, gives a rough sense of whether the target is achievable.

Subscribe now

What NVIDIA has actually committed to the neoclouds

As described above, a neocloud has to commit large capital to power, buildings, cooling and GPUs before any revenue starts. Several years pass between securing power and customer acceptance, and during that period there is no revenue while interest and depreciation accrue.

NVIDIA intervenes in that window in three ways. It buys compute capacity from the neoclouds directly, it connects financial institutions to widen funding channels, and it attaches guarantees to some projects. The structure is one where NVIDIA buys back services run on the very GPUs it sold. For the operator, a minimum level of demand exists even during the construction period with no revenue, and that contract supports borrowing at a lower rate.

The market has taken this as a sign that the funding problem for neoclouds has been solved.

It is therefore worth checking, against the filings, how much NVIDIA actually guaranteed, to whom, and on what terms.

In the 10 Q filed on 26 August, NVIDIA’s maximum guarantee exposure was $108.5B.

Of that, $105B relates to the roughly 4.25 GW project SB Energy is developing in Ohio, connected to an OpenAI affiliate, and is unrelated to the neoclouds. AI cloud related guarantees total $3.5B.

That figure, however, is disclosed as an aggregate guarantee for select AI cloud partners, and public disclosure alone does not establish that it is allocated to CoreWeave, Nebius and IREN.

Separately from guarantees, NVIDIA agreed to buy cloud capacity from these operators directly. The commitment totals $36B, typically over six years. Where a guarantee is a ceiling against default, this is a capacity purchase commitment that converts into revenue when the conditions are met. That said, if operators sell that capacity to other customers, NVIDIA’s purchase obligation falls accordingly, so the full $36B is not fixed revenue.

The third form of support is the funding channel. On 10 August, NVIDIA announced a compute financing platform with Apollo, BlackRock, Blackstone, Brookfield, Goldman Sachs and KKR aimed at mobilizing over $500B. The structure supplies capital against AI infrastructure as collateral, and the market read it as a sign that the neocloud funding constraint was easing.

It is, however, a memorandum of understanding with large financial institutions rather than committed capital, and individual projects still have to be underwritten separately. NVIDIA may support the residual value of some GPUs, but there is no disclosure guaranteeing the price of all equipment.

Subscribe now

Even with NVIDIA’s help, power and bring-up remain each company’s own work

The risk NVIDIA has removed is concentrated in funding and early demand. Bringing power in, completing the data center, stabilizing the GPUs and passing customer acceptance all remain with the operator. This is where results diverge even when the three companies buy the same NVIDIA equipment and receive the same support.

The first source of that remaining risk is equipment delivery. NVIDIA stated in its most recent 10 Q that Rubin and Blackwell supply is constrained and that the complexity of data center systems can cause production delays and lower yields. Even with funding secured, racks arriving late push the schedule back by the same amount.

Revenue does not begin the moment the racks arrive either. A new generation cluster has to stabilize power and cooling, isolate faulty nodes, and verify network and storage before customer workloads confirm performance and stability. During this bring-up period a gap opens between installed MW and the MW a customer can actually use, and the revenue start date slips by that gap.

Bring-up time differs by company and shrinks from batch to batch within the same company, because the failure modes and inspection procedures found in the first cluster can be reused in the second and third. That repeated experience is what separates how fast secured power turns into revenue.

So where do the three neoclouds stand now, and what should be watched most closely?

That covers the basics of what a neocloud is.

Below, the same standards are applied to the three companies. The article then sets out why CoreWeave’s RPO, Nebius’s ARR and IREN’s contracted ARR cannot be compared on the same yardstick, breaks down what survives beyond a single GPU generation across five axes, and closes with a detailed investment view and approach for each name.

Read more

I have been analyzing Marvell closely since the beginning of this year and have shared a positive view across several articles.

To start with the conclusion, I remain very positive on Marvell.

This piece looks in detail at how far the five checkpoints from Part 2 have been validated, which parts can still remain as upside for the stock, and where the valuation stands. For readers who want a fuller picture of the company itself, the earlier articles below are the place to start.

Disclaimer

This note is a personal research piece written for information and industry analysis. It is not a recommendation to buy or sell any security. The author may hold positions in the securities discussed and may transact in them in the future. Revenue estimates, valuation, scenarios, and architecture interpretations that are not attributed to company guidance or public disclosure are the author’s judgment and may differ from actual outcomes. The system architecture read-through connecting NVIDIA, SK hynix, and Marvell does not imply any disclosed three-party co-development agreement.

Subscribe now

1. Part 2 checkpoints: four of five passed

Part 2 laid out five checkpoints for the May 27 print. Four have passed and one is partly confirmed. The largest change is that the Interconnect earnings contribution arrived faster than expected.

Part 2 update

The upside case at the time centered on next-generation Custom Silicon ASP. In practice the FY27 earnings revision came from Interconnect first. The larger Custom growth contribution remains ahead in FY28.

Subscribe now

2. FY27 = Interconnect, FY28 = Custom

Marvell FY26 revenue was $8.195B.

The current company outlook is roughly $11.5B for FY27 and roughly $16.5B for FY28. Incremental revenue is about $3.3B in FY27 and a further $5.0B in FY28.

The FY27 Data Center growth guide is about 50% and the Interconnect guide is above 70%. The Q1 investor presentation put Custom growth above 20% in FY27 and above 2x in FY28. FY27 earnings leverage sits with Interconnect, while the FY28 growth mix shifts toward Custom.

Subscribe now

3. Optical, Interconnect and Switching: the most de-risked FY27 driver

Read more

Rubin Ultra’s HBM configuration is being reviewed downward, from HBM4E 12-Hi toward HBM4 8-Hi. Supply constraint is the direct cause, but supply alone does not make the choice work. It works because a wider scale-up domain and software tiering absorb the loss of per-GPU memory capacity at the system level.

That direction leads to optics and memory disaggregation. A Nature Electronics review published on August 20 lists an SK hynix researcher as a corresponding author. Marvell is co-developing scale-up networking and silicon photonics with NVIDIA, and CXL-based CMM-Ax with SK hynix.

Disclaimer This article is written for informational purposes and does not recommend buying or selling any security. The 8-Hi configuration for Rubin Ultra is unconfirmed information based on industry checks, and the final specification may change. The yield and revenue figures in this article are scenario sensitivities, not company guidance or shipment forecasts.

1. The stack count fell because the GPU domain grew

On August 4, TrendForce said NVIDIA is evaluating several lower-capacity configurations alongside 12-Hi HBM4E and that the final specification has not been set. The Information reported prototype testing of 192GB and 256GB configurations, and SemiAnalysis sees the main SKU moving down to HBM4 8-Hi 192GB while compute performance holds and memory bandwidth rises slightly.

As someone who has covered this issue before, what I hear from the field is that the major memory makers are now running daily meetings with the customer on the basis of HBM4 8-Hi as the main Rubin Ultra configuration and are entering spec decisions on that footing. Some sources mention a 4-Hi configuration, but there is no verifiable basis for it, so it is not treated here.

All of this points to one thing.

On Rubin Ultra, pushing HBM density as high as it will go is no longer the only path to system optimization.

Why is this happening?

The first reason for the spec change is supply.

DRAM is likely to stay tight through 2027, and HBM4E still carries schedule uncertainty in qualification at the performance level NVIDIA is asking for. Twelve-layer stacking also costs more in throughput and yield than eight.

The second is system structure.

The field view is that on Rubin Ultra, bandwidth and data movement are becoming a larger constraint than memory density. In other words, the system has entered a regime where improving how data moves contributes more to performance than adding capacity.

Package-level constraints compound this.

Adding HBM stacks around the GPU raises package area and interposer-edge I/O cost. Raising the layer count inside the same stack instead increases stacking process complexity, yield loss, thermal load and package complexity. The higher HBM density goes, the more the system has to pay for each additional unit of capacity.

A wider scale-up domain, by contrast, lets some of that capacity move to peer HBM and to memory tiers outside the package. Where the gap narrows between the value of additional local capacity and the package cost of obtaining it, improving data movement can serve system efficiency better than pushing memory density further.

8-Hi is a response to supply constraint. What makes it workable is a system structure that leans on scale-up bandwidth and memory placement instead of pushing local memory density higher.

The software and memory placement techniques that keep performance intact at HBM4 8-Hi 192GB were covered separately in “What Does Rubin Ultra’s 8-Hi HBM Mean?” on August 10. Details are here.

I see two changes following from this. Optics widens the physical reach of the NVLink domain, and memory disaggregation narrows the range of memory capacity that must sit inside the GPU package.

SK hynix and NVIDIA each gave their own reasons for the CPO paper and for the $2B investment in Marvell. The public statements from NVIDIA and Marvell cite NVLink Fusion, custom XPU, scale-up networking and silicon photonics.

What I do is read both moves as signals from the same system-level data movement problem. NVIDIA, SK hynix and Marvell: the sections below look at the picture the three are drawing, and at what this change could mean for memory makers and for HBM.

2. From NPO to CPO: optics moves from rack-to-rack links toward package I/O

The difference between NPO and CPO is how close the optical engine sits to the ASIC. NPO keeps the optical engine outside the XPU or switch package but places it as close as possible on the board. Marvell cites the advantage that optics can be adopted without redesigning the XPU package.

CPO puts the photonic die and the electronic die inside the same package or interposer. That shortens the distance high-speed electrical signals travel outside the package, which helps bandwidth density and energy efficiency, but it raises the burden of thermal management, optical alignment, package yield and reliability.

SemiAnalysis reads the optical technology used in Rubin Ultra NVL576 as an NPO structure, and my previous article was written on that basis. Public disclosure and industry reporting should be separated here, though. NVIDIA has disclosed that NVL576 binds eight racks of 72 GPUs into a single 576-GPU NVLink domain and uses direct optical connections between racks.

NVIDIA’s public technical material does not state whether the packaging topology of that optical link is NPO or CPO.

What it does provide is the technical basis for why optical I/O has to move closer to the package after NPO, and how the next step leads to CPO and the photonic interposer.

For a detailed discussion of the technological evolution and market dynamics of NPO and CPO, please refer to the article below.

The Nature Electronics review with an SK hynix corresponding author sets out the physical background of this shift. In the paper, compute throughput has tripled roughly every two years, while over the same two-year cycle interconnect bandwidth advanced only about 1.4-fold. Even when HBM lifts bandwidth inside the package, resistive loss and capacitive loading on copper traces grow again once data crosses the board and cables. At high rates, equalization, retimers, DSP and FEC correct for that while adding power and latency.

The integration roadmap in the paper points the same way. From pluggable and on-board optics through 2.5D CPO, 3D TSV, hybrid bonding and monolithic integration, the optical conversion point moves closer to compute at each step. 2.5D is the compromise between manufacturability and performance, while 3D delivers higher bandwidth density and lower energy per bit at the cost of thermal and bonding yield.

On that reading, the NPO reported for NVL576 is better understood as an intermediate step that lowers package redesign and serviceability burden, rather than the end of the optical transition. The move to CPO gains economic footing as bandwidth density and I/O power become the binding constraints.

The energy and bandwidth density figures are the representative values the paper gives for each integration method. The pluggable column is on a transceiver basis.

The significance of this paper is that a technical review with a memory maker involved extends the AI system bottleneck from HBM bandwidth itself to data movement between compute and memory, and sets out a direction that pulls optical I/O closer to the package.

I read this as the key technical inflection in the memory and optics roadmap that SK hynix, Marvell and NVIDIA are drawing.

3. Disaggregation: extending local HBM capacity into a system memory domain

Local HBM remains the fastest memory tier. On the field checks used in my earlier work, Rubin’s local HBM bandwidth is about 21 to 22TB/s, while NVLink 6 delivers roughly 3.6TB/s of GPU-to-GPU bandwidth per GPU. Another GPU’s HBM does not replace local HBM. What changes is how much of the weights and KV cache must sit in one GPU’s HBM.

NVL576 implements that change inside the scale-up domain first. Active weights and hot KV stay in local HBM, while distributable weights and KV spread to peer HBM. Software hides remote memory latency by prefetching what is needed or by sending the token to the GPU that holds the data. One step further and capacity itself moves outside the GPU package, which is memory disaggregation.

This is not a structure that removes local HBM. Latency-sensitive data stays in HBM, and only data that needs capacity and reach moves to a larger memory tier.

The long-term structure in the Nature Electronics review takes this further. Fig. 4 shows XPU pool and memory pool separated and linked by a photonic interposer. The paper does not treat this as a finished architecture. It names an optical memory interface that preserves coherence while lowering latency, low-power photonic switching and multi-hop signal regeneration as problems still to be solved. The drawing is therefore a research direction for the optical architecture that separating memory and compute would require, not an SK hynix commercial product roadmap.

XPU pool and memory pool are separated and linked by a photonic interposer and optical fibre. This is the long-term direction the paper sets out, not a commercial product roadmap.

Subscribe now

SK hynix’s shipping products move on electrical disaggregation first. CXL pooled memory lets multiple hosts share memory, and CMM-Ax attaches compute to CXL memory so that long-context inference can handle a KV cache larger than GPU HBM. Disclosed in August 2026, CMM-Ax is a co-developed product combining Marvell Structera A with SK hynix DRAM and software stack. The actual link between SK hynix and Marvell shows up first in the CXL memory tier, not in optical memory.

This widens the scope of SK hynix’s business from the DRAM die itself to a broader memory architecture. HBM and custom base die sit in the local fast tier, CXL and CMM-Ax in the external capacity tier, and the photonic memory pool in the paper in a longer-dated optical tier.

Disclosed products and one paper are not enough to confirm a transition into a system-level memory vendor, but the direction is visible: the scope that has to be co-designed with the customer widens from the memory die to the controller, base die, packaging and memory placement. If that structure does reach commercial deployment, the value-added and customer stickiness SK hynix captures could exceed what selling bits alone provides.

Marvell is extending its portfolio into the next layer. It secured Photonic Fabric through the Celestial AI acquisition in February 2026, and at FMS in August it disclosed a structure in which multiple XPUs and racks reach a shared warm KV cache tier of up to 32TB over an optical fabric within 50m. The 32TB, the 50m and the 2 to 3x token throughput are vendor targets set by Marvell. They are not figures that indicate commercial deployment.

What deserves attention here is that the optical structure Marvell presents and the drawings in this Nature Electronics review share the same layer structure.

Fig. 4c of the paper shows a multi-chip module cross-section with ASIC, CPU, HPU and HBM chiplets placed together on a photonic interposer with embedded optical waveguides.

The Photonic Fabric Module Marvell obtained through the Celestial AI acquisition is a 2.5D package that places an ASIC, HBM cache and DDR5 channels together on an active photonic interposer.

On Marvell’s disclosed specification it supports up to eight DDR5 DIMMs, 72GB of HBM cache and 7.2Tbps of optical bandwidth. If Fig. 4b of the paper is a structure that links XPU pool and memory pool through a photonic interposer, Marvell’s memory module and PF-NIC configuration is closer to an implementation of that structure at module level, arriving first.

The problem definitions overlap as well.

The paper points to I/O density at the die periphery limiting channel count and routing flexibility in 2D configurations, and in Fig. 4d it sets out vertical connection between stacked PIC and EIC through optical TSVs. Marvell instead takes optical I/O out from the middle of the die rather than the die edge. The implementation differs, the constraint being solved is the same.

That similarity is an observation at the structural level. The inclusion of Celestial AI’s Photonic Fabric in the paper’s table of industrial platforms is verifiable, but there is no announcement that SK hynix and Marvell are co-developing in the optical memory space. The confirmed contact points between the two remain custom HBM and CMM-Ax. Sharing a structure and having a joint product are two different things.

In parallel, NVIDIA and Marvell announced co-development of NVLink Fusion compatible scale-up networking and silicon photonics in March 2026, and NVIDIA invested $2B in Marvell. Marvell sits at the point where optical I/O for XPU scale-up and CXL-based memory disaggregation meet.

NVIDIA offsets local HBM density with system-level scale-up, SK hynix extends the memory tier outside HBM, and Marvell supplies the electrical and optical interconnect both structures require. The link between the three companies is built out of that division of roles. Moving right, verifiable evidence thins out and the content moves closer to a research direction.

4. HBM implications: bandwidth, yield and memory hierarchy matter more than stack height

If the 8-Hi transition is confirmed as the final specification, HBM capacity per GPU clearly falls.

Against the same GPU shipment volume, that is a mix down in HBM content. Under the current supply constraint, though, the per-GPU bit decline cannot be applied directly as a decline in HBM market revenue. Lowering stack height raises the number of stacks that can be built from the same DRAM and stacking capacity, and NVIDIA can absorb that into additional GPU shipments.

HBM does not lose its role. Local HBM stays the fastest tier, and part of the additional capacity spreads into peer HBM, CXL memory and optical shared memory.

In the sensitivity used in my earlier work, moving from 12-Hi to 8-Hi gives up to a 1.5x effect from stacking time and a 1.084x improvement in stack yield under a 98% per-core-die yield assumption.

Multiplying the two puts the effective output ceiling at about 1.63x. Assuming all of that volume is absorbed into additional NVIDIA GPU shipments, total HBM revenue lands at about 81.5% of plan if 8-Hi is priced at 50% of the planned HBM4E, and about 98% at 60%. These are sensitivities to stack count, not shipment forecasts.

The competitive variables change too. In the regime where 12-Hi and 16-Hi stack height was itself the axis of differentiation, high-stack qualification and stacking technology carried a large premium. If the life of 8-Hi extends, yield, capacity, supply stability and base die functionality carry more weight. SK hynix expanding custom base die and a CXL-based memory tier together sits against the same change.

For Marvell the read-through runs the other way. Keeping accelerator utilization intact while lowering local HBM capacity raises the importance of scale-up bandwidth, optical reach and external memory tiers. That is why NVLink Fusion and silicon photonics work with NVIDIA, CMM-Ax with SK hynix, and Photonic Fabric through the Celestial AI acquisition connect inside one portfolio.

As optics moves closer to compute, from pluggable through NPO, CPO and the photonic interposer, supply chain value moves with it. Volume growth in optical engines and laser sources comes first, and silicon photonics PICs, drivers and TIAs, fiber coupling, 2.5D interposers and 3D bonding gain weight after that. If CPO and optical memory reach high-volume production, the beneficiary set is not limited to light source makers but widens across photonics and advanced packaging.

Subscribe now

This is why I keep calling for a second round in optical investing, and why I keep saying that the start of the optical theme is the reason the memory theme is running. My belief is that only by following this technology and market shift can you hold real conviction and a real thesis in semiconductor, memory and optical investing from here.

Sources and verification

1. TrendForce, reporting on Rubin Ultra HBM configuration review, Aug. 4, 2026. Confirms evaluation of multiple configurations including 8-Hi HBM4E, 12-Hi HBM4 and 8-Hi HBM4 alongside 12-Hi HBM4E, and that the final spec is unset.

2. The Information, reporting on lower-capacity Rubin Ultra prototypes, 2026. Confirms prototype testing of 192GB and 256GB configurations.

3. SemiAnalysis, Vera Rubin analysis, 2026. Confirms the main configuration moving to HBM4 8-Hi 192GB with compute held and memory bandwidth slightly higher, and the view on the rack-to-rack optical link.

4. Damnang Research, What Does Rubin Ultra’s 8-Hi HBM Mean?, Aug. 10, 2026. Basis for the Section 1 field checks, the NVL576 and NPO structure, the software hierarchy and the HBM revenue sensitivity.

5. Kim, B. et al., Co-packaged optics for high-performance computing and artificial intelligence, Nature Electronics (2026). Basis for electrical interconnect limits, the CPO integration hierarchy, and the photonic interposer and XPU/memory pool architecture.

6. NVIDIA, “NVIDIA Vera Rubin POD: Seven Chips, Five Rack-Scale Systems, One AI Supercomputer,” 2026. Confirms the 8-rack, 576-GPU NVLink domain and the direct optical rack-to-rack connection in NVL576.

7. Marvell, “Scale-up Network Solutions for AI Infrastructure,” 2026. Confirms NPO and CPO product positioning and packaging differences.

8. Marvell, “The Evolution of AI Interconnects.” Describes the architecture in which the optical engine moves closer to the XPU from NPO to CPO.

9. NVIDIA, “NVIDIA AI Ecosystem Expands as Marvell Joins Forces Through NVLink Fusion,” Mar. 31, 2026. Confirms Marvell’s NVLink Fusion compatible scale-up networking, the silicon photonics collaboration and NVIDIA’s $2B investment.

10. Marvell / SK hynix, “Accelerating AI Infrastructure with Marvell Structera A and SK hynix CXL Memory,” Aug. 5, 2026. Confirms CMM-Ax co-development and the CXL-PNM structure.

11. Marvell, “Marvell Advances AI Memory Infrastructure Portfolio to Accelerate Agentic AI Inference,” Aug. 4, 2026. Confirms the 32TB, 50m and 2 to 3x token throughput vendor targets for the Photonic Fabric shared-memory tier.

12. Marvell, “Marvell Completes Acquisition of Celestial AI,” Feb. 2, 2026. Confirms completion of the Photonic Fabric acquisition.

HBF's name leads many investors into technical misunderstanding. The technology itself also remains contested, on access latency, endurance under repeated writes, heat and packaging yield.

Earlier this year I reviewed the H3 paper published by SK hynix on what HBF means technically and where its limits are.

This article analyzes the HBF that Sandisk presented at its Investor Day and in related public materials, and estimates bottom up how much the technology could move Sandisk’s share price if it reaches commercial deployment.

The report judges HBF’s commercialization prospects in four steps.

Section 1 compares the four architectures Sandisk presented in terms of data placement, write load and latency, and identifies which configuration is technically most realistic. It focuses on what the presentation did not sufficiently explain, namely KV cache write burden, prefetch, heat and endurance, and examines the conditions under which each architecture holds.

Section 2 examines HBF’s competing technologies, asks why HBF is needed relative to them, and tests whether it can lower system TCO despite its complex packaging cost.

Section 3 compares the platform structures of Google, Meta, Tenstorrent and NVIDIA, and analyzes technically why Google and Meta have a relatively strong incentive to evaluate HBF while NVIDIA is judged unlikely to.

Finally, Section 4 uses accelerator unit pools, HBF attach rates and stacks per accelerator at Google and Meta to derive Sandisk’s direct HBF revenue bottom up, and separates the second-order effects on product mix, NAND wafer absorption and revenue structure that could matter more to the share price.

For investors who have wondered how HBF actually works and what it could mean for Sandisk’s revenue and share price, this article is written for you.

Subscribe now

Disclaimer

This document is prepared for information purposes using public information and the author’s own analysis, and does not constitute a recommendation to buy or sell any security. Product specifications, participating companies and public roadmaps rely on disclosures from the companies and standards bodies involved. Items that have not been disclosed, including HBF and HBM data placement, cost structure and customer-level adoption likelihood, are identified as analytical assumptions or inference. Specifications, commercialization timing and customer adoption may change, and responsibility for investment decisions and their outcomes rests with the investor.

CONTENTS

1. Technical feasibility: comparing the four configurations

2. Economics: alternatives and capacity per package

3. Customer adoption

4. HBF’s economic impact: a Google and Meta bottom-up model

1. Technical feasibility: comparing the four configurations

The four configurations differ in what data HBF handles and whether HBM remains. HBF Only fills every stack site with HBF and stores both weights and KV cache there. HBF (+HBM) replaces only part of the stacks, splitting the roles with HBM.

HBM/HBF Cached places HBM as a front-end cache and stores weights and KV cache in HBF. Disaggregated separates prefill from decode and puts HBF on the decode accelerator to hold decode weights and KV cache.

The technical assessment that follows uses these data placement differences as its basis.

The first question is what role HBF can play in real inference workloads.

The four configurations differ in the data HBF handles and whether HBM remains. The comparison therefore uses write load, access pattern and latency sensitivity on a common basis rather than capacity alone.

1.1 HBF Only

HBF Only fills every stack site with HBF and uses no HBM. Because both model weights and KV cache reside in HBF, the configuration gains capacity but must also absorb the continuous write load of the KV cache directly.

In internal testing presented at Investor Day, four HBF-only accelerators matched the token-per-second throughput of eight HBM-only accelerators.

By the company’s explanation, the performance gap arises when HBM runs short of KV cache capacity, pushing some data into system memory and lowering GPU utilization. HBF relieves that capacity constraint by providing larger on-package capacity.

The company cited this as 8x capital efficiency and 2x GPU efficiency. The comparison is internal, however, and key premises such as GPU capacity and numeric precision may differ from real deployments.

The case for HBF Only therefore has to be verified in two steps.

First, whether the same on-package capacity shortfall still occurs on the latest accelerators and at lower inference precision.

Second, whether HBF can absorb the KV cache write load along with its thermal and endurance requirements.

The first issue is the comparison conditions.

The presentation used 192GB of HBM per GPU and BF16 to derive 960GB of weights for a 480B model, and concluded that eight accelerators were required once KV cache was included. Applying FP8 reduces the same weights to 480GB, and in the 288GB-per-GPU environment this report uses as its basis, the weights themselves fit within the combined HBM capacity of two accelerators.

The HBM left after loading weights can serve KV cache and runtime, though the actual headroom depends on KV cache, activation and runtime configuration. The capacity shortfall the presentation assumes may therefore be weaker than stated, but that does not mean two accelerators are sufficient for the entire serving footprint.

The second issue is the write behavior of the KV cache.

In HBF Only, KV cache is written directly to HBF. New data is written on every token and discarded when the session ends, so unlike read-dominant model weights it produces a continuous write load. NAND flash carries cell endurance limits under repeated writes. Any structure that writes KV cache directly to HBF must therefore treat write lifetime and thermal management as design variables.

Published on August 12, 2026, Li et al. (2026, arXiv:2608.11668) evaluated HBF-based KV serving using four production traces and five dense and MoE models. Write traffic exceeded read traffic across every trace, thermal limits were reached before peak bandwidth, and TLC configurations wore out faster than the SSD pool being replaced. The conclusion is that without SSD-class write management, storing transient KV cache directly in HBF is not sustainable. HBF Only has no HBM cache or write buffer, so it is the most exposed to that constraint.

Follow-up studies released on August 14 assess HBF in more conditional terms.

Son et al. (2026, arXiv:2608.13868) finds that HBF can raise batch size and throughput and reduce GPU count, but only on the premise of read bandwidth close to HBM and substantial endurance improvement.

Kim et al. (2026, arXiv:2608.14333) reports 1.94x throughput from a structure that separates immutable weights from mutable KV cache and supplies them over two parallel paths. Both are simulations, and both point to data type and movement path separation mattering more than capacity.

Taken together, I judge HBF Only to have low applicability to general conversational inference.

The more plausible fit is batch-style workloads that generate limited KV cache while repeatedly reading large weights or reference data. Where session counts and context lengths keep growing, KV write load rises and the applicable range narrows without additional write-management technology.

1.2 HBF (+HBM)

HBF (+HBM) configures k of N stack sites as HBF and keeps the rest as HBM. The essential feature is that data placement between HBF and HBM can be split according to workload characteristics.

This article takes as its base case a configuration that places model weights in HBF and KV cache in HBM. That separates the continuous KV cache write load, the largest constraint in HBF Only, from HBF.

The company has not disclosed the value of k, the data placement criteria between HBF and HBM, or the software policy. The assessment below therefore rests on the analytical assumption that weights sit in HBF and KV cache remains in HBM.

That assumption aligns reasonably well with HBF’s physical characteristics and the access profile of model weights.

First, model weights are written at load time and thereafter mostly read during inference, so the endurance burden from repeated writes is limited.

Second, weight access is dominated by large sequential reads at layer granularity, which matches NAND’s page-level read behavior comparatively well. It wastes less of each page read than small random access would.

Third, where the next required weight location is predictable, prefetch can overlap flash’s microsecond-class read latency with computation. Actual performance depends on prefetch accuracy and software scheduling.

The study cited above likewise assessed read-dominant model weights as a comparatively good fit for HBF, provided that placement is selective and that write budgeting and thermal control are in place.

Under this report’s assumptions, at k=4 node memory capacity rises from 288GB to 2.2TB while total bandwidth falls from 22TB/s to 17.4TB/s. As k increases, HBF capacity expands but HBM capacity and bandwidth decline.

The optimal k therefore depends jointly on weight size, KV cache demand and the bandwidth loss that can be tolerated.

HBF (+HBM) applies best under two conditions.

First, model weights must be large enough, in the hundreds of GB or more, that HBM alone cannot hold them. Second, data placement and prefetch must be optimizable alongside the accelerator software. The remaining item to verify is whether throughput and latency hold at target levels once prefetch is applied.

1.3 HBM/HBF Cached

HBM/HBF Cached uses HBM as a low-latency cache tier and stores both model weights and KV cache in HBF. It reduces the frequency of HBF access relative to HBF Only, but the write constraint persists as long as KV data lands in HBF.

This configuration favors services where identical or similar contexts are referenced repeatedly and a high cache hit rate can be sustained. Document-grounded question answering and workloads that repeatedly reference the same codebase are plausible candidates. In general conversational services where context differs sharply by session, HBF access and write load rise and applicability falls.

1.4 Disaggregated

The Disaggregated configuration separates prefill and decode accelerators. In the published diagram, GDDR is used on the prefill side while decode-side HBF stores decode weights and KV cache.

The assessment differs by data type. Decode weights are read-dominant after the initial load and fit HBF well. KV cache still requires write management, since new K and V are written on every token even during decode.

Separate from technical feasibility, the economics against competing architectures also need checking. The comparison cases in this report already include disaggregated designs that use no HBF, such as a GDDR7-based prefill accelerator and an SRAM rack for decode. For an HBF-based disaggregated design to be adopted, it must demonstrate an advantage in cost per token or system efficiency over those designs.

1.5 Assessment across the four configurations

The relative appeal of the four configurations comes down to what data sits in HBF and how much writing that data generates. Placing read-dominant model weights in HBF carries a light endurance burden, while writing KV cache directly to HBF adds write management and thermal control on top.

The following sections provide an in depth analysis of the HBF architecture.

The analysis examines alternative technologies that could compete with HBF, the technical factors that could nevertheless support HBF adoption, the types of customers most likely to deploy it, and whether NVIDIA could become one of them.

Finally, a proprietary analytical model is used to assess how HBF commercialization could affect Sandisk’s share price.

Read more

Earlier this year, the theme that led the market’s big rally was without question the optical sector. Almost every optical stock set new all time highs, from at least 100% to several hundred percent, and every investor was looking at optics.

But was it that they had run up too far?

From June on, the entire market including semiconductors and memory began to correct, AI skepticism started to surface, and the optical sector too had to watch its share prices fall by more than half from the highs.

Some investors caught in the drawdown left the game altogether, saying the optical sector was finished.

And now in August, share prices look to be riding an upward trend again on the back of earnings from several optical companies.

Is this really the time to come back to optical investing?

Or is it only a temporary bounce, with a sharp rise like the one earlier this year unlikely for a while?

In fact, there was a slight correction after AEHR’s earnings release, but up to now optical stocks have been in an uptrend.

In this article, I want to lay out, in detail, the round two optical investing strategy that follows my earlier optical investment map.

Why this correction happened, where optics stands now and what points to look at closely, and if you were to start investing in the optical sector fresh at this point, which layer and which names in this complex optical ecosystem you should look at first.

If you are an investor who is going to take another serious look at the optical sector, I hope you read this article through.

Disclaimer

This article is personal analysis and is not investment advice. All investing should be done based on your own research. The author may hold positions in some of the names discussed and may change those positions without prior notice.

Why the June and July Correction Happened

The starting point of the optical correction was SemiAnalysis’s CPO analysis. Along with the provocative phrase Massive Delay, it argued on the basis of low yields and cost burden that CPO mass production in the scale up segment would be pushed out from late 2028 to beyond 2029, and that until then copper and pluggable modules would keep the upper hand. After that publication, valuation adjustment appeared across the optical communications value chain.

On top of this, as AEC (Active Electrical Cable, a copper cable with a signal restoration chip built in) shipments grew quickly, a market consensus formed that in rack connectivity would be handled by copper for the time being, and the Optical I/O argument, that the transceiver module market shrinks once the electrical to optical conversion moves inside the GPU package, came back into focus as well.

Beyond that, optical stocks were already at three digit price to earnings ratios, so profit taking and index rebalancing compounded and the drawdown widened.

Subscribe now

August, What Changed

On August 11, Lumentum said demand for ultra high power CPO lasers is increasing, that it had received an initial order for ELS modules, and that several NPO engagements are underway.

For its lead CPO customer, it said high volume shipments for scale up begin in the second half of 2027 with customer deployment in 2028.

The next day Coherent said there had been no case at all of CPO demand being pushed out and that customer interest had in fact increased.

It added that scale out CPO revenue, connecting rack to rack, begins in the second half of 2026, that scale up, which ties multiple compute racks together within a cluster, follows in 2027, and that it is engaged with multiple customers on both CPO and NPO.

SemiAnalysis’s claim that NVIDIA’s scale up CPO mass production could be delayed beyond 2029 was confirmed to be untrue through the earnings calls of these two companies.

On top of this came OCS (Optical Circuit Switch) results.

OCS is equipment that switches the optical path directly instead of using an electrical switch, reducing the number of electrical switching hops. As Google shifted from what it had developed in house to external sourcing, Lumentum announced it had secured a backlog of more than $400 million from three new customers and will ship roughly $400 million in the second half of 2026. Coherent has already announced initial revenue, and on August 12 raised the size of this market to more than $4 billion.

Another of the most important items was Aehr’s earnings.

Test equipment orders are normally a leading indicator that runs several quarters to more than a year ahead of a customer’s production ramp, and Aehr guided FY2027 revenue to $130 million to $150 million, 2.6x to 3x the prior year, of which $100.6 million was announced as effective backlog already received. Its lead silicon photonics customer is already running an unmanned automated line, a follow on production order came in early August, and a new customer has already placed an order for four systems within a few months of first contact and taken delivery, so the order data showed that optical production ahead is going to grow much larger.

Subscribe now

Why Optics, Why Now, Again

As we saw above, the fundamentals of optics have not changed. If anything, guidance has moved higher through this August earnings round. Coherent’s FY2027 first quarter guidance has a low end above the quarter just reported and gross margin is heading up as well. What fell in June and July was not earnings but the valuation multiple.

That optical connectivity grows as AI data centers get bigger is something no investor with conviction in this sector disagrees with. The problem was the market’s misunderstanding that this optical demand was tied to a single CPO timeline. Exactly as I emphasized in my earlier scale up CPO article, Coherent in this earnings round also reconfirmed that the content opportunity it captures is similar whether things go the CPO way or the NPO way.

Supply is still a bottleneck as well.

InP (Indium Phosphide, the compound substrate that laser chips are made on) wafer capacity is expected to grow by 20 to 30 times through 2029, and until then it looks unlikely to keep up with demand. Lumentum’s revenue and operating margin rising together in the quarter just reported is read as a positive signal that this supply bottleneck is currently converting into profitability.

More important is that optical investors’ sentiment is coming back. On August 4 there was a report that the FCC is preparing a draft rule restricting imports of new Chinese optical transceivers, and after that report the share prices of several optical stocks rose sharply.

Whether this regulation actually gets finalized remains to be seen, but it was good signal news in the sense that investors were reminded once again of how geopolitically important the optical theme is in its connection to AI. In addition, LYTE, the ETF that holds only the optical value chain, traded $72 million on its first day of listing on August 6. That is a bullish factor for optical investing and at the same time a crowding signal.

Subscribe now

Still, what makes optical investing hard is that the optical ecosystem itself is so complex, and that each of these factors does not apply to every layer at the same time. To resolve this complexity, back in April I wrote an optical investment map. In this article I intend to use the layer map from that map again for analysis.

Unlike last time’s broad overview, in this article I plan to approach it from a perspective that is more useful to investors. In the end what investors are most curious about is, if I had to start optical investing now, again, which layer and which names among this many layers and names I should look at first, and how to think about investment strategy.

In the rest of this article I want to analyze this investment perspective in detail, together with the current state of optical technology development analyzed above.

Subscribe now

If You Were Starting Optical Investing Now

A note on reading the table

The names highlighted in yellow are ones the author has classified as priority watch candidates according to his own criteria, based on results disclosed to date and the outlook ahead. Whether a name is highlighted is not a recommendation to buy or sell, and a name that is not highlighted does not mean it is unsuitable for investment. The selection criteria and judgments are entirely the author’s own, and investment decisions should be made through your own research.

Read more

Since SemiAnalysis published its institutional report on the HBM4 8-Hi despec, a wide range of interpretations and speculation surrounding it has spread quickly through the market.

As someone who covered the HBM4 8-Hi story in an article about a week ahead of SemiAnalysis, I will also admit that every time I saw misunderstandings and conjecture get amplified and reproduced from fragmentary numbers and information, without a sufficient understanding of the technical context of the matter, it genuinely saddened me.

This article was written conclusion first, and that part will be open to both free and paid subscribers. There is a lot of market misunderstanding around this HBM4 8-Hi move, and I judged that making my analytical view on it public would be meaningful in its own right.

In the paid section that follows, I will walk through, in order, why HBM4 8-Hi emerged, how it became possible, whether memory makers’ revenue will actually head lower, and what to watch going forward for each memory maker. I am confident that this article will be the deepest analysis yet of the technical background and meaning of HBM4 8-Hi, which the market has not properly covered so far.

I hope this piece clears up at least some of the market’s unnecessary misunderstandings around HBM4 8-Hi, and helps investors view this change on a more accurate technical footing.

Disclaimer

This article is written for informational purposes and does not recommend buying or selling any particular stock. Responsibility for investment decisions and their outcomes rests with the investor. The Rubin Ultra specifications in this article are reported configurations under review, not confirmed facts, and the final specifications may change. The figures are based on public reporting, standard specifications, and my own calculations and illustrative assumptions. Anything attributed to industry chatter is hearsay that has not been publicly confirmed.

Subscribe now

Conclusion

To sum up this article, I do not see this 8-Hi transition as a bad signal for memory. This is a market where HBM cannot be sold for lack of it, and I see the HBM4 8-Hi despec as a special measure by NVIDIA to build more GPUs out of that scarce HBM. As discussed in the body, this despec is a move that would be hard for anyone but NVIDIA, which has built technical vertical integration spanning chip design, optics, and software.

Also, despite this despec, my calculations suggest the ratio of output gains to HBM price decline can come out higher, so total HBM revenue holds at 81.5% to 98% of planned 4E revenue. This means that if 12-Hi HBM4E could not actually meet more than 81.5% of the required volume, this despec can actually result in higher revenue.

Of course, what this NVIDIA request means for the memory makers is also significant. As I said in a previous article, NVIDIA has shown through this despec card that it has the influence to actually steer memory price negotiations.

Conversely, I think this may be a case that illustrates the view of SK hynix Chairman Choi Tae-won, that current memory prices are too high and that suppliers should move forward together with customers by coordinating them well.

In the end, I think it would be hasty to read this despec as a signal that the memory cycle has broken, and the right way to read it is as a signal that suppliers and customers have entered a phase of moving together, coordinating price, volume, and even design approaches.

Table of Contents

1. Why 8-Hi HBM4 Instead of 12-Hi HBM4E

2. Is the Performance Possible with 8-Hi 192GB

3. Does Memory Supplier Revenue Shrink

4. What It Means for Each Memory Supplier and the Optical Names

Further reading

Read more

The biggest confusion in discussions of CPO is that scale-out and scale-up are used without distinction. With NPO added to the picture, the market’s misreadings have grown further.

I have written several articles on CPO so far. In this one I want to start by resolving this structural confusion. From there I will use my own model to examine how much room the scale-up CPO market has to be repriced, and analyze the laser names that would be repriced along with it.

Let us begin.

Contents

1. Pluggable, CPO, and NPO

2. How Long Will NPO Last

3. Three Misconceptions About CPO

4. The Real Variable Is Scale-Up

5. Which Laser Companies Get Repriced

Disclaimer

This material is written for informational purposes and does not recommend the purchase or sale of any security. The author may hold positions in the names mentioned, and scenario results depend on assumptions and may differ from actual outcomes.

Responsibility for investment decisions and their outcomes rests with the investor. Conduct your own research and invest on that basis.

1. Pluggable, CPO, and NPO

The method mainly used in optical communications today is pluggable.

Because the structure plugs an optical module into a cage on the front of the switch chassis, a failed module can be pulled and replaced in place, and the same form factor can be purchased from multiple suppliers.

The problem with pluggable, as shown in the figure, is the distance from the switch chip to the faceplate. Electrical signals become harder to maintain in waveform as they travel further, and this can lead to signal loss.

So at this stage a DSP (digital signal processor) that restores the signal is essential. Of the 15.5 watts a typical 800G module uses, 6 to 7 watts are consumed by this DSP chip, and adding the share used by the transmit circuitry on the switch chip side to drive the long board section brings the power on a single port to around 30 watts.

What is being developed to solve this power consumption is CPO (co-packaged optics),

which puts the optical components in the same package as the switch chip.

As the electrical path shortens to microns, the DSP can be removed. Power per port falls more than 70 percent versus pluggable, from 30 watts to 9 watts, and emptying the front cages also removes the limit on how many ports a single chassis can hold.

However, the disadvantages of CPO are also clear.

Because the optical engine must be attached in the same package as the switch chip, manufacturing difficulty rises sharply, and the more tightly the optical engine is integrated into the package, the larger the replacement unit becomes on failure.

To reduce this manufacturing, yield, and service burden of CPO,

the compromise that took shape alongside CPO is NPO (Near-packaged optics).

NPO places the packaged switch ASIC and the optical engine close together on a common high-performance substrate, with the optical engine configured as a separate part or a socketed module.

This allows the ASIC and the optical engine to be tested separately and then joined, and depending on the implementation, rework or replacement at the optical engine level is also possible.

The electrical path is longer than CPO but far shorter than conventional front-panel pluggable, so a significant portion of the power benefit can be secured.

Subscribe now

2. How Long Will NPO Last

From the explanation so far, NPO looks technically better than CPO. It secures a significant portion of the power saving versus pluggable while allowing the switch chip and optical engine to be tested separately and then joined, and depending on the design, replacement at the optical engine level is also possible. It is easy to keep away from the heat source, and once a standard is established, optical engines can be chosen from multiple suppliers. So why not have every company go with NPO instead of CPO?

The obstacle lay in the fact that the socket itself is a part that had to be newly created. A high-density connector of the NPO type that sends 200G on a single lane while holding loss to a minimum did not exist before, and putting optical engines from different companies into the same socket requires agreeing on a specification first. For this, a number of suppliers launched the Open CPX MSA in 2026 to develop a common specification. Participating companies include Ciena, Coherent, Marvell, Molex, Samtec, and TeraHop.

The larger constraint is the speed the socket can withstand. When the rate carried on a single lane doubles from 200G to 400G, more signal is stripped over the same trace, and the amount reflected back at surfaces where parts meet also increases. It means a point arrives where inserting a single socket is enough for the signal to fail. The OIF (Optical Internetworking Forum) and the Open CPX MSA have this on their roadmap.

If it is solved, NPO carries into the next generation.

If not, CPO without the socket has to be used.

To summarize, once manufacturing difficulty, reliability, and serviceability are solved, performance itself favors CPO.

But in the current generation the gap NPO concedes is small, so a trade holds in which it is exchanged for assembly yield, supply chain flexibility, and serviceability. In systems where power and bandwidth density must be pushed to the limit, CPO leads; in systems where part separation, test yield, supply chain, and serviceability matter more, NPO is retained.

NPO is commonly thought of as an interim step before CPO.

But given that the two methods have different optimal points and that this optimum is divided by the character of the system, I view it as closer to a parallel structure than an interim step. For reference, the industry movement I hear from optical specialists in the field points in the same direction. The model calculated later is also set on this view, with both methods in use through 2030.

Subscribe now

Read more

Optical stocks surged across the board in today’s premarket session following reports that the United States is preparing restrictions on imports of Chinese optical transceivers used in data centers.

The companies most directly exposed to the news are $COHR, $LITE, and $AAOI, which compete with Chinese suppliers in optical transceivers and critical optical components.

$FN could benefit from growing demand for optical manufacturing capacity outside China, while $MRVL supplies the optical DSPs used inside transceivers. The potential benefits for $AEHR and $POET are more indirect and longer term, through semiconductor and photonics testing for $AEHR and optical engine platforms for $POET.

Subscribe now

What exactly was reported today?

According to Reuters, the Trump administration is drafting a plan that could block imports of newly introduced Chinese data center components into the United States.

The company with the most obvious exposure would be China’s Zhongji Innolight.

Innolight is estimated to hold roughly 27% of the global market for data center optical transceivers, with approximately 90% of its revenue coming from outside China. Reuters also noted that while $COHR and $LITE possess competitive technology, they may not currently have enough capacity to replace Chinese suppliers’ total volume in a short period.

Subscribe now

Why an immediate, comprehensive supply-chain replacement would be difficult

Nvidia provides a useful example.

Innolight is widely understood to be a major supplier to global AI infrastructure customers, including Nvidia and Google.

Optical transceivers are not completely interchangeable commodity components that can simply be removed and replaced with another product offering the same transmission speed. They require customer qualification, reliability testing, and compatibility validation with switches, DSPs, and broader network architectures.

Abruptly removing an already qualified supplier could delay the construction of AI clusters.

The US government is unlikely to want restrictions on Chinese supply chains to unintentionally slow data center expansion by American hyperscalers.

Given Innolight’s substantial market share, immediately banning existing products could create shortages and push transceiver prices higher because $COHR, $LITE, and other non-Chinese suppliers may not have enough capacity to replace the displaced volume immediately.

Reuters also noted that such restrictions could increase costs for US cloud providers such as AWS.

For that reason, I believe the most realistic scenario would be to allow products that have already been qualified and deployed to remain in use for a transition period, while restricting newly qualified products and future generations.

For example, instead of immediately removing every existing 800G transceiver, regulators could focus on new 800G revisions, 1.6T products, and later generations, encouraging customers to prioritize non-Chinese suppliers in new designs.

This is my interpretation based on the reported proposal, not an announced policy.

However, given the scale of the existing supply chain and the time required for customer qualification, a gradual transition beginning with new designs appears much more realistic than an immediate comprehensive ban.

Subscribe now

The market was waiting for a redistribution of demand, not another demand forecast

I have been waiting for phase two of the optical investment cycle since $AEHR reported earnings.

Interestingly, the actual trigger was not a new optical technology announcement or a major customer order. It was a geopolitical development.

Perhaps that is not surprising.

Most market participants already understand that optical connectivity demand will increase as AI clusters become larger. The transition from 800G to 1.6T, followed by even higher transmission speeds, is not a new story.

The argument that optics, lasers, and related components could become bottlenecks in AI infrastructure is also already well understood.

The market may not have needed another explanation of why optical demand would grow.

What it needed was an event that could change which suppliers capture that demand.

This news does not create new end demand for optical transceivers by itself. Instead, it raises the possibility that some of the future volume previously expected to go to Chinese suppliers could shift toward $COHR, $LITE, $AAOI, and other suppliers operating outside China.

The market reacted more strongly to the potential redistribution of volume than to another forecast of overall demand growth.

Subscribe now

Earnings must now validate today’s rally

Today’s premarket move was triggered by a policy headline.

For the rally to continue, however, upcoming earnings reports must confirm that the underlying optical market remains strong.

The first company to watch will be $AAOI, which is scheduled to report earnings on August 6.

It will not be enough for management to simply say that AI demand remains strong. Investors should look for evidence that expanded 800G and 1.6T capacity is translating into actual shipments.

It will also be important to understand how much of the company’s new capacity is being allocated to internally produced lasers and optical components versus complete transceiver modules.

Another key question is whether customers are purchasing components from $AAOI, finished transceivers, or both.

$LITE is expected to report on August 11.

For $LITE, investors should focus on whether supply remains tight for data center lasers and optical components, how quickly expanded capacity is improving lead times, and whether strong 800G and 1.6T demand is producing higher margins as well as higher revenue.

Even if transceiver assembly shifts away from Chinese suppliers, the structure of the underlying laser and optical component supply chain may not change in exactly the same way.

The company assembling the finished transceiver could change while some of the critical component suppliers remain the same. Therefore, the investment case for $LITE must ultimately be supported by higher content and stronger positioning in next-generation optical sources, not merely by a shift in final module assembly.

$COHR is expected to report on August 12.

Because $COHR is one of the companies most directly connected to the potential replacement of Chinese suppliers, investors should pay close attention to how much available capacity it has to absorb incremental orders.

Important areas include production capacity for 800G and 1.6T modules, indium phosphide lasers, customer qualification progress, backlog, and margins.

The market treated $COHR today as a potential substitute for Chinese suppliers. Its earnings report must demonstrate that the company has both the scale and execution capability to play that role.

$FN is expected to report on August 17.

For $FN, the key question will be whether customers are increasingly shifting optical module manufacturing outside China, particularly toward its production platform in Thailand.

Investors should monitor utilization rates, capacity additions, and any signs that customers are accelerating geographic diversification of their manufacturing supply chains.

If the proposed policy eventually translates into actual changes in procurement, non-Chinese manufacturing platforms capable of producing optical modules at scale should become increasingly valuable.

$MRVL is expected to report on August 27.

For $MRVL, investors should focus on shipments of 800G and 1.6T optical DSPs, as well as new design wins.

However, a shift in module production from Chinese suppliers to American or other non-Chinese suppliers does not necessarily increase the total number of transceivers required.

Investors therefore need to distinguish between a simple change in module suppliers and a more meaningful increase in $MRVL’s market share, product content, and exposure to non-Chinese customers.

Power comes up in every conversation about AI infrastructure bottlenecks.

The problem is that it usually takes some engineering background to follow. I’ve written about power investing before, and the main thing I took away was that readers wanted the technical side explained much more simply.

So this piece does that.

It walks through why the power delivery bottleneck exists, in the plainest terms I can manage, and then lays out how to think about the four companies closest to it: MPWR, Vicor, Murata and Samsung Electro-Mechanics. If you’ve been investing here without a clear picture of the mechanics, this should help.

Note: There’s a whole other part of the delivery bottleneck around 800VDC distribution, where 800 volts of direct current runs all the way to the rack. That’s big enough to need its own article, so I’ve left it out. I’ll cover it next time.

Disclaimer

This article is for information only. It does not recommend buying or selling any security. Investment decisions and their outcomes are the reader’s own responsibility. Figures cited are based on public sources and may have changed since publication.

Subscribe now

Power at Three Scales, and Where It Gets Stuck

Power is measured in watts, written W. A kilowatt is a thousand watts, a megawatt is a thousand kilowatts, and a gigawatt is a thousand megawatts. For scale, the microwave in your kitchen pulls about a kilowatt.

AI infrastructure power is easiest to picture at three levels. A GPU accelerator, the part doing the computing, sits at roughly a kilowatt. A rack, the cabinet servers get stacked into, runs in the hundreds of kilowatts today and is heading toward a megawatt. A campus is measured in gigawatts. That’s what people mean when they say hyperscalers are building gigawatt data centers. A gigawatt is roughly what a million American homes use, and several AI campuses are due to hit that scale in 2026.

And the buildout keeps accelerating. When OpenAI announced Stargate, it committed to ten gigawatts of AI infrastructure by 2029. By April 2026 the company said it had already passed that, having added more than three gigawatts in the previous ninety days alone. Meta has said it will build tens of gigawatts this decade. AI data centers are about to consume a staggering amount of electricity.

So why has power become the next bottleneck? Two very different problems get bundled under that one word.

The first is getting the electricity in the first place.

Moving power from a plant to a data center means new transmission lines and new substations, and in the US the wait just to get an interconnection approved can run for years. That’s a grid and permitting problem, and it belongs to utilities and power equipment makers.

The second is getting that electricity to the chip without losing it on the way.

A GPU can be as fast as you like on paper, but it won’t hit those numbers if the power can’t reach it cleanly. That second problem, the power delivery bottleneck inside the data center, is what this article is about.

Subscribe now

Two Problems in Delivering Power

So what does delivering power well actually mean?

The easiest way to picture electricity, as you probably learned in school, is water moving through a pipe.

(If that’s gone fuzzy, no problem, here it is again)

Voltage is pressure.

Current is how much water is flowing.

Power is the two multiplied together, P (power) = V (voltage) × I (current).

Which means that for a fixed amount of power, higher pressure lets you move less water, and lower pressure forces you to move more.

Think about a water pipe.

To do the same job, high pressure gets by with a narrow pipe. Low pressure means pushing far more water through, and the pipe has to get much fatter to handle it.

Here’s the problem with a chip: it needs a lot of power but runs at almost no voltage.

A modern data center GPU pulls more than a thousand watts, while the voltage it actually accepts is under one volt. Pushing all that power through at rock bottom pressure sends the current into the thousands of amps. A typical home, for comparison, is served by around 200.

So what goes wrong once the current gets that high?

Back to the water. As it travels down a pipe it rubs against the walls, and in electricity we call that friction resistance. Pushing power through resistance costs you some of it, and the loss scales with the square of the current. Double the flow and the loss is four times. Triple it and it’s nine. Since the voltage can’t be raised at the chip, the only lever left is to make the path shorter and thicker.

Then there’s a second problem. So far we’ve assumed the water flows at a steady rate, but it doesn’t. Say you’re in the shower and the washing machine starts up while someone flushes the toilet. The amount of water being drawn jumps several times over in an instant. The water is coming from a main some distance away, so it can’t meet that demand right away, and your shower goes weak for a moment.

The same thing happens at the chip. When the compute load spikes, the current it needs jumps instantly, and a power source sitting some distance away can’t respond fast enough. The voltage sags briefly, and the chip throttles itself to avoid glitching.

Leave either problem unsolved and a good share of the power you brought onto the campus turns into heat before it ever reaches the chip. Buy all the high end GPUs you want; you won’t get the performance they were built for. That’s the power delivery bottleneck.

Subscribe now

Two Fixes

Read more

Many investors are worried that the sharp rally in semiconductor stocks that began last year may now be coming to an end, marking the start of a broader downturn.

Personally, I think the market's biggest concern right now is AI capex, and memory semiconductors may be one of the main reasons behind that concern.

Is this level of concern actually warranted?

Can the SK hynix results released on July 29 put it to rest?

This piece examines what the worry assumes and whether those assumptions hold.

Disclaimer

This article is for information only and is not a recommendation to buy or sell any security. Forecasts and judgments are the author’s own, and investment decisions remain the reader’s responsibility.

Subscribe now

Why It Fell This Hard

On the surface the biggest driver was news out of China. CXMT, which listed in Shanghai the previous day, jumped 471% on debut and passed Intel in market value, and a report that China had begun developing immersion DUV lithography sent ASML down 5.8%, Nvidia 5.0% and Micron 2.3% in New York overnight. On top of that, the disclosure that Nvidia had backstopped $250 billion of OpenAI data center investment revived concerns about circular financing inside the AI complex.

Analysts largely converged on the same read. The move did not make sense on its face; no genuinely new negative had emerged, and weakened sentiment plus unstable flows had amplified the decline; no single headline explained it; and the drop looked excessive relative to fundamentals.

Subscribe now

What the Market Is Worried About

So what is the worry that is actually driving this? In my view every part of it starts with memory.

Memory is a price-cycle industry, so earnings do not compound in one direction forever. Forecasts diverge on when profits at Samsung Electronics, SK hynix and Micron turn over.

Working backward from today’s price shows what the market is picturing. SK hynix peak quarterly EPS of 150,000 won annualizes to 600,000 won, and today’s close of 1,555,000 won is 2.6 times that. No company whose peak earnings hold gets a 2.6 multiple. If earnings instead fall to roughly 30% of peak, or 180,000 won, the multiple becomes 8.6 and lands in ordinary territory. Today’s price, in other words, is a selloff that assumes 70% of peak earnings disappears, and the market appears to expect that disappearance to be complete within four quarters.

For that to hold, supply has to arrive all at once in 2028 while AI demand stops at the same time. It requires three things to be true together: that 2028 oversupply is already locked in, that China's entry is structural, and that hyperscalers can no longer keep funding the buildout. Will that actually happen? From here I take each scenario in turn.

Subscribe now

Record Capex, Fewer Bits

The market’s worry is that $75 billion of combined capex at the big three lands as bits in 2028. I think that is somewhat overdone. The step that converts capex into bits does not appear to account properly for HBM.

HBM structurally lowers the number of bits that come off a given wafer. What matters is that the multiple is not fixed; it grows with each generation. Shipping 1GB of HBM3E consumed roughly the wafer area of 3GB of commodity DRAM, and 16-high HBM4 pushed that to four times.

Read more

A separate TL;DR is provided for paid subscribers.

In mid-July 2026, when CXMT confirmed its listing date, Micron fell 8% and lost its one trillion dollar market capitalization, and Samsung Electronics and SK hynix sold off alongside it. The threat of Chinese memory had landed as a psychological shock.

Then on 27 July CXMT opened at 49.50 yuan, 472% above its IPO price of 8.66. At that opening price its market capitalization of 3.31 trillion yuan made it the most valuable company listed on the A-share market, and by midday it had reached 54.65 yuan, up 531%. Samsung Electronics and SK hynix were both weak the same day, with foreign investors net sellers.

What exactly is the market worried about? In one sentence: that CXMT will break DRAM prices and shrink the profits of Samsung Electronics, SK hynix and Micron, the three incumbents. The claims that CXMT sells 30% below them and is taking their orders, that it already holds 8% of the market, that its IPO proceeds will flood the market with new supply, and that it is catching up in HBM all point at the same outcome.

This article takes those concerns one at a time and checks them against the available data, then lays out where the incumbents’ share prices go from the listing.

Disclaimer

This article is for informational purposes only and is not a recommendation to buy or sell any security. The figures and facts it contains are compiled from publicly available sources including corporate disclosures, press reports and market research, and may carry errors from those sources or from the compilation. The author may hold positions in the companies mentioned. All investment decisions and their consequences rest with the reader.

Subscribe now

CXMT prices at or above the incumbents

Start with what the 30% figure actually is.

It compares CXMT’s company-wide average selling price against the company-wide average selling price of the three incumbents. The gap is that wide because roughly two thirds of CXMT’s revenue comes from low-priced mobile memory, which pulls its company average down for reasons that have nothing to do with competitiveness.

Compare like for like and the picture changes.

The gap is 5% to 10% per the prospectus CXMT filed for its listing, and in server products the sign flips altogether.

In July 2026, a 64GB DDR5 server module built with CXMT chips sold for 18,999 yuan in Chinese online retail, while the same specification built with Samsung and SK hynix chips sold for 18,595 yuan, putting CXMT 2.2% higher.

In the contract market, CXMT was reported to have quoted above Samsung’s supply price of 1,240 dollars per module.

Subscribe now

CXMT has no room to sell cheap

Why can CXMT not push its price lower? The reason is cost.

At its current level of technology, CXMT spends more than 30% more than the incumbents to produce the same amount of memory. Its chips are about 40% larger in area, and its yield, meaning the share of manufactured chips that work correctly, falls short of the 85% to 90% the industry treats as standard.

What about its market share? The 8% figure the market cites is a revenue number: CXMT booked 7.31 billion dollars of DRAM revenue in the first quarter of 2026, or 7.7% of the world market.

But revenue is a number that grows when prices rise.

Measure how much CXMT actually makes and the picture differs. It puts 13% of the world’s DRAM wafers into production and turns them into 6% of the memory capacity. Its chips are large and its yield is low, so half of what goes in disappears on the way from wafers to capacity. Wafer starts can be bought with capital, but the capacity that comes out of those wafers cannot.

The number that startled the market is CXMT’s first quarter 2026 revenue, up 719% year over year. Measure that same quarter against the one before it, though, and the memory capacity it shipped grew only 11% while its average selling price rose 57%. What that means is that CXMT’s growth so far has been built by memory prices rather than by its own volume.

Subscribe now

The cost gap narrows but does not close

Conventional engineering wisdom says CXMT must carry a cost disadvantage because it uses DUV lithography, which prints circuit patterns with deep ultraviolet light, rather than the EUV equipment it cannot buy.

It has nonetheless kept shrinking its circuits using multi-patterning on DUV tools, printing the same layer in several passes. It brought 16nm-class DDR5 to volume production with no EUV option available to it at all.

The problem is development pace. CXMT took 18 to 24 months to move from its third process generation to its fourth, and its fifth generation has already run past 24 months.

The incumbents also turn a generation roughly every 24 months, so even assuming CXMT holds that same pace, the fair conclusion is that the three generation gap to the major memory makers stays where it is.

The gap itself has closed quickly since 2019. But CXMT’s fifth generation has slipped toward the second half of 2026 while the incumbents ramp 1c and move on to the following node and to HBM4, so the rate of closing has flattened.

Subscribe now

Moving into HBM pushes prices up, not down

CXMT allocates less than 2% of its total wafer starts to HBM, or high bandwidth memory, which stacks DRAM chips vertically and connects them with electrodes running through the silicon. Its HBM3 schedule slipped on thermal problems, and the combined yield on the eight-high product is estimated at around 25%. The incumbents build the same product at 70% to 90%.

As CXMT shifts more capacity into HBM, ordinary DRAM prices come under upward pressure rather than downward. Stacking chips means HBM consumes roughly three times the wafer area of ordinary DDR5 for the same gigabyte. If CXMT converts 20% of its DRAM lines to HBM, the ordinary DRAM capacity coming off those lines drops to one third of what it was.

The same thing has already happened on the incumbents’ side. In 2026 HBM consumes 22% of global DRAM wafer capacity while producing only 9% of the memory capacity. That gap is the underlying reason ordinary DRAM is short today.

My view is that the concern about a flood of new supply from CXMT also needs to be placed on a different clock. Turning IPO proceeds into equipment orders and equipment into production takes 18 to 24 months, so the fair conclusion is that the supply this listing funds arrives in 2028 at the earliest.

There is also a story going around that CXMT will double its DRAM production capacity by 2030. It is said to have come out of a conference call Goldman Sachs held just before the listing.

I cannot verify whether the story is true, but there is something I want to stress.

Lately I keep seeing pieces written by press outlets and institutions without the technical background to write them, vague and rounded off, circulating as though they were established fact. The problem is that these distort what is actually happening, and the distortion reaches share prices.

Even if the story is true, what doubles is the number of wafers, not the memory capacity. Look again at the earlier figures. CXMT holds 13% of wafer starts and 6% of capacity. Doubling the wafers still leaves it near 12% on a capacity basis, and over those four years DRAM demand is set to grow far faster than that. Unless the technology gap between CXMT and the major memory makers narrows dramatically, I do not think this doubling figure has much effect on the market.

It makes more sense to read this news as good news for Chinese equipment makers such as Naura and AMEC than for the memory companies.

Subscribe now

Where this goes from here

CXMT opened on 27 July at 49.50 yuan against an IPO price of 8.66, up 472%. At that opening price its market capitalization of 3.31 trillion yuan, or 489 billion dollars, made it the most valuable company on the A-share market. By midday it had reached 54.65 yuan, lifting the market capitalization to 3.7 trillion yuan, with more than 100 billion yuan traded in half a session.

Nomura initiated coverage on the day of the listing with a target price of 116 yuan. It expects CXMT to earn 5.80 yuan per share in 2028 and applies a multiple of 20 times to that, which works out to a market capitalization of 7.76 trillion yuan. Working backwards, the target assumes 2028 net profit of 388 billion yuan. That is 3.9 times what the market expects CXMT to earn this year, and more than Micron earns at the peak of a cycle.

So where does the share price sit today? At its opening price, CXMT trades at roughly 30 times this year’s expected earnings. On the same day Samsung Electronics traded at 4.32 times, SK hynix at 4.69, and Micron at 6.42.

The closest precedent in the same market for judging where CXMT goes next is likely SMIC. It opened at 95 yuan on its first day in July 2020, up 246%, and never traded above that price again. Ten weeks later it was 48% below that high, and a year later it was still 32% below it. It never broke its IPO price, though, and remained 136% above it a year on.

CXMT does have far fewer freely traded shares than SMIC did, and it lists into the particular conditions of an AI infrastructure bottleneck, so its path may differ.

My forecast is nonetheless that the first day of trading, the point where expectations run hottest, marks the high, and that the high holds for a considerable period. A large correction around the first lockup release in January 2027 is also possible. I do not expect the stock to fall below its IPO price: state capital sits behind it, it is the only pure-play DRAM stock on the A-share market, and memory remains a powerful theme.

So what happens to the three incumbents’ share prices over that stretch?

The market has already priced much of its concern about slowing earnings growth into memory shares, and for the decline to keep going you need either memory earnings to turn down or AI demand to reverse outright. Neither price nor volume shows any sign of coming down, and that is how the industry reads it today.

Let me look at the three companies more closely.

SK hynix has its earnings concentrated in HBM. CXMT cannot enter that segment, so of the three SK hynix is the most insulated from it. Against that, those earnings rest on the capital spending plans of the handful of customers buying AI accelerators. What moves SK hynix shares is that, not CXMT.

Samsung Electronics sells the largest share of its output to Chinese smartphone and PC manufacturers. It is hit first and hardest if Chinese procurement directives widen to memory, and what CXMT can push outside China is commodity mobile and PC DRAM rather than server product, so Samsung meets that price competition first as well. Against that, its forward multiple of 4.32 times is the lowest of the three, so much of the risk already sits in the price.

Micron has the least left to lose in China. Its mainland China revenue fell from 12% of the total to the 7% range after the 2023 purchase ban, and it exited the Chinese server memory business in October 2025. Its automotive and smartphone business in China continues, but the data center exposure is already gone. It competes with CXMT for the same customers outside China, but if the United States moves to exclude Chinese memory from data centers and government procurement, that competition never arrives in North America. Its multiple, at 6.42 times, is the highest of the three.

As I mentioned earlier, I am currently in Korea for vacation. Although this is technically a vacation, I have continued networking and meeting with professionals across the finance and semiconductor industries.

In this article, I will answer the questions you submitted previously while also sharing some of the hottest topics currently being discussed in Korea’s semiconductor industry, particularly in the memory sector.

By the end, I believe you will have a much clearer understanding of what is currently happening on the ground in the memory industry, as well as the key concerns surrounding the Korean memory business that I was not fully aware of before speaking with these experts.

I am confident that this article will give you a broader and more practical perspective on the current state of the industry.

Let’s get started.

Subscribe now

The Memory Power Struggle Hidden from Public View and the Limits of the Memory Business

The memory business has traditionally been one in which suppliers held very little power. For decades, customers largely controlled memory pricing. Whenever the balance between supply and demand broke down, memory companies were forced to endure another downturn, making memory one of the clearest examples of a cyclical business.

However, the rise of AI has shown the potential to break this structure.

As the bottleneck in large scale AI workloads shifted from compute toward memory, demand for memory surged. The memory business was no longer driven solely by ordinary imbalances between supply and demand. Supply shortages gave memory companies a stronger position, while multiyear LTA contracts improved their earnings visibility and bargaining power.

These changes were also among the main reasons behind the recent rise in memory stocks.

Another reason I have continued to maintain a bullish view on memory is that the memory business is moving away from a commodity based cyclical structure and toward the custom market. This transition has been the foundation of my bullish thesis on memory.

However, after visiting Korea and speaking with several industry experts and professionals working in the field, I realized that the situation inside the industry is more complicated. I was also reminded once again of how intense the power struggle surrounding memory has become.

Why has this power struggle become so intense?

I would like to examine that question in detail in this section.

The first thing we need to recognize is that although the memory business is moving toward custom products, it is still largely a commodity based business in which suppliers have limited power. As has always been the case, the greatest weakness of this business is that memory companies ultimately have to meet customer requirements.

One of the most surprising things I heard from multiple industry professionals I met in Korea was this.

Subscribe now

Questions Covered in the Second Half of This Article

1. How will memory evolve beyond the traditional separation between memory and compute, and what is the long term roadmap for HBM, near memory computing, PIM, and logic memory integration?

2. As HBM4 and HBM4E move toward 12 high and 16 high stacks, what are the most serious physical and manufacturing bottlenecks?

3. Is CPO already entering production for scale out networks, or does it remain in the development and sampling stage? What are the biggest practical barriers to commercialization?

4. How significantly will the capacity expansion of CXMT and YMTC affect the global DRAM and NAND supply chains?

5. What capacity expansion and bit growth should we expect from Samsung and SK hynix in 2027 and 2028?

6. Should HBM be classified as a standardized, semi custom, or fully customized product? How should future near memory and logic integrated memory architectures be classified?

7. Among HBM, DDR, NAND, and HDD, which layer is likely to benefit the most from AI scaling and changing workload patterns?

8. What hardware, software, and algorithmic innovations could reduce KV cache requirements and ease bottlenecks across different memory and storage tiers?

9. How long are the LTAs currently being discussed in the memory industry, and how large could the contribution of RPO become relative to future revenue?

10. What is the realistic commercialization timeline for SPHBM4, and could its architecture give Samsung an advantage in yield, cost, or pricing?

Read more

Subscribe now

Semiconductor investors have had a run of unsettling news.

Hyperscalers are announcing far fewer new data centers, and chip stocks that rode the AI story have been sliding. On top of that, on July 16 Moonshot released Kimi K3, a 2.8-trillion-parameter model that placed level with the top US models in blind arena (LMArena) rankings, with its weight files scheduled for release on July 27. A wave of open models promising frontier-level performance at a fraction of the price has fed a simple worry,

Has the market really moved past the semiconductor theme? This article works through the numbers in public view: how open models make money, what frontier developers’ margins actually look like, and what happens to hyperscaler profits when workloads move to cheaper models. The goal is to answer one question. Does the switch to open models make hyperscalers buy fewer chips, and where does compute and memory demand go from here?

Contents

1. How open models make money, and how fast they are spreading

2. The arithmetic of frontier margins

3. How the switch to open models changes hyperscaler revenue

4. Does the extra profit turn into compute investment?

5. Will memory investment grow?

6. Conclusion

Disclaimer

The figures cited here come from public sources: earnings releases, filings, public reports, and press coverage. All interpretation and outlook are the author’s personal analysis, and nothing here is a recommendation to buy or sell any security.

Subscribe now

1. How open models make money, and how fast they are spreading

A token is the smallest unit of text an AI model processes, and model fees are charged per token. Frontier models are the top closed models, such as OpenAI’s GPT or Anthropic’s Claude, sold only through a paid API. Open models are models whose weight files are published, so anyone can download and serve them (they are also called open-weight models).

Open-model developers earn money differently from the frontier. Because the weights are public, a company that hosts the model itself owes the developer nothing. Even when the developer sells its own paid API, competing hosts selling the same model cap what it can charge.

The economics of that hosting business are real. Inference specialists report serving 70B-class models profitably at $0.90 to $2.00 per million tokens, and Together AI said its annualized bookings passed $1.15 billion. Serving margins exist in the open camp. What is zero, on the self-hosting path, is any mandatory premium for the model itself.

On capability, Epoch AI’s measurements put the best open models about four months behind the frontier. Kimi K3, released July 16, placed above the previous frontier generation in blind arena rankings right away, and level with the newest frontier models.

Usage data shows the shift too. OpenRouter is a routing platform that lets developers call hundreds of models through one API, so where its traffic goes is a widely used proxy for model market share. On this platform, the share of calls sent to models priced under $1 per million tokens rose from 18% in January 2026 to 41% in June. The share of tokens processed by Chinese open models such as DeepSeek, Qwen, and Kimi went from under 2% in late 2024 to about 61% in May and June 2026. Counting all open-weight models, the latest weekly snapshot is about 69%.

Subscribe now

2. The arithmetic of frontier margins

While volume moves at this speed, where has the money gone? Start with frontier developers’ margins.

Public reports put Anthropic’s gross margin (revenue minus the cost of serving) at -94% in 2024, improving to the mid-60s by 2026, with a 36% operating margin before model training costs (a research estimate). OpenAI’s inference cost reportedly quadrupled in 2025 and its adjusted gross margin slipped from 40% to 33%.

Add training costs and the picture changes. Reporting based on OpenAI’s 2024 investor projections put its 2026 loss at $14 billion. Anthropic told investors it expects its first quarterly operating profit in Q2 2026, $560 million at roughly a 5% margin, while stopping short of promising a profitable full year. The accounting bases are not identical, but for scale: Anthropic’s 36% before training (a research estimate) sits near AWS’s 35–38% operating margin (37.7% in Q1 2026), while the margin including training is around 5% even in the first profitable quarter, far below the hyperscalers’ 30s.

What matters more than the margin rate is that the premium is actually being paid. As of late May 2026, Anthropic processed about 11% of tokens on OpenRouter but took about 42% of estimated model spend on the platform. Benchmark gaps have narrowed to a few points, yet a 25x price gap ($25–30 per million tokens for Opus-class versus under $1 for budget open models) keeps getting paid. The reason is that companies do not pay for benchmark points. They pay for an agent that does not fall apart in the middle of a fifty-step job, for reliability where failure is expensive, and to avoid re-validating their whole workload on a new model.

So the real risk that open models pose to frontier developers is not a falling margin rate. It is the loss of mid-tier volume. Open-model hosts now offer APIs that are drop-in compatible with OpenAI’s and Anthropic’s, so switching models can be a one-line change. Work with well-defined requirements, such as document classification, extraction, summarization, and simple code, is leaving first. What stays on the frontier is the hardest work, where the capability gap is worth money. Prices there are holding, but as the middle drains away, frontier revenue growth depends more and more on how fast that top tier expands.

Subscribe now

3. How the switch to open models changes hyperscaler revenue

Hyperscaler AI revenue comes through three accounts: renting compute to frontier developers, charging cloud customers for model usage, and selling their own products with models inside, like Microsoft Copilot. The open-model switch changes the customer account first.

Take a base case: an enterprise pays $25 for a job of one million tokens on a frontier API, and follow the money before and after the switch. Before, the payment reaches the hyperscaler indirectly. The $25 first becomes revenue for a developer such as OpenAI or Anthropic. At the reported 65% gross margin, the developer spends $8.75 of it on serving cost, that is, buying cloud compute, and keeps $16.25 to fund payroll and training. The hyperscaler books $8.75 of revenue. At a 35% infrastructure operating margin, that is $3.06 of profit and $5.69 of server operations.

For simplicity, this model treats the developer’s entire serving cost as hyperscaler compute purchases, assumes that after the switch the company deploys directly on the same hyperscaler’s cloud, and prices every tier on the same basis of one million output tokens.

When the same company moves to an open model, the pass-through disappears. The weights are free, so nothing is owed to a developer, and everything the customer pays to run the model in the cloud is infrastructure revenue for the hyperscaler. Structurally, this is better revenue: compute rental that used to ride on someone else’s model becomes a direct sale to the hyperscaler’s own customer. What comes with it is a price collapse. The same million tokens drops from $25 to under $1.

That $1 is not a discounted version of the $25 model. It is the price of a small model whose active parameters, and therefore compute per token, are a tenth to a fiftieth of the frontier’s. Open serving prices form where competing hosts add a margin over cost (hence the profitable $0.90–2.00 price points). So on both paths, the compute the hyperscaler provides equals about 65% of the revenue it books.

If the switch is to a top open model, there is little to weigh.

K3’s compute payment of $9.75 is larger than the frontier’s $8.75, so hyperscaler revenue and server operations rise the moment the switch happens, regardless of any reuse. The case where the outcome can go either way is the trade-down to a budget model, so the base case continues on the $1 path, the least favorable one for the hyperscaler.

Under the assumptions above, the size of the customer account after the switch is set by the reuse rate: how much of the saved $24 goes back into tokens. If none of it does, hyperscaler revenue is $1 and profit is $0.35, far below the $3.06 before. If 32% is reused, profit matches at $3.06. If all of it is reused, revenue is $25, profit is $8.75, and server operations are $16.25, which is 2.9 times the level before the switch.

If companies re-spend just a third of what they save, the hyperscaler earns as much as before. So where are companies actually sitting? Reports keep coming of annual AI budgets used up in a single quarter, and token usage is growing five to seven times a year. That is not re-spending savings; that is growing the budget itself. Reality sits to the right of the rightmost bar in the chart. In that zone, the $16.25 premium that used to go to the frontier developer disappears, and hyperscaler profit and server operations fill the space.

The other two accounts point the same way. In the own-product account, swapping in an open model removes the license fee from the cost line and improves margins. The developer-rental account carries a risk: if mid-tier volume loss squeezes frontier revenue, developers could slow their training compute purchases. But multi-year rental contracts with OpenAI and Anthropic sit in backlog, so this account is not one that breaks quickly.

Read more

In my late-March analysis of Aehr Test Systems (AEHR), I laid out three things the thesis still needed to prove: that bookings would convert to revenue, that gross margin would recover, and that customers in the evaluation stage would move into actual production.

The thesis at the time was that the economic value of burn-in testing is rising structurally, driven by the power density of AI processors, the optical stabilization requirements of silicon photonics, and the spread of hybrid bonding, and that AEHR sits on the receiving end of that demand with both wafer-level and package-level burn-in platforms.

The fiscal 2026 fourth quarter results released on July 14 showed progress on all three. Quarterly bookings of $60.7M, an effective backlog of $100.6M, and fiscal 2027 revenue guidance of $130M to $150M, which is 2.6x to 3x the year just ended. The stock jumped 31% in after-hours trading.

This is not simply a recap of AEHR’s earnings release. It digs into what the numbers actually mean and extends from there to the changes underway across the entire optical value chain. What does this quarter tell us, how should an optical investor frame the next 12 to 24 months, and how should conviction be allocated across the layers of the value chain?

I hope this article helps you pressure-test your own conviction in optical investing.

Table of Contents

• Checking the March Homework

• Signal 1: Where the Optical Production Cycle Stands

• Signal 2: How Rising Reliability Requirements Reshape Cost Structures and Entry Barriers

• Signal 3: The Spread Into Adjacent Markets

• Layer-by-Layer Outlook: The Next 12 to 24 Months in Optics

• Conclusion

Disclaimer

All figures cited in this article come from public sources, including the company’s earnings materials, conference call, and filings. Interpretations and forecasts are my personal analysis, and nothing here is a recommendation to buy or sell any security.

Checking the March Homework

First item, converting bookings to revenue. Fourth quarter bookings came in at $60.7M, more than five times the $11.1M a year earlier. Year-end backlog stood at $80.6M, and adding the $20M received in the weeks right after the fiscal year closed brings the effective backlog to $100.6M. With fiscal 2026 revenue at $50M, the backlog alone equals twice last year’s revenue. On that base, the company guided fiscal 2027 revenue to $130M to $150M, growth of 160% to 200%. Management said it is not capacity constrained even at the $150M level and sees room to raise guidance as additional orders firm up.

Second item, margin recovery. The question in March was whether gross margin, which had fallen into the 30s, could get back above 40%. Fourth quarter non-GAAP gross margin was 45%, up 10 points from 35% a year earlier, which the company attributed to higher revenue, better factory utilization, and favorable product mix. Full-year gross margin was still 38.5%, so one quarter does not settle the annual margin question. But with the company guiding fiscal 2027 non-GAAP pretax income to 18% to 22% of revenue, this quarter at least confirmed that gross margin can climb back into the mid-40s when volume recovers.

Third item, evaluation customers converting to production. This is where the progress was largest. A new silicon photonics customer, described as a global leader in networking, went from first contact last November to ordering two nine-wafer FOX test cells and two FOX NPs within months, and has told the company to expect additional systems this year to support next-generation hyperscale data center deployments.

On the AI processor side, a wafer-level burn-in benchmark with a top-tier semiconductor company that supplies AI accelerators, CPUs, and network processors was completed with results that exceeded the customer’s expectations. The benchmark was originally meant to qualify a next-generation device, but the customer has now asked to evaluate pulling the process into its current high-volume device and to run a second device in parallel. Meanwhile, the lead AI processor customer moved all of its production burn-in screening to wafer level and eliminated system-level screening entirely.

The caveats deserve equal clarity.

A full year of execution sits between guidance and realized revenue, and management itself acknowledged that this business will remain lumpy and cyclical. Three customers each accounted for more than 10% of fourth quarter revenue, and the hyperscale customer’s second device has already slipped once.

Even so, the fact that three items left unverified in March were largely confirmed within four months is very encouraging. And the reason this matters goes beyond the movement of one stock: AEHR’s order data serves as a progress indicator for the entire optical value chain.

Subscribe now

Signal 1: Where the Optical Production Cycle Stands

Burn-in equipment has to be purchased and installed before a customer starts volume production. Orders follow a sequence of benchmark, pilot qualification, and development agreement before production purchase orders arrive, and production begins only after the tools are on the line. Equipment bookings therefore lead customer production ramps by anywhere from a few quarters to more than a year, which means AEHR’s optical bookings offer an early read on when, and at what scale, optical device makers plan to produce.

The lead silicon photonics customer is already ramping, with follow-on orders over the past year and more placed in the current fiscal year. Its line is fully unmanned, with automated wafer handling integrated with AGVs (automated guided vehicles). This is not a couple of trial units; it is mass-production infrastructure that runs without human hands.

The new networking customer first met the company last November and, within months, ordered four systems and took delivery. New process technologies typically take years of customer persuasion. Adoption at this speed suggests wafer-level burn-in has moved past the unproven-technology stage in optics and is approaching standard-process status among lead customers. Management also noted that sales cycles are shortening across the industry.

On the numbers, fiscal 2026 revenue from optical device test and burn-in was roughly $10M, a figure that includes data center transceivers, chip-to-chip I/O, and HDD-related optical testing. Applying the company’s stated fiscal 2027 silicon photonics mix of 15% to 20% to the guidance range gives roughly $20M to $30M, a path where optical-related revenue at least doubles within a year.

800G transceivers are in their volume phase and the 1.6T transition has begun. CPO (co-packaged optics, which integrates optical components into the same package as the switch chip) and optical I/O are in hyperscaler pilots and early deployments, and the debate over commercialization timing continues. Yet test infrastructure orders are already being executed against both trends. The next volume phase of optics is moving beyond industry forecasts into concrete production equipment orders.

One point from the March analysis is worth restating. Optical burn-in performs defect screening and optical stabilization (aging) in a single step. Devices that contain lasers go through an early period where output power and wavelength drift, and burn-in walks them through that window so they ship in a stable state. In ordinary logic chips burn-in is purely a screen; in optics it is a required step of the production process itself. The lead customer’s fully automated line and the new customer’s rapid adoption are evidence that this process structure is taking hold on real production floors.

Subscribe now

Read more

The last piece followed the structural evolution from HBM to cHBM to memory-on-logic, using HBM as the reference application, and argued that the memory business is turning into a custom silicon business.

This piece takes a more general, functional angle. The idea is to classify custom memory by function, by what memory does beyond storing data. The industry uses three names for this axis, CIM, PIM, and PNM. The boundaries between these terms vary from company to company and paper to paper, and some use PIM as an umbrella for every compute-in-or-near-memory structure. To avoid confusion, this piece draws the lines by the physical location of the compute.

• CIM (Compute-In-Memory) uses the electrical behavior of the memory cells or the array itself to perform computation. It exists in both analog and digital forms, and it is the deepest that compute can go.

• PIM (Processing-In-Memory) keeps the cells as storage and places compute units inside the same DRAM die, next to the banks or in the peripheral circuitry.

• PNM (Processing-Near-Memory) moves the compute onto a separate logic die outside the memory die, connected to the memory through a very wide, very short path. It is the shallowest of the three, and Qualcomm’s recently announced HBC is exactly this structure.

My thesis has been the same since the last piece.

The memory business is turning into a custom silicon business, and commodity memory that anyone could buy is evolving into custom memory designed for specific customers. The last piece traced that transition along the position axis, where the memory sits. The three names in this piece trace the same transition along the function axis.

Memory is moving toward compute, and compute is walking into memory, two flows in opposite directions converging on the same place. With two axes, a single product can hold one coordinate on each, so a product from the last piece appearing again here does not mean its classification changed.

This piece unpacks the technology so that readers without a hardware background can follow it comfortably. It closes with my own view, worked out in numbers, on how much this transition dampens the next cycle that could follow the record capacity buildout. Anyone with money in memory should read through to the end.

Contents

1. Why Compute and Memory Live Apart

2. The Balance AI Broke

3. The Price of Custom Memory, and Why It Is Back on the Table

4. Two Answers, PIM and PNM

5. My View, How Much Custom Memory Dampens the Next Cycle

Disclaimer

The figures and facts cited in this piece are drawn from public sources. The interpretations and scenarios are entirely my own analysis. Nothing here is a recommendation to buy or sell any security, and investment decisions and their consequences rest with the investor.

1. Why Compute and Memory Live Apart

Every computing system splits into two parts, a processor that does the work and memory that holds the data. Keeping compute and storage apart and wiring a path between them is the basic skeleton of nearly every computer we use.

It is called the von Neumann architecture.

Watch a processor do a single addition. The structure of A+B=C lives on the processor, while the values A and B live in memory. The processor fetches 2 and 1 from memory, adds them, and writes the result 3 back. Compute on the processor, data in memory, signals shuttling between them without pause. That is the basic motion of computing.

On physics alone, the best structure is obvious. Put the processor and the memory as close together as possible. Shorter signal paths mean lower latency and lower energy at the same time, so integrating both on one die wins on performance and on power.

Such a memory exists. It is SRAM, the memory that sits on the processor die and answers within nanoseconds, the one we know as cache. Its problem is density. Each cell burns six transistors, so growing the capacity would let the memory eat the processor’s die. SRAM stays a small vessel for only the hottest data.

DRAM is the answer from the other side. One transistor and one capacitor per cell packs far more data into the same area, but it cannot ride on the same die as the processor.

The reason sits in the process technology. A DRAM process is tuned entirely around building capacitors densely and keeping them from leaking. Draw logic circuits on that ground and the same circuit comes out larger and slower than it would on a logic process. Building DRAM cells on a leading-edge logic process fails the other way, on cost. Ground that suits logic and ground that suits memory are different ground. Economically integrating high-density commodity DRAM with cutting-edge logic on one die has never worked.

So the industry chose separation. Processor die here, DRAM die there, wired together across the board. Out of that choice came the memory hierarchy we know, near and small SRAM, far and large DRAM, farther and larger storage. The split von Neumann drew on paper was carved into silicon as a die boundary.

The road memory technology has walked since then fits in one sentence. Get close again, somehow. Interface speeds rose generation after generation. GDDR widened the pipe beside graphics chips. HBM took the seat right next to the GPU on a shared silicon substrate, the interposer. Memory-on-logic, covered in the last piece, puts memory directly on top of the processor, the physical end of this road.

This history of closing the distance shares one trait.

What moved was always the data or the memory. Caching copies hot data next to the processor. HBM and memory-on-logic pull the memory die itself toward the processor. Moving the compute has never been the industry’s mainstream answer. Which leaves one idea on the table.

Do not move the data. Move the compute.

This is where the names CIM, PIM, and PNM were born. Carve off part of the compute logic and send it across the boundary. The moment you send it, a question follows. How far, and into which die. To see why this idea got serious now of all times, start with what AI began demanding from memory.

2. The Balance AI Broke

The shuttling between processor and memory was not a problem for a long time. Once the processor fetched data, it had plenty to do with it. As long as compute was heavy and data was light, the cost of the round trip hid behind the compute time.

AI inference flipped that ratio.

Each time an LLM produces one output token, it must read the entire set of model weights from memory once. It pulls in hundreds of gigabytes, and the work it does with them is multiplication and addition. Measured in operations per byte, the workload is punishingly light, so the processor spends much of its time waiting for data to arrive. GPU compute jumps several fold each generation while memory bandwidth does not, and the gap keeps widening.

This is why HBM commands its price, and why memory became the accepted bottleneck. No matter how fast the processor gets, a narrow data path wastes it.

The wall has an old name, the von Neumann bottleneck.

The industry’s first response was more of the same closing of distance. Closer, wider. HBM has widened its interface and stacked higher with each generation. But there is a question that comes at the problem from a different angle.

Can the amount of data sent be reduced at the source?

Return to the addition example. If the processor must compute (A+B), (a+b), and (α+β), the existing structure moves six values, A, B, a, b, α, and β, from memory to the processor. If the memory knows how to add, only three values need to cross, C, c, and γ. Not one wire was added and not one pin ran faster, yet the data crossing the boundary halved. The effect equals doubling the usable bandwidth. Fewer bytes moved also means less power spent moving them.

The example also previews a fork in the road. The idea of taking compute to the data is shared, but how far you take it splits the path in two. Bring it inside the memory die, as CIM and PIM do, and the volume of data sent shrinks at the source, but touching the inside of the memory makes this a road only memory makers can walk.

Take it only to the memory’s doorstep, as PNM does, and the raw data still crosses to the adjacent logic die, but only results travel on to the host, easing the outer path. Since nothing inside the memory changes, companies that buy memory can walk this road too. In silicon terms the fork reduces to one sentence. Put the compute inside the memory die, or attach it beside the memory as a separate die.

Compute where the data lives, and send only the results. That single line is the skeleton of CIM, PIM, and PNM alike. If the bottleneck is movement, reduce the movement. It sounds more obvious the longer you listen. Yet this obvious idea sat in papers for decades and is still climbing over the threshold into commercial products. Not because the technology fails, but because of the price tag attached to the idea.

From here, the piece walks through why this good idea went uncommercialized for decades, how it is becoming product in the field right now, and my view on how much this transition dampens the next cycle.

3. The Price of Custom Memory, and Why It Is Back on the Table

Standard memory’s strength lies in a fully standardized interface.

A DRAM die operates only on commands agreed at JEDEC, the standards body that sets the memory industry’s common specifications. Give it an address and it returns a value. Give it a value and it stores it. Whether the chip above it is an Intel CPU, an NVIDIA GPU, or some startup’s accelerator, the behavior is identical. That is what made same-spec memory from Samsung, SK hynix, or Micron interchangeable commodity. Memory prices riding the spot market, and customers holding the bargaining power, both flow from that interchangeability.

Put compute into the memory and the interchangeability breaks. Suppose the memory learns to add. It must now understand a command that does not exist in the standard, fetch the result of the addition. The processor must be designed around that command. The software must know where the boundary sits and split the code, deciding which operations go to the memory. Memory and processor become locked into each other’s designs. That is the first tax, the dependency tax.

There is a second tax, the area tax. Compute logic occupies ground where cells would have stood. Worse, a DRAM process is barren soil for logic, so the same compute circuit draws larger there than it would on a logic process and takes even more room. Recall that a memory maker’s profit formula starts with how many bits come off one wafer, and the weight of this loss becomes visible. A company whose core business is packing more bits into the same area is planting compute where bits would grow. The density loss is a cost-structure problem before it is a spec-sheet problem.

So why is it back on the table now?

Because the math of gains against taxes has changed.

On the gains side, AI inference pushed memory bandwidth to the top of the bottleneck list, making the value of reduced data movement larger than it has ever been.

On the tax side, the base die’s move to logic processes created a place for compute to land, and the dependency that remains can be passed through to price.

Memory tailored to a specific customer bills a premium and development fees rather than the commodity spot rate. A business that rode the spot market can become a business that holds contracts. That is why this story about taxes is ultimately a story about margins.

4. Two Answers, PIM and PNM

Of the three answers, the deepest, CIM, uses the electrical behavior of the cells or the array itself for computation. Research continues in both analog and digital forms, but its taxes are the heaviest and no product carries a commercial schedule, so it ranks low on this piece’s investment clock.

The two that have crossed into actual products are PIM and PNM. Last month’s piece mapping nine branches of solutions to the memory bottleneck introduced these two as one branch, and that branch already has concrete products and companies standing on it.

Subscribe now

Read more

Since starting my Substack, I get asked for stock picks all the time.

“Could you pick the three best names in optics?” “Which stock will make the most money in the next three months?”

Questions like these honestly leave me a little embarrassed.

Think about it.

If I actually knew which stock would rise the most in the next three months, what would I have done? I would have gone all in with everything I own, gotten rich, and retired already. (If only I had put it all into SanDisk last year instead of buying a house….)

So I don’t make stock recommendations on my Substack. I believe the value I can offer lies elsewhere. Then what is it that I do best? For the semiconductor companies, technologies, and industry dynamics sitting at the bottlenecks of the AI era, it comes down to three things.

1. Breaking down the technical challenges in the simplest terms possible

2. Passing along what I hear on the ground in Silicon Valley

3. Explaining how all of it connects to actual investment angles

That, I believe, is what I do best.

And what I ultimately hope for is that readers of my articles learn to analyze technology and investment value on their own, and build their own investment methodology.

Still, I am an investor too.

Not on the level of the great professional investors, but I have my own perspective and my own principles. Today I want to share them.

Let me be clear up front. This article is not a buy or sell recommendation for any specific stock, and it is not a foolproof method for making money.

It is the AI infrastructure and semiconductor investment philosophy of one Substack writer who happens to be an engineer with a passion for market analysis.

I write it hoping it helps my subscribers build their own methodologies.

Disclosure

I currently hold shares in some of the companies mentioned in this article and may buy or sell them at any time without notice. Unless otherwise stated, I have received no money or other compensation from any company mentioned in connection with this article.

This article is personal research and opinion written for a general audience. It is not investment advice and does not take into account any individual reader’s investment objectives, financial situation, or risk tolerance. All investment decisions and their consequences rest with the reader. Assessments of companies and industries are based on information available at the time of writing and may change as new information emerges.

Step 1. Conviction in a Sector

Before I look at any individual stock, I think about which sector to invest in.

The biggest theme of our time, as I see it, is AI infrastructure and semiconductors, and within that theme I break things down into the following sectors.

My goal on Substack is to raise readers’ technical understanding of each sector and help them form their own conviction.

That is why I focus my articles on the sectors where I hold the strongest conviction. I keep researching and studying, and I share the results in articles written to be as accessible as possible. My hope is that subscribers use that research as raw material for building sector conviction of their own.

Step 2. Drawing an Investment Map of Each Sector

Once the big picture of a sector is in place, the next step is to break it down. How does the sector divide into themes, and which companies belong to each one?

Take the compute sector, which is about as simple as it gets.

GPUs mean NVIDIA and AMD.

CPUs mean Intel, AMD, ARM, Qualcomm, and so on.

If you want to stretch the sector a bit further, you can extend it to the custom ASIC firms that help hyperscalers design their own chips, the startups building inference chips, and the foundries, equipment makers, and OSATs (outsourced assembly and test providers) that actually manufacture all of it.

Some sectors, on the other hand, are genuinely complicated.

Optics and power are the prime examples.

For sectors like these, it is hard to draw an investment map in one sitting without real technical understanding of the field. That is why I have written investment map articles for them. I like to think they have been useful for investors who found it hard to know where to start.

Of course, no classification is perfect. But if you keep practicing this way of organizing an industry, you eventually develop the ability to draw your own maps and define your own sector themes.

Subscribe now

Step 3. The Five Factors

Once the sector homework is done and it is time to pick stocks, I run through five factors, phrased as questions I ask myself.

How I Analyze Stocks, With Real Cases

In this section I want to share my actual thought process. Why do I build conviction in certain stocks, and how do I size up risk in others?

My calls have not always been right, but the cycle of analyzing, deciding, and revising has done a great deal to harden my conviction over time.

The analysis below reflects my own criteria, so treat it as reference material and settle on your own approach. All assessments are as of July 10, 2026.

What INTC Taught Me

I was bearish on Intel starting last year. (This was also before I had built the systematic approach to investing described here.)

Every time I talked with engineers working at Intel here in Silicon Valley, I heard discouraging things about the company, such as the recurring layoffs and the rigid culture.

On top of that, as a former foundry engineer, I knew how brutally hard it is to catch up on leading-edge process nodes. I could not picture Intel suddenly leapfrogging TSMC and Samsung Foundry on yield and winning customers. The CPU business was steadily ceding server share to AMD, and with the center of gravity in data center spending shifting to GPUs, I did not see how CPUs alone could carry a growth story.

But as everyone knows by now, INTC went from around $20 to briefly above $140, and even after the recent pullback it is still up roughly five times.

Watching that unfold taught me the following lessons. Those lessons went a long way toward shaping the five factors above, and they continue to guide how I set my investment criteria.

Let me share them.

1. When the entire pie is growing, the number two or number three player does not have to become number one. The portion flowing to second and third place grows on its own. This is why Factor 1 asks about the sector before it asks about the company.

2. Stocks directly tied to government and policy attract flows that have nothing to do with fundamentals. The US government’s equity stake and strategic investments from SoftBank and NVIDIA were announced in quick succession, and a purely technical lens misses that force entirely. This is why Factor 5 asks separately about catalysts outside of earnings.

3. Design talent and technology accumulated over decades become an investment asset in their own right when demand takes off. (Factor 2)

4. The same was true of packaging. Even while Intel was written off on leading-edge nodes, packaging assets like EMIB and Foveros kept compounding. As AI chip competition expanded into a system-level packaging race that binds HBM and chiplets together, those assets emerged as a meaningful option in the re-rating of Intel’s foundry business. (Factor 4)

5. Never underestimate a CEO like Lip-Bu Tan. Layoffs are painful for employees, but investors often read them as a sign the cost structure is getting healthier. Sitting where I sit, listening to engineers, I read that same news only one way, and it was the wrong way. (Factor 5)

Looking back, my analysis was buried entirely in the competitiveness of a single company. I underestimated what happens to the number two and three players when the whole pie expands, and how powerful catalysts from outside the income statement can be.

AMD

AMD is one of the stocks where my investment has worked out so far, and one of the reasons it worked was precisely the Intel lessons above. I am still bullish on AMD, and here is why, mapped onto the factors.

1. AMD has outstanding chip design talent across CPUs, GPUs, and FPGAs, all relevant to AI. Chip design talent remains a severe bottleneck. As Intel lesson 3 taught me, in a demand surge that accumulation is an asset in itself. (Factor 2)

2. The AI era is creating a CPU bottleneck, and AMD, with its strong moat in CPUs, stands to benefit. Its server CPU share keeps climbing. (Factors 1, 2)

3. The same goes for data center GPUs. AMD’s share is small next to NVIDIA’s, but when demand itself explodes, the overflow inevitably lands with the number two player. This is Intel lesson 1 applied directly. Second place does not have to become first. (Factor 1)

4. The question hanging over AMD is always “Can it beat NVIDIA?” But AMD has a more urgent goal. Can it grow fast enough to absorb the demand exploding right now? The gap between those two questions is exactly where the investment opportunity sits. (Factor 4)

5. Never underestimate a CEO like Lisa Su, who took a company from the edge of bankruptcy to where it stands today over the course of a decade. This is the same line item as Intel lesson 5. (Factor 5)

On top of all this, CPUs, GPUs, and FPGAs each serve independent demand, so even if the accelerator race disappoints, the company does not fall apart. That floor (Factor 3) is what makes this a position I can hold for a long time.

MRVL

Marvell is another stock I have been analyzing and watching closely for months. Here is why I like it.

1. The AMD frame applies to Marvell too. It does not need to be number one in custom ASICs. With every hyperscaler racing to build its own chips, Marvell’s DSP-based ASIC business captures the expansion of the pie directly. (Factor 1)

2. Marvell has been quietly widening its scope. People still call it a custom ASIC company, but in reality it has been expanding into a data infrastructure semiconductor company that designs custom compute together with scale-up networking and optical interconnect. I judged there was still time before that gap fully closed, and that was the investable window. (Factor 4)

3. NVIDIA’s investment was decisive. NVIDIA has repeated a pattern of putting equity directly into the supply chain it cannot do without, and Marvell joining that list signaled a bigger role for the company in optics and interconnect. Once I understood why NVIDIA invested, my conviction hardened another notch. (Factor 5)

4. Marvell has been hiring at a staggering pace lately. Reading the direction of the job postings, the company appears to be preparing to run multiple hyperscaler custom programs in parallel while integrating optics. Hiring is a signal that moves several quarters ahead of earnings. (Factor 2)

Stocks I Watch Through the Same Lens

Here are some of the names I follow with interest. Most of them I have already covered in depth in individual articles, so if you want the full analysis, I encourage you to look those up. Here I will only note which factors put each one on my list.

Subscribe now

QCOM. Longtime readers of my articles will know this one well. People still call it a smartphone chip company, but in reality it is expanding into data center accelerators and custom silicon (Factor 4). The diversification is also attractive, with data center being added as a new growth axis on top of handsets, automotive, and IoT (Factor 3).

That said, while the data center strategy and product roadmap are now official, meaningful revenue contribution has yet to show up. This is a stock where I build conviction quarter by quarter as customer wins convert into actual revenue.

CSCO. The image of an aging network equipment company hides what Cisco has become. It owns an AI infrastructure stack across multiple layers, from its own networking silicon (Silicon One) to the optics internalized through the Acacia acquisition to security (Factors 2, 3). And it has in fact been hiring aggressively in exactly these directions lately.

What matters is that the time it takes for that image to catch up is itself the investment opportunity (Factor 4).

MU. I believe the change HBM has brought to the memory profit structure is a structural shift that outlasts the cycle (Factor 1). Growing HBM share and capacity investment point in the direction of growth (Factor 2), and being the only DRAM producer manufacturing in the United States adds policy-driven flows on top (Factor 5).

The business is concentrated in memory alone, so theme resilience (Factor 3) is low. But for someone with strong conviction in the memory sector, that concentration is the appeal. The catch is that this conviction is sustained only by continued study of the sector, and keeping up that study is the precondition for owning a stock like this.

LITE & COHR. The starting point is the judgment that lasers and InP capacity are the bottleneck relative to AI bandwidth demand (Factor 1), and NVIDIA’s investments and purchase commitments were the catalysts that arrived ahead of the earnings (Factor 5). Their laser portfolios are diversified across EML, CW, and VCSEL, so whether the technology goes pluggable or CPO (co-packaged optics), their lasers are needed either way. Indeed, Lumentum has announced laser capacity expansion including a new US production facility, and Coherent is also expanding its manufacturing capacity and R&D investment (Factor 2).

Still, these are companies concentrated in the single layer of optics (Factor 3), so as with Micron, continuous study of the sector is essential.

CRDO & ALAB. These are interconnect names structured so that you do not have to call the direction of the technology (Factor 3). Their hedges run along different axes, though. Credo straddles both sides of the media debate, copper through AECs (active electrical cables) and optics through its optical DSPs, so it benefits however the argument resolves. Astera Labs sits where a new product slot opens with every protocol generation, from retimers to switches to scale-up standards.

Because both are small companies, their stamina to survive gaps between catalysts (Factor 5) is something I verify every quarter.

NOK. There is a wide gap between the old name, a telecom equipment vendor, and what Nokia is today after vertically integrating the optical stack through the Infinera acquisition (Factor 4). On top of that, NVIDIA paired an equity investment with an AI-RAN partnership. Putting AI compute into the radio access network (RAN) means telecom infrastructure itself could be redefined as AI infrastructure, an option entirely separate from optics. The investment is the catalyst, and the cash flow from the telecom business is the stamina (Factor 5).

But optics and AI are still a modest share of the whole company, so the pace of the transition (Factor 2) has to be checked quarter by quarter.

STM. Spread across analog, power, sensors, and MCUs, it has the highest theme resilience (Factor 3) on this list. It holds meaningful positions in individual themes like SiC power semiconductors and silicon photonics, but each business is a small enough share of total revenue that no single theme can carry the stock on its own.

In June of this year, however, the company raised its 2026 data center revenue target to roughly $1 billion, citing AI infrastructure demand and progress on capacity ramp-up, and indicated revenue could double again in 2027 if current customer engagements hold (Factor 5). What I hear on the ground points the same way. With broad, evenly distributed businesses providing stability and an industry tailwind now on top, it is well suited to the role of dampening portfolio volatility.

Step 4. Buying and Selling

Once the sector and stock assessments are done, I set my own rules for buying and selling. As I said in the introduction, this article is not a stock or portfolio recommendation, so I will not be disclosing my positions. What I want to show instead is the algorithm, with examples, of how I think about buying and selling and by what criteria.

Position size is set by the factors.

Only stocks in sectors where I hold firm conviction earn a large position.

For me, that is the memory sector.

Conversely, even with conviction in the sector, for stocks where only one or two factors check out, or where the volatility exceeds what I can absorb, I never commit serious money no matter how hot the theme.

Small optical component names like Sivers are an example.

The durability of the optics sector (Factor 1) is clear and the technology is interesting, but revenue is small, the business leans on a handful of customers and partnerships, and the financial stamina to survive delayed catalysts (Factor 5) is unproven.

A company like that can be right about the theme and still not live long enough to see the theme become earnings. So when I do enter, I either keep the position small or treat it as a short-term trade.

Once you start sizing positions by expected return, you get the inversion where the riskiest stock gets the most money. Size should be set by conviction, not by hope.

Never buy in a hurry.

I believe the market always offers another chance. With a position bought in a chase, out of fear of missing out, the bad cost basis is a problem, but the bigger problem is that you end up holding it in an anxious state. A high cost basis makes you flinch at every small dip, and the moment you start flinching, price takes over the judgment that the factor assessment should be making.

In the recent big correction across semiconductors, I waited. All I did on the way down was check whether any factor assessment had changed, and once I concluded nothing had, I rebalanced part of the portfolio and actually added to my highest-conviction names. For anyone who keeps assessment and price separate, a correction is a discount window.

I do look at charts for timing. Not as a believer, but every so often I have AI run a chart analysis to see whether the trend is intact and where the meaningful price levels sit, and I treat that as one input among several.

When do I sell?

I sell when the answers to the factors deteriorate. That means my judgment on sector durability has changed, or the company has stopped hiring and investing in the direction of growth. If any of those breaks, I cut the position regardless of the price. Conversely, if the assessment is intact and only the price has fallen, that is closer to a buying opportunity.

There is one more case. When the perception gap has fully closed in the good direction, meaning everyone now tells the same story about the company, my edge is gone. Then I trim, even if nothing has gone wrong.

Stop losses work the same way. I sit through declines as long as the factor assessment holds, but a sharp drop is my alarm to re-examine everything. If the second look reveals something I had missed, meaning the price knew before I did, I cut without hesitation. A stop loss is the cost of admitting you were wrong, and trying to avoid paying it can cost you both your account and your judgment. The reference point when cutting is the current factor assessment, never the cost basis. What you paid has nothing to do with the company’s future.

Rebalancing happens alongside the quarterly factor review. The principle is simple. Check whether each position’s weight matches the size of my conviction, and fix it where it does not. Names that have run up beyond their conviction get trimmed, and the proceeds move into names whose assessment is unchanged but whose price is depressed.

The rebalancing I did in the recent correction was this principle in action. The result is that you end up selling what got expensive and buying what got cheap, not because you predicted the market, but as the automatic byproduct of keeping conviction and weight aligned.

Closing Thoughts

Let me stress once more that I am not a professional investor.

I am still learning, watching and studying the many excellent professional investors ahead of me. What I have written here is simply how I analyze sectors and stocks and how I invest, grounded in the areas where I am confident, which are technology and industry analysis.

The factors and principles laid out here are not finished either. With every earnings report, every new organization, every published standard, the assessments keep moving, and my method will keep getting patched every time it turns out to be wrong.

I hope this article becomes the start of many discussions with my subscribers, where we learn from each other’s perspectives. If, through that process, each of you ends up building an investment methodology of your own, this article will have done its job.

The market and a number of recent reports have grown bearish on memory. The thesis runs that the wave of capacity additions arriving around 2027 and 2028 will let supply catch up with demand, prices will roll over, and a de-rating will follow just as the old cyclical script dictates.

I disagree with that view.

The bear case still prices memory as a commodity, and the technological terrain underneath that assumption is changing right now. Trace the structural shift underway in the memory industry at the engineering level and it becomes clear that the “same standard part no matter who makes it” premise, the one the cycle-top argument leans on, applies less and less to the memory that is coming.

At the center of that shift is the HBM base die.

The base die moves to a logic process, customer circuitry moves into it (cHBM), and at the end the memory sits directly on top of the customer’s compute die (memory-on-logic).

I expect that at the end of this progression memory companies will shift from commodity manufacturers toward custom silicon partners, and the way memory companies are moving today points in that direction. This article walks through the technical background, the actual moves being made across the industry, and what this change means for memory revenue structure and valuation. By the end, you will understand why I remain bullish on memory, and why that reasoning rests on a change in the structure of the memory business itself.

Table of Contents

1. HBM, cHBM, and memory-on-logic, three structures distinguished

2. From cHBM to memory-on-logic, the order in which the industry is moving

3. What Silicon Valley is hearing

4. How the memory revenue structure changes

5. What to watch from an investment standpoint

   ---

Disclaimer

The figures and timelines in this article are drawn from company announcements, public reporting, and published papers. The interpretation of those figures and the outlook expressed here are entirely my own analysis, and nothing here constitutes a recommendation to buy or sell any specific security. Responsibility for investment decisions and their outcomes rests with the individual investor.

Subscribe now

1. HBM, cHBM, and memory-on-logic, three structures distinguished

HBM

HBM (High Bandwidth Memory) stacks DRAM dies eight, twelve, or sixteen high and connects them vertically with TSVs (Through-Silicon Vias). At the bottom of the stack sits the base die. It handles memory control, I/O, and test logic, and serves as the gateway that links the DRAM stack to the outside world. The finished HBM stack is placed beside the GPU on a silicon interposer and connects to the GPU through the interposer’s wiring. The stack itself is 3D, but because it sits side by side with the GPU rather than on top of it, the arrangement is called 2.5D. In HBM4 the interface between the GPU and the stack is 2,048 bits wide, and those signals leave the PHY at the edge of the GPU die, cross the interposer wiring, and pass into the base die.

The DRAM core dies above are largely fixed in structure, since the capacitor array that stores data occupies most of their area, so the only layer in the stack that carries logic is the base die at the bottom. Every byte moving between the GPU and memory passes through this layer, which makes the base die’s processing capability the determinant of the entire stack’s bandwidth and power efficiency, and that burden grows with each generation. The interface has doubled from 1,024 bits in HBM3 to 2,048 bits in HBM4, per-pin speed has risen alongside it, and the complexity of signal processing, channel management, and power management has climbed with each step.

For that reason memory companies have worked on the base die first when improving HBM performance, and the starting point was the process used to build it. Through HBM3E the base die was fabricated on a DRAM process, the same as the DRAM dies above it. A DRAM process is specialized for high capacitor density, so putting logic circuitry on it yields something larger and slower than the same logic built on a logic process. The work the base die must do grows every generation, and a DRAM process cannot keep pace, so from HBM4 the base die began moving to a logic process.

SK hynix builds it on TSMC’s 12nm process and Samsung on its own foundry’s 4nm process, while Micron alone keeps the base die on its existing DRAM process through HBM4 on cost grounds and plans to move to a TSMC logic process from HBM4E.

Even with the process change, though, the design of this base die still belongs to the memory company, and the product remains a standard part conforming to JEDEC specifications.

cHBM

Moving the base die to a logic process does not remove the problem built into the HBM structure. The GPU side is chronically short of die area for computation, and part of that scarce area is taken up by data-moving circuits such as the HBM interface and the memory controller.

This is where the concept of cHBM (custom HBM) begins.

The idea is to push the circuits on the GPU side that communicate with memory down into the base die, which sits directly beneath the memory stack anyway. The GPU then spends the freed area on computation, and the memory-related logic runs right below where the data is stored. Handle preprocessing or compression in that same spot and the volume of data that has to travel to the GPU falls, which saves power as well.

The catch is that a JEDEC-standard base die is the common denominator for every customer, so it has no room for this kind of relocation tuned to a specific customer’s architecture. In other words, cHBM is the approach of departing from the standard and designing the base die in a custom way to build the HBM. As an aside, the industry sometimes refers to conventional HBM that follows the standard as sHBM (standard HBM).

There are several ways to design a cHBM base die. They range from the memory company designing it to the customer’s specification, to an ASIC (Application-Specific Integrated Circuit) partner such as Marvell defining the base die jointly with the three memory makers, to the customer designing the base die logic directly, as NVIDIA does.

What does it mean for part of a customer’s circuitry to be designed into the base die.

It means the business model of memory, long regarded as a commodity, begins to shift from this point. An HBM carrying a customer’s circuitry is that customer’s dedicated part and cannot be sold elsewhere, and from the customer’s side, memory with its own circuitry embedded cannot easily be swapped for another supplier’s product. The commodity premise that anyone’s part is interchangeable as long as the spec matches no longer holds from cHBM onward.

memory-on-logic

By the time you reach cHBM, the base die looks less like a chip for managing memory and more like the customer’s logic chip. The interface belongs to the customer, so does the controller, and so do the functions filling the remaining area. That raises a new question. Is there a reason to build a separate base die at all. Why not place the GPU proper, the compute die, in the base die’s position and stack the memory directly on top of it. That structure is memory-on-logic. The compute die absorbs everything the base die used to do, the interposer disappears, and the memory stack is stacked in 3D directly on top of the compute die.

At this point the physical conditions of the connection change. Instead of joining the HBM stack and the GPU through a 2,048-bit edge interface, the connection points spread across the entire bonded interface. Vertical bonding methods such as TSV and hybrid bonding push the connection density from tens of thousands to hundreds of thousands, and the distance a signal travels shrinks from the millimeters of interposer wiring to tens of micrometers in the vertical direction.

Bandwidth changes by an order of magnitude, and so does the energy needed to move a bit. A 2025 paper from Georgia Tech and SK hynix analyzed a memory-on-logic structure as capable of 64 times the throughput and three times the energy efficiency of 2.5D. The numbers are academic modeling and need not be taken at face value, but they are a good illustration of the direction and magnitude of the improvement.

The unsolved problem with memory-on-logic is heat. Place DRAM on top of a compute die burning hundreds of watts and the DRAM blocks the compute die’s thermal escape path while absorbing that heat itself. And DRAM, which is sensitive to temperature, sees its refresh interval shorten under heat, opening the door to degraded performance. In the end, how this heat problem is solved is likely to be the single largest variable determining the success of memory-on-logic.

The story so far starts not with HBM as a whole but with a single die at the bottom of the stack.

That base die moves from a DRAM process to a logic process (HBM4), customer logic moves into it (cHBM), and finally the base die’s position is taken by the customer’s compute die (memory-on-logic).

This progression need not be read as a linear generational replacement, however. cHBM is the most realistic commercialization path for the next several years, while memory-on-logic is a more radical structure that reaches the mainstream only once the heat and yield problems are solved. Rather than one immediately replacing the other, the two structures are likely to be adopted in parallel for some time, depending on the customer’s power budget, workload, and packaging cost.

The closer the base die moves to the customer’s silicon, the closer the memory business moves to a custom silicon business, and….

I think this is precisely the point the market has yet to properly grasp about the upside in the memory business.

The three memory makers are still valued as cyclical stocks that move on bit growth and ASP, and the changes in revenue and margin structure that the custom transition will produce have never yet shown up in reported numbers, so they have had no opportunity to be reflected in valuation models. That is how significant this change is, and exactly how the business model and revenue structure change is covered in detail below.

Subscribe now

2. From cHBM to memory-on-logic, the order in which the industry is moving

cHBM is already visible on product roadmaps. Application of cHBM keeps coming up for the NVIDIA Feynman generation slated for 2028, though NVIDIA has not officially confirmed the details, so this is better treated as a key point to watch for that generation than as an established fact. Samsung, SK hynix, and Micron have all placed custom HBM on their official roadmaps, and Marvell has announced that it is co-developing a custom HBM architecture with the three memory makers. The shift of customer-specific base dies into the place of standard base dies has already entered the execution stage.

While cHBM settles onto roadmaps, the next stage, memory-on-logic, has produced its own case with a disclosed production timeline.

It is not yet a product on the market. The AI250 accelerator carrying HBC Gen1 is slated to launch in mid-2027, with Gen2 on the follow-on roadmap. HBC belongs to the memory-on-logic family, but the memory is LPDDR (Low Power DDR) rather than HBM, and the die placed beneath the stack is a dedicated near-memory accelerator rather than the GPU proper.

Consider the structure. Qualcomm separated the AI accelerator from the SoC. It stacks an LPDDR stack on top of the separated accelerator die and connects them with TSVs. This HBC unit and the SoC sit side by side on an ordinary organic substrate. There is no silicon interposer, and no 2.5D advanced packaging such as CoWoS. The design is meant to route around two bottlenecks at once, the HBM supply shortage and the shortage of advanced packaging capacity.

Qualcomm chose LPDDR for capacity and power. LPDDR is a DRAM family designed for low power, so the power burden per stack is small and it lends itself to capacity expansion. Qualcomm’s answer to the memory-on-logic heat problem sits right here. Rather than the hot GPU proper, it places a low-power dedicated accelerator underneath, and uses low-power LPDDR for the memory as well, so the whole thing fits within the thermal budget.

Qualcomm claims six times the bandwidth per watt of HBM and 200 times the capacity per watt of on-chip SRAM. The stated figures will need a detailed whitepaper and real hardware to verify, but movement on the customer side has already appeared.

Saudi Arabia’s Humain has included AI250 racks carrying HBC Gen1 in its deployment plans, and according to reports Microsoft’s Nadella also spoke of deploying Qualcomm HBC in Azure data centers. That a product still a year from launch already has the names of a large infrastructure customer and a hyperscaler circulating around it is itself evidence of demand for this structure.

Qualcomm is not the only one moving toward memory-on-logic.

There were reports that SK hynix had been discussing, since 2023, a method of stacking HBM directly on top of a processor with several fabless companies including NVIDIA, and a manufacturing approach of bonding memory onto a logic wafer using TSMC’s wafer bonding came up alongside it.

Taiwan’s ASIC design house GUC has proposed DoL (DRAM-on-Logic), placing one to four layers of DRAM on top of logic. Whether what goes on top is LPDDR or an HBM-style DRAM stack, and whether the logic underneath is a dedicated accelerator or the GPU proper, varies from case to case, but the direction is the same in every one, memory drawing steadily closer to logic until it finally sits on top of it.

And as noted, memory sitting on top of logic means memory becoming one body with a specific customer’s chip.

At the end of this progression from cHBM to memory-on-logic, the memory business gradually turns into a custom silicon business, and what Silicon Valley has been hearing lately suggests that the change has already begun at the organizational level.

So what matters from an investment standpoint.

It is how this shift toward memory companies becoming custom silicon companies changes their revenue structure, where along the supply chain the profit moves, and what to track the progression by. These are the things to focus on. What follows below covers them in detail.

3. What Silicon Valley is hearing

There is one more piece of evidence that memory-on-logic is a real trend.

Read more

NVIDIA headlines these days mention rack names more often than chip names. Just this past weekend, a SemiAnalysis claim that a rack called Kyber had slipped by more than a year collided with NVIDIA’s rebuttal that its roadmap is intact, and the related stocks swung on it within a single day.

The trouble is that reading this kind of news requires knowing far too many names just to parse a single sentence. What is Rubin? What is Kyber? Why did NVL144 suddenly become NVL72? And when is Feynman supposed to arrive?

The confusion is entirely reasonable. A name like Vera Rubin Ultra Kyber NVL576 packs five separate pieces of information into one string: the CPU, the GPU, the generation, the rack architecture, and the scale of the interconnect.

It looks complicated, but these names sort neatly into three layers: chip, link, and rack. Start at the bottom with the chip and work your way up one layer at a time, and the naming maze turns out to run on surprisingly simple rules.

Once you have those rules in hand, you can take the SemiAnalysis claims that shook the market and pick them apart sentence by sentence. I’ll say this up front: I believe those claims planted far more fear in the market than what they actually said. The rest of this piece walks through why, one piece of evidence at a time.

Table of Contents

1. The chip layer: the logic behind the names

2. The link layer: NVLink and scale-up

3. The rack layer: Oberon, the present

4. Kyber: the problem the next rack has to solve

5. Reading the sentences closely

6. So is midplane yield actually a problem?

The figures in this piece are based on public sources; the interpretations and outlook are my own analysis. Nothing here is a recommendation to buy or sell any security. Roadmaps and specifications reflect announced plans and may change.

1. The chip layer: the logic behind the names

NVIDIA ships a new GPU architecture every year. An architecture is the chip’s underlying design, and a single architecture spawns multiple products, from data center parts to workstation cards.

The generation selling today is Blackwell. The one ramping in the second half of this year is Rubin, and the one arriving in 2028 is Feynman. All of them are named after scientists: Blackwell for the mathematician David Blackwell, Rubin for the astronomer Vera Rubin, and Feynman for the physicist Richard Feynman.

Separately from its GPUs, NVIDIA also builds its own CPUs, which sit next to the GPU and handle data preparation and orchestration. These follow their own naming lineage, Grace, Vera, Rosa, all named after women scientists.

So when the news says Vera Rubin, it isn’t referring to a single chip. It’s a platform name that bundles a CPU called Vera with a GPU called Rubin.

The origin of this naming scheme is a fun bit of trivia. The previous generation, Grace Hopper, split one computer scientist’s name in two: Grace for the CPU, Hopper for the GPU. Vera Rubin does the same with one astronomer’s name.

Starting with the next generation, Rosa Feynman, the pairing shifts to two different scientists, the Nobel-winning medical physicist Rosalyn Yalow and the physicist Richard Feynman. But the rule itself survives: the CPU name comes first, the GPU name second. Know that rule and you can decode any codename on sight, even one you’ve never seen before.

Finally, there’s Ultra. Ultra is not a new architecture. It’s a beefed-up version of the same generation, and what gets beefed up is the number of dies in a package.

A die is the actual silicon chip. Because there are physical limits to how large a single die can grow, the industry now places multiple dies side by side on one substrate and packages them as a single giant chip. Rubin uses two dies per package.

Rubin Ultra adds more dies and also increases the HBM (High Bandwidth Memory), the high-speed memory stacked right next to the GPU. In AI workloads, how fast the GPU can be fed data determines performance, so HBM capacity and bandwidth effectively set the ceiling on what the chip can do.

To sum up, the chip layer has three kinds of names: the GPU generation (Rubin, Feynman), the CPU generation (Vera, Rosa), and the enhanced-variant marker (Ultra). But today’s AI models have grown too large for any single GPU to handle, no matter how good. The real competition is happening one layer up, in how many of these chips you can bind together to act as one.

2. The link layer: NVLink and scale-up

Training or serving a large language model takes dozens or hundreds of GPUs. The catch is that these GPUs don’t work independently.

A single model is sharded across many GPUs, so they constantly exchange intermediate results mid-computation. If that communication is slow, even the most expensive GPUs end up idle, waiting on data. How fast and how widely you can bind GPUs together determines the performance of the whole system.

There are two ways to connect them. In the current generation, wiring GPUs together inside a single rack over ultra-fast dedicated links, so they effectively behave as one giant GPU, is called scale-up. Stitching those racks together over a conventional network to grow the overall cluster is called scale-out.

For the same data, scale-up traffic inside a rack is far faster and lower latency than scale-out traffic between racks. So the basic recipe is to pack the chattiest GPUs into one rack, then network those racks together to reach data center scale.

NVIDIA’s scale-up technology is NVLink, a dedicated pathway that connects GPU to GPU far faster and with far less latency than a regular network. GPUs bound by NVLink can read and write each other’s memory as if it were their own.

The chip that ties all of these GPUs together, acting as a switchboard, is NVSwitch. NVSwitch belongs to the scale-up side along with NVLink, sitting among the GPUs inside a rack so that any GPU can talk directly to every other GPU. The set of GPUs that moves as one body this way is called a scale-up domain.

Scale-out, the rack-to-rack connection, is handled by separate networking gear. NVIDIA offers two options there: InfiniBand and the Ethernet-based Spectrum-X.

NVL is short for NVLink, and the number after it is the count of GPUs bound into a single scale-up domain. NVL72 means a system where 72 GPUs move as one. The bigger the number, the larger the model you can run without hitting communication bottlenecks.

There’s one source of confusion: the counting convention changed midstream. The first Rubin rack was originally announced as NVL144. With two dies per GPU package, NVIDIA counted 144 dies across 72 packages and called it 144.

It then reverted to counting packages, for naming consistency with the prior generation’s GB200 NVL72, and the rack was finalized as VR200 NVL72. VR200 is the successor to the GB200 numbering: just as GB200 stood for the Grace CPU plus Blackwell GPU combination, VR200 stands for Vera plus Rubin. So the NVL144 in last year’s articles and the VR200 NVL72 in this year’s articles are the same machine.

3. The rack layer: Oberon, the present

The rack, which kept showing up above as the boundary of scale-up, is the third layer. A rack, using GB200 NVL72 as the reference, is a cabinet 0.6 meters wide, about 1.1 meters deep, and about 2.2 meters tall, weighing in at 1.4 tons, with trays of GPUs slotted in one on top of another.

NVIDIA gives its racks their own codenames too. The rack in service since the GB200 generation, carrying through to this year’s Rubin, is Oberon.

In Oberon, compute trays carrying GPUs and CPUs slot in horizontally, layer by layer, with switch trays carrying NVSwitch chips in between. At the back, bundles of copper cables connect every tray’s GPUs to the NVSwitches.

The heat is intense enough that the rack is liquid-cooled rather than air-cooled. In the VR200 NVL72 configuration, one rack holds 72 GPU packages and 36 Vera CPUs.

Copper is the cheapest and most proven way to wire GPUs together. Its weakness is distance. The longer the wire, the more the signal degrades, at which point you either add amplifiers or switch to optics.

Optics requires converting electrical signals to light and back again, and the transceivers doing that conversion burn double-digit watts per port. That’s a cost copper simply doesn’t have, since electricity flows through it directly.

So the GPUs you want to fuse into one body over NVLink need to sit within copper’s reach, which means packing them as close together as possible inside a single rack. That’s why 72 GPUs get crammed into one cabinet, and it’s also why the physical size of the rack caps the size of the scale-up domain.

Copper saves power on the wiring, but the power the GPUs themselves draw stacks up with density. With 72 GPUs each pushing past 1 kW packed into one cabinet, per-rack power has climbed from roughly 40 kW in the Hopper generation to 120-130 kW for Blackwell and 190-230 kW for Rubin.

A single rack now draws as much electricity as a small factory. At that point the rack becomes the anchor around which a data center’s entire power and cooling design is built. And the next rack triples that number.

4. Kyber: the problem the next rack has to solve

Kyber is the new rack debuting alongside Rubin Ultra in the second half of 2027. The goal is simple to state: where Oberon slots its trays horizontally, Kyber stands its compute trays vertically to push density higher, fitting 144 GPU packages in one rack. That’s exactly double Oberon, and per-rack power surges to the 600 kW class.

In the Rubin Ultra generation, Kyber is slated to come in three configurations: NVL72, NVL144, and the flagship NVL576, which binds the GPUs of multiple racks into a single NVLink domain.

What separates scale-up from scale-out isn’t the physical boundary of the rack itself but whether the GPUs are fused into one body over NVLink. Multiple racks joined by NVLink still count as one scale-up domain. The Blackwell generation actually deployed exactly that: GB200 NVL36x2, which linked two racks with copper cables so that 72 GPUs split across them formed a single NVLink domain.

But copper’s practical reach ends at the adjacent rack, which is why scale-up has so far stayed within one or two racks. NVL576 is the first configuration to push that boundary out to a multi-rack scale.

The key question is what connects those 144 GPUs. One analysis puts the copper cable count at over 20,000 if you wire it the Oberon way. Assembly, repair, and reliability management all turn into a nightmare.

NVIDIA’s answer is a midplane: one enormous circuit board. Everything those 20,000 cables did gets replaced by traces printed on the board, and the trays plug directly into it. It’s a design that eliminates the cable harness outright.

But building it is another matter. According to industry analysis, the board laminates three 26-layer boards into a 78-layer stack nearly one square meter in area, with trace spacing at or below 25 micrometers, while holding electrical characteristics within a 5 percent tolerance so the ultra-high-speed signals don’t smear.

More layers kill the signal. A bigger panel means one defect anywhere scraps the whole thing, dragging down yield. The heart of the rack is no longer the GPU. It’s this one board.

The recent delay controversy erupted at exactly this point.

made four claims.

Kyber NVL144 has slipped to 2028 over the manufacturability of this midplane.

NVL576, which binds multiple racks with optics, may also be delayed or limited to small volumes.

NVL72x2, the stopgap that would have doubled scale-up by placing two Oberon racks back to back over copper, was cancelled after pushback from cloud providers.

And the four-die Rubin Ultra was cancelled, leaving only the two-die version.

All four claims live in the layers we just covered: board manufacturability, optical scale-up, copper’s distance limit, and dies per package.

NVIDIA immediately pushed back, saying its roadmap is intact.

5. Reading the sentences closely

NVIDIA’s rebuttal was one sentence: “Our roadmap is intact.”

It did not respond point by point to the specific claims about midplane yield or the NVL72x2 cancellation. So I went the other way and read the SemiAnalysis claims themselves, sentence by sentence.

Let me say this up front.

The problem with these claims is not that every sentence is false.

Each sentence is written so it can be walked back no matter how things turn out, while the overall framing makes it read as if the entire Rubin Ultra generation is wobbling.

That gap, planting far more fear in the market than what was actually said, is the problem.

Start with their headline.

It opens loud: “MASSIVE DELAY” and “delayed by more than 12 months, pushing it back to 2028.”

But Kyber’s official schedule is the second half of 2027, so a delay of more than 12 months means late 2028 at the earliest. Slipping from the second half of 2027 into early 2028 and slipping by more than 12 months, into late 2028 or, taken literally, 2029, are enormously different outcomes. Yet they blur the two together, framing the headline as if something catastrophic has happened to NVIDIA.

Next, the NVL576 sentence: “likely delayed or restricted to small volumes.” That hangs two options off a word of estimation. If it ships on time but in small volume, they’re right. If it ships late, they’re also right.

For this sentence to be wrong, a flagship top-end configuration would have to ship in high volume in its first year, and that rarely happens with any product. They’ve essentially stated the obvious.

What about the NVL72x2 cancellation? NVL72x2 was never officially announced by NVIDIA. It existed only in supply chain reporting as an internal alternative under consideration.

The cancellation of a product that was never announced has no reference point against which to check it, and a company shelving one internal option is entirely routine. Yet this unverifiable sentence gets dressed up with the language of heavy pushback from CSPs and hyperscalers, made to sound like something is deeply wrong inside NVIDIA.

The phrase “no proven solution” works the same way. Even granting, as they claim, that the CPO NVSwitch doesn’t arrive until Feynman, concluding that there is no way to expand scale-up is a leap.

Rubin Ultra’s expansion vehicle was always the copper midplane, meaning Kyber. There is a deployed precedent for copper crossing the rack boundary in NVL36x2. And if optics becomes necessary, an intermediate path like NPO (Near-Packaged Optics), which can be considered without redesigning the switch chip, exists. NPO carries its own homework in signal integrity, power, and serviceability, but a path existing and no solution existing are entirely different statements.

And I’d like to turn the question around: by that same standard, do the AMD MI500X or the next TPU, the products they name as seizing the opportunity, have a proven solution? Measured by products a customer can actually buy, the only company with production-proven rack-scale scale-up is NVIDIA, which has been deploying NVL72 since late 2024.

After building the piece out of sentences this slippery, their final tweet abruptly says NVIDIA will sell far more Oberon-based Rubin racks and Rubin Ultra racks to cover the shortfall. If that’s true, the revenue stays inside NVIDIA and the only thing that changes is the shape of the rack.

In fact, NVIDIA’s official roadmap commitments are made at the chip-generation level (Rubin Ultra, second half of 2027), not for any particular rack format. So if Rubin Ultra ships in 2027 inside Oberon racks, their facts and NVIDIA’s “roadmap is intact” rebuttal are both true at the same time. A piece that opens by scaring people with “MASSIVE DELAY” ends up describing a rack mix adjustment.

And the real problem is that this is not the first post like this.

This year alone, they’ve published a report that put Micron’s share of Rubin HBM4 at essentially zero, a report that made a SOCAMM volume reduction read like a collapse in memory demand, and a report on CPO and 800VDC production slipping that sank the entire optical complex in a single day.

Check back later and Micron has been certified as a Vera Rubin HBM4 supplier alongside SK hynix and Samsung, with volume shipments announced. On the CPO story, NVIDIA’s head of networking publicly rebutted any shipment delay, and the report itself noted that some NPO projects might actually be pulled forward. Optical demand keeps flowing through intermediate paths like pluggables and NPO.

The pattern has been the same every time. Hit the market with strongly worded assertions, and by the time the counterarguments arrive, prices have already moved.

When an information business that sells paid research, consulting, and data products repeatedly publishes reports that move sector prices this way, the rigor of its fact-checking, its conflicts of interest, and the transparency of its positions become questions the market is entitled to ask.

I consider SemiAnalysis reports genuinely valuable as technical material. But I consider the repetition of this kind of behavior genuinely dangerous.

They have become a globally influential institution whose single report can move the price of an entire sector. That influence should come with a matching awareness of the weight their words carry.

6. So is midplane yield actually a problem?

Strip away all the slippery language and this controversy comes down to one technical claim: midplane yield. The only people who know that yield precisely are the board makers and NVIDIA, but public information alone narrows the range of inference considerably.

First, the midplane approach itself is already in production validation. The compute trays of the VR200 NVL72 shipping this year are already a midplane-based, cable-free design, and Jensen Huang personally highlighted at his CES 2026 keynote that this cut tray assembly time from two hours to five minutes.

The small midplane has already graduated to marketing material. What remains in question is one thing: the yield when you scale it to one square meter and 78 layers.

Second, the 78-layer spec is itself a yield-defense design. Building three 26-layer boards separately and laminating them means, under standard practice for high-layer-count boards, you can electrically test each sub-board and join only the known-good ones.

The final yield then no longer collapses as the product of all 78 layers’ process steps. It separates into the yield of the lamination and alignment steps. The chain of defect probability gets cut in the middle, by design.

Third, the economics mean low yield doesn’t kill the schedule. The midplane is a passive component with no active silicon. A 50 percent yield doubles the cost of the board, but on a rack costing millions of dollars, that is not the line item that breaks the cost structure.

The natural consequence of poor yield is higher cost and constrained early volume. For it to translate into a 12-month delay, yield would have to be bad past the point of economics and into physical unmanufacturability. With boards of the same family already running on production lines, I put low odds on that scenario.

Fourth, the timeline. A board for a product launching in the second half of 2027 should, right now, be at the engineering sample and qualification stage, where yields are always low. The observation that “it’s hard to make today” is true of every leading-edge component 15 months before launch, so on its own it is not evidence of a delay.

The diagnostic signals come next. If the schedule is real, orders and capacity expansion should start leaving traces at Taiwan’s high-layer-count PCB makers and CCL (Copper Clad Laminate) suppliers from the second half of this year. Taiwan is one of the few markets in the world where listed companies must disclose monthly revenue, around the 10th of each month, so those traces show up month by month, no need to wait for quarterly earnings.

So here is my judgment. The observation that the midplane is hard to build is probably true. But the logic that translates that observation into a 12-month delay and an opening for competitors does not hold up within SemiAnalysis’s own claims, and sentences that can be walked back no matter the outcome were sold to the market as a confirmed disaster.

The question with actual information value in this debate is which way the 2027 rack mix tilts, Kyber or Oberon. The answer will show up in Taiwan’s monthly board supply chain revenue in the second half of this year, well before any GTC keynote. I’ll be watching those numbers.

Hyperscalers, meaning companies like Amazon, Microsoft, Google, and Meta that build massive data centers, spent about 8% of their total capex on memory in 2023 and 2024.

According to estimates from SemiAnalysis, that number climbs to around 30% in 2026. Nearly a fourfold jump in two years. AI has been vacuuming up memory, supply started falling behind, prices spiked, and even at those prices buyers still cannot secure the volume they need.

One of the cards hyperscalers have pulled out in this crisis is

CXL (Compute Express Link).

Just two or three years ago the industry pronounced it “dead in the AI era,” and several companies quietly shelved their CXL projects.

In 2026, the technology’s standing looks completely different.

Meta went as far as publishing a paper showing how it pulled aging memory out of servers headed for the scrap heap, revived it with CXL, and deployed it at scale in production data centers.

This article takes up three questions.

What is this technology, exactly.

Will it actually see broad adoption.

And if it does, which companies benefit and which get squeezed, and in particular, does HBM demand take a hit.

The last question gets answered with a single investment map: the CXL ecosystem broken into layers, controllers, modules, IP, switches, software, and optical interconnect, showing who occupies each layer, at which stage each layer starts making money, and how much of the expectation is already in the stock price.

Disclaimer

This article is for informational purposes only and does not constitute a recommendation to buy or sell any security. All judgments about the companies and technologies mentioned are personal opinions based on publicly available information, and responsibility for any investment rests with the investor.

Part 1. The Concept: What Is CXL

1. From the Ground Up: How Data Moves Inside a Computer

To understand CXL, you only need to grasp two concepts first: what PCIe is, and what a memory channel is.

Think of a computer as a city.

The CPU is city hall.

Every decision about who does what and when gets coordinated there.

The GPU is a giant factory.

City hall issues the orders, but the actual heavy labor, the AI computation, happens at the factory.

The SSD and the other components are buildings scattered across the city, and the city runs on data moving between these buildings and city hall.

When you plug a graphics card into a motherboard, when you connect an SSD, it all rides this road.

Each generation gets faster: PCIe 6.0 is twice as fast as 5.0, and 7.0 doubles it again.

The speedups come from two levers.

Send trucks more often (push more electrical signals per second), or load more cargo onto each truck (pack more data into each signal). In practice, 6.0 doubled the information carried per signal (PAM4), and 7.0 doubled the signaling rate itself. There is also a lane count: x4, x8, x16, with more lanes carrying more data at once.

The names look complicated, but PCIe is simply the standard road that computer components use to exchange data.

Memory, meaning DRAM, has never used this highway.

Memory exchanges data with the CPU so frequently and so fast that the system puts a dedicated warehouse right next to city hall and lays a private direct road between them. That direct road is the memory channel. No traffic lights, city hall vehicles only, and incomparably faster than the highway.

The problem is that the number of these direct roads is fixed when the city hall building, the CPU, is designed.

The reason they cannot be added later comes down to the physical structure of the chip. Attaching one channel requires hundreds of pins (the metal contact points where wires connect) coming out of the CPU die, plus a memory controller circuit etched into the silicon to run that channel.

Pins and circuits are like doorways set in a building’s concrete: the moment the chip comes off the line, they are final. You cannot pave new direct roads to a finished city hall, so a server CPU with eight memory channels has eight, and one with twelve has twelve. You can build a bigger warehouse at the end of each road, but high-capacity modules get expensive fast. The total memory a server can hold is chained to the CPU bottleneck.

That constraint has been the founding assumption of server design for decades.

The factory (GPU) side solved the same problem in a more extreme way: it eliminated the need for roads entirely by stacking the warehouse directly on top of the workbench.

That is HBM (High Bandwidth Memory).

DRAM dies are stacked vertically inside the same package as the GPU and wired straight in through thousands of connections, so no road on earth can match it for speed.

The price is equally extreme. The moment the package is assembled, the capacity is locked forever, and the manufacturing difficulty makes it the most expensive memory there is. City hall and the factory end up trapped in the same dilemma: the faster the memory, the less freedom you have on capacity.

2. What Exactly Is CXL

If you cannot pave any more private direct roads, the idea is to connect a distribution center to the highway that is already there. Getting to the distribution center means taking the highway, so round trips take a bit longer than the direct road. In exchange, you can build as many distribution centers as you want, at whatever size you want. Decoupling memory capacity from the CPU’s fixed count of private roads, and paying for it with a little speed, is the essence of the CXL trade.

But wait, does mixing in slower memory slow the whole system down?

It does not, because not all data demands top speed. Data has temperature. There is hot data you touch every moment, lukewarm data you look at a few times a day, and cold data you merely keep. Hot data belongs in fast memory, but there is no reason to give lukewarm data the most expensive seat in the house.

So a well-built system divides storage into tiers and places data in the tier that matches its temperature. This is memory tiering.

The real hierarchy is far more granular, from SRAM caches inside the CPU die down to archival storage, but for this story only two tiers matter.

In city logistics terms: the dedicated warehouse next to city hall, and the long-term storage warehouse on the outskirts. Closer means faster, pricier rent, and less space; farther means slower, cheaper, and vast.

In computer terms, local DRAM next to the CPU is the dedicated warehouse, and the SSD is the long-term warehouse on the edge of town.

This two-tier structure was good enough for decades. Most data was either hot or cold, and what little fell in between was small enough in absolute terms that buying a bit more DRAM covered it without strain.

However, AI changed the game.

First, the top of the pyramid ran out of speed. DRAM could not keep up with the pace at which GPUs churned out computation, so, as we saw in Section 1, a new tier appeared at the very top: HBM, the warehouse stacked directly on the workbench.

Then, even with a taller peak, capacity ran out. The data that needed processing swelled beyond what a single server’s DRAM could hold, and with DRAM prices soaring, the cost of “just buy more” became unbearable. For the first time, lukewarm data, too precious for the SSD but too wasteful for DRAM, started demanding a seat of its own. The dedicated warehouse (DRAM) and the outskirts warehouse (SSD) differ in round-trip time by a factor of several hundred, so a new tier was needed between them, and CXL memory slots into exactly that spot, the distribution center.

AI put HBM on top of the pyramid, and now it is wedging CXL into the middle.

The thing to remember is that CXL memory is not a picture of one memory displacing another. It is a picture of adding a floor to the pyramid. Getting this exactly right is very helpful for understanding the investment insights that come later.

With its place in the pyramid settled, what remains is how it works.

Technically, CXL keeps PCIe’s physical wiring as is and layers memory-specific conversation rules on top.

There are three: CXL.io for basic communication, CXL.cache for when a device peeks into the CPU’s cache, and CXL.mem for when the CPU reads and writes external memory as if it were its own. Devices are classified as Type 1, 2, or 3 depending on which rules they combine, and you can skip the taxonomy.

Just remember that in practice, “CXL memory” effectively means Type 3, a device that purely adds memory capacity.

Plug a Type 3 device into a server and what does the operating system see?

A chunk of memory with no CPU attached. Essentially a warehouse that sits a little farther away, and operating systems already know how to handle this distinction between near and far memory (it is called NUMA, Non-Uniform Memory Access).

Linux already supports CXL memory in the mainline kernel, and Windows Server 2025 is bringing in CXL memory support in preview form. The software groundwork is effectively in its final stretch.

That covers how a single Type 3 device works. But the same Type 3 memory serves completely different purposes depending on where and how it is installed. Deployment evolves in three stages.

First, direct attach.

A CXL memory module in an SSD-like E3.S form factor plugs into the server. The name invites confusion, but there is no exotic new memory chip inside: the module holds ordinary DRAM, and the CXL controller chip next to it simply translates that DRAM into PCIe language and puts it on the highway. Latency is lowest and the structure is simplest, which is why this is where real volume is flowing today.

Second, pooling.

Dozens of memory modules sit behind a distribution box called a CXL switch, and multiple servers draw from them as needed. A shared distribution center serving several city halls. This matters because data centers have a serious “bought memory sitting idle” problem. Every server gets fully stocked with memory sized for peak demand, and most of the time a good chunk of it sits empty. It is like building a warehouse for the busy season and leaving it half empty most of the year, and this locked-up idle memory is called stranded memory. With a shared distribution center, each city hall keeps only the warehouse it normally needs and rents space from the center when volume surges. Commercial memory pools in the 100TiB class already exist.

Third, fabric.

Possible from CXL 3.1 onward, this is the stage where an entire rack, or several racks, shares one giant memory pool. The goal extends to multiple servers reading the same data together without copying it. Technically the most ambitious stage, and, as we will cover later, the least proven one.

That leaves one last question: how much speed do you actually give up using CXL memory?

The conventional wisdom says “it is far away, so it is too slow to use.” Measurements say otherwise.

In a public demo by controller maker Montage Technology, a server with 1TB of plain DRAM was compared against a server mixing 512GB of DRAM with 512GB of CXL memory, running a database workload. Application-level latency rose by 5 to 10µs (microseconds), while throughput came in at 95 to 100% of the DRAM-only baseline.

With a design that keeps lukewarm data in the distribution center, the perceived penalty has shrunk to nearly nothing.

Part 2. The Verdict: Will It Actually Get Used, and Who Wins

3. So Why Was It Declared Dead

The CXL standard launched in 2019 under Intel’s leadership, but the memory industry pushed it hardest. Samsung opened the game in 2021 with the industry’s first CXL memory expansion device, and even as DRAM slid into a historic glut and price collapse in 2022 and 2023, memory makers kept expanding their CXL lineups.

The math was that lifting the channel constraint would raise the total DRAM per server and create a new outlet for surplus inventory. For memory companies, CXL was a technical standard and, at the same time, a sales pitch for moving more DRAM through a downcycle.

Then the AI boom hit in 2023, data center budgets stampeded toward GPU clusters, and nobody was left to listen to the memory industry’s pitch about loading servers with more DRAM.

AI at the time meant large-scale training, and training means lashing thousands of GPUs into one block that exchanges data nonstop, a world where bandwidth, the road’s total carrying capacity, is everything.

And that world had its own roads. Unlike PCIe, the standard road linking components inside one computer that we saw in Section 1, GPU-to-GPU links get dedicated roads like NVLink, and server-to-server links get a separate road called Ethernet.

Why these roads beat PCIe comes down to paving technology. The wiring technology that fires data between chips is called SerDes (Serializer/Deserializer), and you can think of it as the pavement that determines how much speed a single lane can handle. At the time, PCIe-class SerDes offered low error rates and low latency but comparatively low speed, while the class used by NVLink and Ethernet (112G, 224G) gave up a little on errors and latency and ran roughly three times faster.

In a training world where bandwidth was everything, that 3x gap was fatal. For a chip designer, spending precious die area and power on PCIe/CXL meant forfeiting the chance to run wiring three times faster with the same resources.

So between 2023 and early 2024, hyperscalers and major semiconductor companies suspended or killed CXL projects one after another.

In early 2024, the semiconductor research outlet published an analysis titled “CXL Is Dead In The AI Era,” and its diagnosis, a market where only the sellers were enthusiastic and the buyers had left, became the industry consensus.

That diagnosis was not wrong for its time.

But it rested on one assumption: that AI workloads meant training.

What changed was not so much the CXL technology as that assumption.

4. Three Reasons This Time Is Different

The first change: the workload shifted from training to inference.

Inference is what happens when an AI service like ChatGPT actually answers a user’s question. Inference has a different bottleneck than training. When an AI works through a long conversation or document, it stores intermediate results in memory so it does not have to recompute everything it already read. That notepad is the KV cache. It snowballs as the conversation grows, and on the latest large models the KV cache alone eats 80 to 120GB per GPU. HBM is a resource where adding capacity is brutally expensive, so keeping this entire notepad in HBM is like stacking newspapers in a bank vault.

The approach of pushing rarely-read notes down to CXL memory came out of exactly this, and public demos are reporting meaningful gains in throughput and energy per token. In the age of inference, capacity matters as much as bandwidth, and capacity is what CXL does well.

The second change: the economics of the memory crisis.

Look at the numbers from the opening again. Memory going from 8% to 30% of hyperscaler spending. When DRAM gets this expensive and this hard to find, two behaviors become rational. One is not letting the memory you already bought sit idle: pooling. The other is reusing memory you were going to throw away.

The Vistara paper Meta presented at ISCA 2026 is exactly the latter. Meta pulled DDR4 memory from servers at end of life, attached it via CXL as expansion memory for new DDR5 servers, and deployed it at hyperscale in production. It built everything itself, from a dedicated ASIC to operating system support. A calculation that never penciled out when memory was cheap now pencils out because of the memory crisis.

The third change: the underlying PCIe got fast enough.

The 3x relative gap with the Ethernet class from Section 3 has not closed, since both sides keep evolving, so CXL still is not replacing the GPU expressways. But for memory expansion, its native job, what matters is the absolute bandwidth the PCIe link itself carries, and that bandwidth has crossed into memory-channel territory. A PCIe 5.0 x16 link at roughly 64GB/s per direction was worth just over one DDR5 memory channel (about 51GB/s per channel), but PCIe 6.0 x16 at roughly 128GB/s does the work of two and a half channels. With a single CXL link starting to deliver genuine memory-channel bandwidth, the road to the distribution center has finally gotten wide enough to make the distribution center worth using.

The proof that these three changes are not just theory is named commitments from the demand side. A technology pushed by vendors and a technology deployed by buyers carry different weight, and through 2023 CXL was the former. Not anymore. Microsoft unveiled CXL-attached memory in its Azure M-series VM preview in November 2025, and Meta published the full-stack paper described above.

The spec and the ecosystem followed.

CXL 3.1 became the default target for new server platforms, CXL 4.0 arrived in November 2025 with bandwidth doubled to 128GT/s on a PCIe 7.0 base, and a large share of new server platforms are now designed with CXL support as a given.

The question of whether servers support it is settled, the question now is how much gets used.

Read more