If Broadcom Is Helping OpenAI Build AI Chips, Here's What They Might Look Like

Analysis OpenAI is allegedly developing a custom AI accelerator with the help of Broadcom in an apparent bid to curb its reliance on Nvidia and drive down the cost of its GPT family of models.

Citing, you guessed it, unnamed sources familiar with the matter, the Financial Times reports that Broadcom's $10 billion mystery customer teased by CEO Hock Tan on Thursday's earnings call was none other than Sam Altman's AI hype factory OpenAI.

While Broadcom isn't in the habit of disclosing its customers, it's an open secret that the company's intellectual property forms the foundation of much of the custom cloud silicon out there.

Speaking to analysts on Thursday's call, Tan said Broadcom is currently serving three XPU customers with a fourth on the way.  

"Last quarter, one of these prospects released production orders to Broadcom, and we have accordingly characterized them as a qualified customer for XPUs and, in fact, have secured over $10 billion of orders of AI racks based on our XPUs," he said. "And reflecting this, we now expect the outlook for our fiscal 2026 AI revenue to improve significantly from what we had indicated last quarter."

OpenAI has been rumored to be developing an in-house alternative to Nvidia and AMD's GPUs for some time now. The chip is now expected to make its debut sometime next year, but will primarily be used internally rather than being made available to external customers, sources told FT.

Whether this means that the chip will be used for training as opposed to inference or simply that it'll power OpenAI's inference and API servers but not virtual machines based on the accelerator (as Google and AWS have done with their TPUs and Trainium accelerators) remains an open question.

What it'll look like

While we don't know exactly how OpenAI intends to use its first-gen silicon, Broadcom's involvement offers some clues as to what it may end up looking like.

Broadcom produces a number of foundational technologies necessary for building AI compute systems at scale. These technologies range from the serializer-deserializers (SerDes) used to move data from one chip to another to the network switches and co-packaged optical interconnects required to scale from one chip to thousands to the 3D packaging tech required to build multi-die accelerators. If you're curious, we've explored each in depth here.

OpenAI is likely to use some combination of all of these with Broadcom's 3.5D eXtreme Dimension System in Package tech (3.5D XDSiP), a likely candidate for the accelerators themselves.

On the left you see a typical accelerator built using 2.5D packaging and on the right Broadcom's XDSiP 3D-packaging tech - Click to enlarge

The architecture is reminiscent in many ways of the AMD MI300-series of accelerators more than anything we've seen from Nvidia just yet and involves stacking advanced compute tiles atop a base die containing the chip's lower-level logic and memory controllers. Meanwhile, package to package communication happens via discrete I/O dies. This modular approach means customers can bring as much or as little of their own intellectual property to the design as they like and let Broadcom fill in the gaps.

Boadcom's 3.5D XDSiP demo chip seen here looks surprisingly similar to AMD's MI300X, but is open to anyone to license. - Click to enlarge

The largest of Broadcom's 3.5D XDSiP designs will support a pair of 3D stacks, two I/O, and up to 12 HBM stacks on a single 6,000 mm2 package. The first of these is expected to begin shipping next year, right in line with when OpenAI is said to bring its first chips to market.

Alongside Broadcom's XDSiP tech, we wouldn't be surprised to see OpenAI leverage Broadcom's Tomahawk 6 family of switches and co-packaged optical chiplets for scale up and scale out networking, a topic we've explored in depth here. However, Broadcom's focus on Ethernet as the protocol of choice for both these network paradigms means that they wouldn't have to use Broadcom for everything.

The missing MAC

While Broadcom's 3.5D XDSiP seems a likely candidate for OpenAI's first in-house chips, it's not a complete solution on its own. The AI startup will still need to provide or, at the very least, license a compute architecture equipped with some seriously high-performance matrix multiply-accumulate (MAC) units, sometimes called MMEs or Tensor cores.

The compute units will need some other control logic and ideally some vector units as well, but for AI, the main thing is a beefy enough matrix unit with access to an ample supply of high-bandwidth memory.

Because Broadcom would be responsible for providing just about everything else, OpenAI's chip team could focus completely on optimizing its compute architecture for its internal workloads, making the entire process far less daunting than it otherwise might be. 

This is why cloud providers and hyperscalers have gravitated to licensing large chunks of their accelerator designs from the merchant silicon slinger. There's no sense wasting resources reinventing the wheel when you could reinvest those resources in your core competencies.

• Broadcom admits it's sold a lot of shelfware to VMware customers
• No chips for you! Senator wants Americans to get first dibs on GPUs, restrict sales to others
• Nvidia details its itty bitty GB10 superchip for local AI development
• More than 100 companies are chasing an AI chip gold rush. Few will surive

What if it's not OpenAI?

With Altman laying out plans to plow hundreds of billions of dollars of what's mostly other people's money into AI infrastructure under his Stargate initiative, the idea that Broadcom's new $10 billion customer would be OpenAI wouldn't be surprising. 

However, the startup isn't the only company rumored to be working with Broadcom on custom AI accelerators. You may recall that late last year The Information reported that Apple would be Broadcom's next big XPU customer with a chip due out in 2026 codenamed "Baltra."

Since then, Apple has committed to investing $500 billion and hiring 20,000 to bolster its domestic manufacturing capacity. Among these investments is a Texas-based manufacturing plant that'll produce AI-accelerated servers based on its own in-house silicon. ®