AI Clusters: Optical Interconnects Win by 2028

The insatiable demand for more compute in AI has pushed traditional electrical interconnects to their physical breaking point. As AI models swell into trillion-parameter behemoths, the infrastructure they run on needs a complete overhaul. Next-generation optical interconnects are the essential backbone for any serious attempt at AI scale-up, promising bandwidth and energy efficiency that copper just can’t deliver. These advanced optical solutions are set to redefine the very architecture of future AI superclusters.

The Inevitable Shift from Electrical to Optical Signaling

For years, copper traces did the job, moving data across circuit boards and between boxes. But modern AI workloads, with their massive parallel processing and constant shuffling of data between GPUs and memory, are exposing copper’s fundamental limits. Signal integrity tanks over any real distance, forcing us to add re-timers and repeaters that burn a ton of power. We’re already seeing 200 gigabits per second (Gbps) per lane become the norm, and trying to push that speed electrically creates impossible problems with insertion loss, crosstalk, and just way too much heat.

Optical interconnects, on the other hand, send data as light pulses through fiber or waveguides. Light signals suffer far less attenuation, so they can go longer distances without falling apart, and they’re completely immune to the electromagnetic interference that plagues dense electrical racks. More importantly, optical fibers carry a staggering amount of data. A single strand of fiber can handle multiple terabits per second (Tbps), while even the best electrical traces top out at a few hundred gigabits per second on a short board. It’s a physics problem. The constraints of electrical traces at the high frequencies AI demands create a bottleneck that’s nearly impossible to engineer around.

This transition is about both speed and power efficiency. Pushing electrical signals fast generates a lot of heat, which then demands complex and power-hungry cooling systems. Optical transmission is inherently more energy-efficient, especially over distance. A 2024 report from LightCounting Market Research (LightCounting) projects that optical transceivers will get down to less than 5 picajoules per bit (pJ/bit) by 2027. That’s a huge improvement over electrical links that can burn tens of pJ/bit for the same job. Power efficiency is everything in AI data centers, where your power and cooling budget is often what stops you from expanding. We’re already hitting a point where a 100-kilowatt rack is common, and every watt you save on interconnects is a watt you can give to more compute.

The Rise of Co-Packaged Optics (CPO) and Silicon Photonics

One of the biggest moves toward integrating optics is co-packaged optics (CPO). For a long time, optical transceivers have been pluggable modules you stick in the front of a switch or a NIC. That approach works, but it forces you to run long electrical traces from the high-speed SerDes chips to the optical module, which creates signal integrity headaches and wastes power. CPO solves this by moving the optical engines right next to the processor, often on the same substrate or package as the ASIC or GPU.

By shrinking those electrical traces down to just a few millimeters, CPO slashes power consumption and cleans up the signal. A standard 800 gigabits per second (Gbps) pluggable module might burn 15 to 20 watts, and a lot of that is just for driving the signal across the board. A co-packaged optical engine doing the same job could use less than half that, maybe under 8 watts, because it gets rid of those power-hungry electrical paths. This is a fundamental architectural shift that will enable much denser and more efficient AI systems.

Silicon photonics is the technology at the heart of CPO and most other next-gen optics. It uses standard CMOS manufacturing to build optical components, waveguides, modulators, photodetectors, directly onto a silicon wafer. This is what allows for the mass production of highly integrated, low-cost optical circuits. Companies like Lumentum (Lumentum) are already leading the way here, developing the high-performance optical engines that get integrated into CPO modules. Being able to cram complex optical functions onto a silicon chip is what makes CPO affordable enough for massive AI deployments.

And the integration is going deeper than just transceivers. We’re seeing silicon photonics used to create on-chip optical switches and even optical memory interfaces. This level of integration is how we’ll finally start to break down the “memory wall” and “I/O wall” that are currently choking AI performance. Imagine an entire AI accelerator where all the components talk to each other and to memory using light, all inside one compact, efficient package. Optical paths can also offer lower latency than their electrical counterparts, which is absolutely necessary for the synchronous operations in distributed AI training.

Addressing the Challenges of AI Scale-Up with Optical Solutions

Scaling an AI cluster past a few hundred GPUs requires a completely different approach to networking. Your standard Ethernet fabric starts to create bottlenecks when you’re trying to move data between thousands of accelerators at once. Optical interconnects are stepping in to fix these problems:

• Ultra-High-Bandwidth Inter-GPU Communication: In a big AI cluster, GPUs are constantly swapping huge amounts of data. PCIe Gen6 and the upcoming Gen7 offer more bandwidth, but they’re still limited by electrical reach. Optical interconnects, sometimes using proprietary protocols like NVLink-C2C over optical links, let you extend this high-bandwidth fabric across an entire rack or even between racks, allowing you to build bigger, more tightly-coupled training clusters without performance falling off a cliff.
• Rack-Scale Optical Backplanes: Instead of a mess of copper cables, future AI racks might use optical backplanes. These could use free-space optics or integrated waveguides to give every server blade terabit-level connectivity. This gets rid of dozens of individual transceivers and cables, simplifying the whole rack design and making it easier to cool.
• Disaggregated Memory and Compute: The idea of pooling memory and letting compute units access it over a fast fabric is getting a lot of attention. Optical interconnects are perfect for this, providing the bandwidth and low latency needed to make remote memory feel local. This allows for much more flexible and efficient use of expensive HBM (High Bandwidth Memory).

Pushing for higher data rates per lane is also part of the story. While 100 Gbps per lane is becoming common, the industry is already gunning for 200 Gbps and even 400 Gbps per lane with advanced modulation like PAM4 (Pulse-Amplitude Modulation 4) and the emerging PAM8. These techniques pack more bits into each signal, effectively doubling or quadrupling the data rate. The trade-offs are real, it makes transceiver design much harder and requires a better signal-to-noise ratio, but the relentless demand for more bandwidth makes these schemes a necessity.

Evolving Optical Technologies: From VCSELs to External Lasers

While silicon photonics is the foundation, other optical technologies are also being pushed to meet AI’s needs. For the really short-reach, high-density connections inside a server, Vertical Cavity Surface Emitting Lasers (VCSELs) are still a great choice. VCSELs are low-power, have a small footprint, and they’re cheap to make. They’re especially good for parallel optical links, where you can array a bunch of them to shoot data over ribbon fibers and get 1.6 Tbps or more out of a single module. With better modulation bandwidths and efficiency, VCSELs are finding more and more use cases.

For longer reaches and higher-power CPO designs, the trend is shifting toward external lasers. Instead of trying to build the laser directly onto the silicon photonics chip (which is tough because of heat and manufacturing issues), you put the laser source off-chip and pipe the light in. This lets you use a higher-performance laser, manage heat better, and even replace the laser if it fails. It’s a much more practical design. Companies like Lumentum are building external laser modules specifically designed to power multiple optical engines in a single CPO package, providing a powerful and scalable light source.

Optical switching is another area seeing new work. While electronic packet switches run today’s data centers, all-optical switching could save a lot of power and cut latency for certain AI workloads. Research on things like MEMS-based optical switches and wavelength-selective switches is ongoing. While still pretty new for large-scale use, the potential benefits for specific communication patterns, like the all-to-all exchanges common in distributed training, are huge.

The Road Ahead: Integration, Standards, and Thermal Management

Getting next-gen optical interconnects widely adopted in AI systems won’t be easy. The complexity of integration is a primary hurdle. When you co-package optical engines with high-power ASICs, you create a thermal nightmare. Light sources, modulators, and amplifiers all produce heat, and getting that heat out of a tiny package that also contains a 300-watt GPU requires some serious engineering. I’ve seen firsthand how fast you blow through your thermal budget with these designs. Liquid cooling, which is already showing up in AI racks, is probably going to be a requirement for any system using advanced CPO.

Standardization is also a big deal. Groups like the Optical Internetworking Forum (OIF) (OIF) are hammering out specs for CPO and high-speed interfaces, but AI development moves so fast that it often outpaces the standards bodies. This leads to proprietary solutions popping up to solve immediate problems, which risks fragmenting the market. For broad adoption and lower costs, we’ll need a unified approach to optical interfaces and management.

And of course, the economics have to work. Optical interconnects offer better performance and power efficiency, but their upfront cost can be higher than copper. As manufacturing scales up and silicon photonics gets more mature, those costs should come down. In the end, the total cost of ownership (TCO), factoring in power savings and increased compute density, will be what drives adoption. The industry needs to deliver both performance and economically sane solutions for these massive hyperscale environments, because at the scale of tens of thousands of GPUs, even a small cost difference per port becomes a massive number.

We are at a tipping point. Electrical signaling is becoming a dead end for the kind of scale advanced AI demands. The industry must embrace optical interconnects as a basic requirement to keep pushing AI forward. The next few years will bring a dramatic change in how AI systems are built, with light becoming a central part of their architecture.

The future of AI is tied directly to the evolution of its communication fabric. Next-generation optical interconnects represent a sea change in AI system design, enabling scale, efficiency, and performance we couldn’t get otherwise. For anyone serious about building the AI supercomputers of tomorrow, adopting these optical solutions isn’t optional.