Computing is at an inflection point. Moore’s Law, which predicts that the variety of transistors on an electronic chip will double every year, is slowing down resulting from the physical limits of fitting more transistors on reasonably priced microchips. These increases in computer power are slowing down because the demand grows for high-performance computers that may support increasingly complex artificial intelligence models. This inconvenience has led engineers to explore latest methods for expanding the computational capabilities of their machines, but an answer stays unclear.
Photonic computing is one potential treatment for the growing computational demands of machine-learning models. As a substitute of using transistors and wires, these systems utilize photons (microscopic light particles) to perform computation operations within the analog domain. Lasers produce these small bundles of energy, which move on the speed of sunshine like a spaceship flying at warp speed in a science fiction movie. When photonic computing cores are added to programmable accelerators like a network interface card (NIC, and its augmented counterpart, SmartNICs), the resulting hardware might be plugged in to turbocharge an ordinary computer.
MIT researchers have now harnessed the potential of photonics to speed up modern computing by demonstrating its capabilities in machine learning. Dubbed “Lightning,” their photonic-electronic reconfigurable SmartNIC helps deep neural networks — machine-learning models that imitate how brains process information — to finish inference tasks like image recognition and language generation in chatbots equivalent to ChatGPT. The prototype’s novel design enables impressive speeds, creating the primary photonic computing system to serve real-time machine-learning inference requests.
Despite its potential, a significant challenge in implementing photonic computing devices is that they’re passive, meaning they lack the memory or instructions to manage dataflows, unlike their electronic counterparts. Previous photonic computing systems faced this bottleneck, but Lightning removes this obstacle to make sure data movement between electronic and photonic components runs easily.
“Photonic computing has shown significant benefits in accelerating bulky linear computation tasks like matrix multiplication, while it needs electronics to handle the remaining: memory access, nonlinear computations, and conditional logics. This creates a major amount of information to be exchanged between photonics and electronics to finish real-world computing tasks, like a machine learning inference request,” says Zhizhen Zhong, a postdoc within the group of MIT Associate Professor Manya Ghobadi on the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “Controlling this dataflow between photonics and electronics was the Achilles’ heel of past state-of-the-art photonic computing works. Even when you’ve got a super-fast photonic computer, you wish enough data to power it without stalls. Otherwise, you’ve got a supercomputer just running idle without making any reasonable computation.”
Ghobadi, an associate professor at MIT’s Department of Electrical Engineering and Computer Science (EECS) and a CSAIL member, and her group colleagues are the primary to discover and solve this issue. To perform this feat, they combined the speed of photonics and the dataflow control capabilities of electronic computers.
Before Lightning, photonic and electronic computing schemes operated independently, speaking different languages. The team’s hybrid system tracks the required computation operations on the datapath using a reconfigurable count-action abstraction, which connects photonics to the electronic components of a pc. This programming abstraction functions as a unified language between the 2, controlling access to the dataflows passing through. Information carried by electrons is translated into light in the shape of photons, which work at light speed to help with completing an inference task. Then, the photons are converted back to electrons to relay the knowledge to the pc.
By seamlessly connecting photonics to electronics, the novel count-action abstraction makes Lightning’s rapid real-time computing frequency possible. Previous attempts used a stop-and-go approach, meaning data could be impeded by a much slower control software that made all the choices about its movements. “Constructing a photonic computing system with out a count-action programming abstraction is like attempting to steer a Lamborghini without knowing easy methods to drive,” says Ghobadi, who’s a senior creator of the paper. “What would you do? You most likely have a driving manual in a single hand, then press the clutch, then check the manual, then let go of the brake, then check the manual, and so forth. This can be a stop-and-go operation because, for each decision, you’ve got to seek the advice of some higher-level entity to inform you what to do. But that is not how we drive; we learn easy methods to drive after which use muscle memory without checking the manual or driving rules behind the wheel. Our count-action programming abstraction acts because the muscle memory in Lightning. It seamlessly drives the electrons and photons within the system at runtime.”
An environmentally-friendly solution
Machine-learning services completing inference-based tasks, like ChatGPT and BERT, currently require heavy computing resources. Not only are they expensive — some estimates show that ChatGPT requires $3 million per thirty days to run — but they’re also environmentally detrimental, potentially emitting greater than double the common person’s carbon dioxide. Lightning uses photons that move faster than electrons do in wires, while generating less heat, enabling it to compute at a faster frequency while being more energy-efficient.
To measure this, the Ghobadi group compared their device to plain graphics processing units, data processing units, SmartNICs, and other accelerators by synthesizing a Lightning chip. The team observed that Lightning was more energy-efficient when completing inference requests. “Our synthesis and simulation studies show that Lightning reduces machine learning inference power consumption by orders of magnitude in comparison with state-of-the-art accelerators,” says Mingran Yang, a graduate student in Ghobadi’s lab and a co-author of the paper. By being a cheaper, speedier option, Lightning presents a possible upgrade for data centers to scale back their machine learning model’s carbon footprint while accelerating the inference response time for users.
Additional authors on the paper are MIT CSAIL postdoc Homa Esfahanizadeh and undergraduate student Liam Kronman, in addition to MIT EECS Associate Professor Dirk Englund and three recent graduates throughout the department: Jay Lang ’22, MEng ’23; Christian Williams ’22, MEng ’23; and Alexander Sludds ’18, MEng ’19, PhD ’23. Their research was supported, partly, by the DARPA FastNICs program, the ARPA-E ENLITENED program, the DAF-MIT AI Accelerator, the USA Army Research Office through the Institute for Soldier Nanotechnologies, National Science Foundation (NSF) grants, the NSF Center for Quantum Networks, and a Sloan Fellowship.
The group will present their findings on the Association for Computing Machinery’s Special Interest Group on Data Communication (SIGCOMM) this month.