ChatGPT's energy hunger could accelerate the GPU revolution


The cost of making further advances in artificial intelligence is becoming as staggering as the hallucinations by ChatGPT. Demand for graphics chips, known as GPUs, essential for large-scale AI training has caused prices of the critical components to skyrocket. OpenAI has said that training the algorithms that power ChatGPT cost the company more than $100 million. The race to compete in AI also means that data centers are now consuming worrying amounts of energy.

The AI ​​gold rush has some startups making bold plans to create new computational shovels to sell. Nvidia's GPUs are by far the most popular hardware for AI development, but these newcomers argue it's time to radically rethink how computer chips are designed.

Normal Computing, a startup founded by veterans of Google Brain and Alphabet's Moonshot LabX, has developed a simple prototype that is a first step toward rebooting computing from first principles.

A traditional silicon chip runs calculations by handling binary bits – that is, 0s and 1s – representing information. General computing's stochastic processing unit, or SPU, uses the thermodynamic properties of electrical oscillators to perform calculations using random fluctuations occurring inside circuits. It can generate random samples useful for calculations or for solving linear algebra calculations, which are ubiquitous in science, engineering, and machine learning.

Faris Sabahi, CEO of Normal Computing, explains that the hardware is highly efficient and suitable for handling statistical calculations. This may someday be useful for building AI algorithms that can handle uncertainty, perhaps addressing the tendency of large language models to “hallucinate” output when uncertain.

Sabahi says the recent success of generative AI is impressive, but far from the final form of the technology. “It's kind of clear that there's something better out there in terms of software architecture and hardware as well,” Sabahi says. He and his co-founders previously worked on quantum computing and AI at Alphabet. The lack of progress in using quantum computers for machine learning led them to think about other ways of harnessing physics to power the calculations needed for AI.

Another team of pre-quantum researchers from Alphabet left to found Extropic, a company still under wraps that has even more ambitious plans to use thermodynamic computing for AI. “We're trying to tightly integrate all of neural computing into an analog thermodynamic chip,” says Guillaume Verdon, founder and CEO of Xtropic. “We are taking our learnings from quantum computing software and hardware and bringing it into the full-stack thermodynamic paradigm.” (Verdon was recently revealed as the person behind the popular meme account on X beef jezosSo-called dominant accelerationism is associated with the movement that promotes the idea of ​​progress toward a “technological capital singularity”.)

The idea that computing needs a comprehensive rethink is gaining momentum as the industry struggles to keep up with Moore's Law, the long-standing prediction that the density of components on chips will shrink. “Even if Moore's Law were not slowing down, you still have a big problem, because the model sizes that OpenAI and others are releasing are growing faster than the chip capacity,” said the Cornell University professor. Says Peter McMahon, who works on it. New methods of computing. In other words, we may need to use new methods of computing to keep the AI ​​hype train on track.