Nvidia's AI Chip Dominance Challenged by OpenAI's Search for Faster Inference Processing

Picture this. You’re Nvidia , the undisputed king of AI chips. Your graphics processing units power everything from ChatGPT to cutting-edge AI research. Companies throw money at you. Your stock price is through the roof. Life is good. Then your biggest potential customer, OpenAI, starts shopping around for alternatives. Awkward. That’s exactly what’s happening right now. As per an article by Reuters, OpenAI has apparently been unhappy with Nvidia’s chips since last year and has been quietly looking for alternatives. And this isn’t just rumor mill stuff. Eight sources confirmed this to Reuters. But wait, weren’t Nvidia and OpenAI supposed to be best friends? Wasn’t Nvidia planning to invest up to $100 billion in OpenAI? What happened? The issue boils down to one word: inference. Let’s break this down. When you build an AI model like ChatGPT, there are two main phases. First, there’s training, where you feed the model massive amounts of data so it learns patterns and relationships. Think of it like teaching a student everything they need to know. Nvidia’s chips are brilliant at this. No complaints there. Then there’s inference. This is when the trained model actually does its job, answering your questions and completing tasks. When you type a prompt into ChatGPT and it responds, that’s inference. And apparently, Nvidia’s chips aren’t fast enough for OpenAI’s liking in certain situations. Specifically, OpenAI has problems with how quickly Nvidia’s hardware can spit out answers for tasks like software development and AI-to-AI communication. The company needs chips that can provide about 10% of its future inference computing power, and Nvidia’s current offerings just aren’t cutting it. Why the speed difference? It comes down to memory architecture. Nvidia’s chips rely on external memory, which is like having to walk to another room every time you need to grab something. It adds processing time. OpenAI wants chips with large amounts of memory embedded right on the chip itself, called SRAM. It’s faster, like having everything you need on your desk instead of in a filing cabinet across the office. So OpenAI went shopping. As per a report by Reuters, they talked to chipmakers like Cerebras and Groq, both of which specialize in chips with that precious embedded memory. They also struck deals with AMD and Broadcom. But here’s where it gets interesting. Nvidia didn’t just sit back and watch. When Groq seemed like it might partner with OpenAI, Nvidia swooped in with a $20 billion licensing deal that effectively shut down those talks. It was a power move. Nvidia essentially locked up Groq’s technology and even hired away their chip designers. OpenAI did manage to strike a deal with Cerebras in January, which Sam Altman publicly acknowledged would help meet the speed demands for coding tasks. But the damage to the Nvidia-OpenAI relationship was already done. Remember that $100 billion investment Nvidia was supposedly making in OpenAI? The one announced in September with a non-binding letter of intent? Well, it’s now February, and that deal has gone nowhere. Nvidia CEO Jensen Huang recently told the press that the $100 billion figure was never actually a commitment. He brushed off reports of tension as nonsense but also made it clear Nvidia isn’t going to be overly dependent on OpenAI. And you can understand why. Look what happened to Microsoft. The company got hit with one of its biggest one-day stock drops partly because investors worried it was too tied to OpenAI. Nvidia doesn’t want that kind of vulnerability. Still, Huang said Nvidia will participate in OpenAI’s current funding round, calling it probably the largest investment Nvidia has ever made. For context, they’ve invested $5 billion in Intel before. So it’s not like they’re walking away completely. They’re just being careful about the optics. This whole situation reveals something important about the AI industry. Training AI models and running them are two very different challenges requiring different technologies. Nvidia might dominate training, but inference is becoming a whole new battleground. And OpenAI isn’t the only one exploring alternatives. Competitors like Anthropic’s Claude and Google’s Gemini already use Google’s custom tensor processing units, which are better optimized for inference tasks. So while Nvidia and OpenAI are still partners, and while both sides are saying nice things publicly, the reality is more complicated. OpenAI needs faster chips for specific tasks. Nvidia wants to protect its dominance and avoid being pigeonholed as dependent on one customer. And both are maneuvering carefully in a rapidly evolving industry where today’s cutting-edge technology could be tomorrow’s bottleneck. The AI chip wars are heating up. And the biggest players are hedging their bets. Sonia Boolchandani is a seasoned financial writer She has written for prominent firms like Vested Finance, and Finology, where she has crafted content that simplifies complex financial concepts for diverse audiences. Disclosure: The writer and her his dependents do not hold the stocks discussed in this article. The website managers, its employee(s), and contributors/writers/authors of articles have or may have an outstanding buy or sell position or holding in the securities, options on securities or other related investments of issuers and/or companies discussed therein. The content of the articles and the interpretation of data are solely the personal views of the contributors/ writers/authors. Investors must make their own investment decisions based on their specific objectives, resources and only after consulting such independent advisors as may be necessary.