Recently, it has been reported that OpenAI is collaborating with Broadcom to develop a custom inference chip. According to Reuters, the discussions between these two companies are highly confidential, and Taiwan Semiconductor Manufacturing Company (TSMC) might act as the foundry for this project. This news has sparked widespread speculation about OpenAI's future direction.
Image source: This image was generated by AI, provided by the image licensing service Midjourney.
So, why does OpenAI need its own inference chip? Firstly, OpenAI's cloud computing costs are enormous, and although partners like Microsoft support some of these costs, controlling the hardware themselves can significantly reduce operating expenses. Many companies have found that building their own data centers is more economical than renting cloud services.
Additionally, developing a specialized chip tailored to their services could be a strategic goal for OpenAI. It is well known that AI applications consume a lot of energy, so optimizing the synergy between hardware and software could make OpenAI's services more efficient.
OpenAI is also showcasing the idea of building large data centers to investors, which are dedicated to running AI services. These data centers, if equipped with custom chips, could also have lower construction or operating costs. Moreover, the consideration of diversifying the supply chain is equally important. Given the limited global semiconductor production capacity, the risk of relying on external suppliers exists, and developing their own chips can reduce dependence on third-party products.
Although it is hard to imagine OpenAI entering the hardware sales industry, which requires substantial investment and an increase in staff, in scenarios where inference tasks often need to be as close to the user as possible, OpenAI might deploy related equipment at the network edge, similar to many content delivery networks and Netflix. This architecture is definitely a good idea.
When it comes to inference chips, they are not unfamiliar in the market. Chips like AWS's Inferentia, Google's Tensor Processing Units (TPUs), and Microsoft's Maia silicon can handle both inference and training workloads.
Interestingly, the news of OpenAI's collaboration with Broadcom also led to a slight increase in Broadcom's stock price. Broadcom's latest quarterly earnings report indicates that it expects to sell $12 billion worth of AI silicon this fiscal year, $1 billion more than previously anticipated, but investors seem somewhat disappointed. Therefore, partnering with the hottest name in the AI software field is sure to excite Wall Street.
Key Points:
🌟 OpenAI and Broadcom are in talks to develop a custom inference chip to reduce cloud computing costs.
💡 A proprietary chip can optimize the synergy between hardware and software, enhancing the efficiency of AI services.
📈 Broadcom expects to sell $12 billion worth of AI silicon this fiscal year, and the collaboration news boosted its stock price.