IBM announced a significant breakthrough in optical technology in Yorktown, New York, which will greatly enhance the efficiency of data centers when training and running generative artificial intelligence (AI) models.
IBM researchers have developed a new type of Co-Packaged Optics (CPO) technology that utilizes the speed of light to enable connections within data centers, replacing the currently used copper cables.
Although fiber optic technology is widely used in global commerce and communication, most data centers still rely on copper cables for short-distance communication. This leads to GPU accelerators often being idle during training, wasting significant time and energy. IBM's research team demonstrated how to introduce the speed and capacity of light into data centers, significantly improving communication bandwidth and reducing GPU idle time, thereby accelerating the processing speed of AI models.
According to a technical paper released by IBM, the CPO prototype module developed by the team can achieve high-speed optical connections, providing five times the power savings and extending the length of internal connection cables from one meter to hundreds of meters.
Additionally, using this new technology, the training speed of large language models (LLMs) can be five times faster than with traditional cables, meaning that the training time for a standard LLM can be reduced from three months to three weeks. This innovation not only significantly lowers the cost of scaling generative AI but also improves energy efficiency in data centers, saving energy equivalent to the annual consumption of 5,000 American households.
IBM's research indicates that CPO technology can increase bandwidth between chips by up to 80 times that of traditional connections, fully leveraging advancements in modern chip technology. IBM's 2-nanometer node chip technology can accommodate over 50 billion transistors, and the introduction of CPO technology will further enhance computing density and efficiency. The successful prototype has undergone testing in high humidity and extreme temperatures during manufacturing, ensuring the reliability and durability of optical interconnections.
This research paves the way for the future development of data centers, and IBM will continue to lead in semiconductor research and development, driving the advancement of more efficient optical communication technologies to meet the growing performance demands of AI.
Key Points:
🌟 IBM launches a new Co-Packaged Optics technology to replace traditional copper cables and enhance data center efficiency.
⚡ CPO technology can reduce the training time for large language models from three months to three weeks, achieving a fivefold speed increase.
🔋 This technology can save a significant amount of energy in data centers, equivalent to the annual electricity consumption of 5,000 households.
