Samsung Electronics' fifth-generation high bandwidth memory (HBM) chip, HBM3E, has passed Nvidia's testing and is qualified for use in artificial intelligence processors. Although the two parties have yet to sign a formal agreement, a supply agreement is expected to be reached soon, with supply anticipated to begin in the fourth quarter of 2024. This achievement represents a key breakthrough for Samsung in competing with local rival SK Hynix. However, Samsung's 12-layer HBM3E chip failed to pass testing, prompting the company to make adjustments to the design in order to address issues related to heat and power consumption.

Meta has trained its latest AI model, Llama 3.1, using 16,384 GPUs, showcasing the astonishing speed of AI technological advancement. However, during this process, an average of one failure occurs every 3 hours, totaling 419 failures, with approximately half related to the H100 GPUs and their HBM3 memory. This data reveals the reliability challenges faced by supercomputing systems in their pursuit of performance breakthroughs. The complexity of the Llama 3.1 training cluster is comparable to a small city's neural network, with frequent failures. The Meta team has implemented strategies such as reducing...

Micron Technology has achieved revenue growth for six consecutive quarters, primarily due to the surge in demand for AI chips. The company's newly launched HBM3E memory chip modules have been well received and are expected to generate hundreds of millions in revenue for the company in 2024. Micron is optimistic about the applications of AI in personal computers and mobile devices, planning to increase capital expenditures in fiscal year 2024 to meet production demands for the new memory modules. The CEO stated that the company’s HBM supply is sold out and predicts a significant increase in sales in the personal computer sector in 2024, driving demand for memory and storage.

The enthusiasm for HBM continues to surge, with Samsung and SK Hynix increasing HBM production, leading to a significant rise in market attention. The release of NVIDIA's H200 has stimulated HBM demand, and both Samsung and SK Hynix have announced substantial expansion plans. HBM technology addresses the memory wall issue, meeting the demands for large-scale data storage and high computational needs. HBM has clear advantages as an AI accelerator card; however, more effort is needed for its adoption in consumer-grade products.

["Samsung confirms that the codename for the fifth generation HBM3E memory is Shinebolt, with a 50% bandwidth increase to 1.228TB.","Shinebolt will stack 24Gb chips in 8 layers, and a 12-layer 36GB product is under development.","HBM3E will provide strong support for the next generation of AI GPUs.","Samsung accelerates HBM3E development, closely following industry leader SK Hynix.","Samsung announces plans to further expand HBM production capacity to offer customized services to customers."]

Samsung Electronics reported a 77.9% year-on-year decline in operating profit in the third quarter, with losses in its chip business. The company aims to launch the sixth generation of high-performance HBM4 memory chips in 2025 to compete for dominance in the AI chip market. The demand for HBM memory continues to grow to support AI systems, data centers, and machine learning platforms. Samsung has been competing with SK Hynix for dominance in the HBM memory market. In 2016, Samsung commercialized HBM memory, becoming the first to enter the AI memory chip market.