Scientists Innovate Technology to Successfully Train Trillion-Parameter Models at ChatGPT Level
站长之家
53
Data to be translated:
Scientists have successfully trained a ChatGPT-level model using only 8% of the computational power of the world's most powerful supercomputer. This breakthrough comes from Oak Ridge National Laboratory, where the research team employed innovative techniques to train a trillion-parameter language model on the Frontier supercomputer. By utilizing distributed training and parallel technologies, they achieved 100% weak scaling efficiency. However, training large-scale language models still presents challenges, particularly in addressing memory issues. This research provides valuable experience for future training of enormous language models, highlighting the critical role of distributed training and parallel computing.
© Copyright AIbase Base 2024, Click to View Source - https://www.aibase.com/news/4803