This product consists of a pretraining codebase for large-scale deep recurrent language models, developed in Python. It is optimized for AMD GPU architecture, enabling efficient operation on 4096 AMD GPUs. The core strength of this technology lies in its deep recurrent architecture, which significantly enhances the model's inference capabilities and efficiency. It is primarily aimed at researching and developing high-performance natural language processing models, especially in scenarios requiring large-scale computational resources. The codebase is open-source and licensed under the Apache-2.0 License, making it suitable for academic research and industrial applications.