OpenAI co-founder Ilya Sutskever recently stated that artificial intelligence researchers must find new ways to scale machine intelligence to overcome current limitations. Sutskever delivered a talk at the 2024 Neural Information Processing Systems (NeurIPS) conference in Vancouver, Canada, where he suggested that the era of AI pre-training is nearing its end and predicted the emergence of superintelligent AI. Sutskever believes that through better hardware, software, and machine...

A research team from The Chinese University of Hong Kong and the Chinese Academy of Sciences has proposed a full modal pre-training paradigm called MiCo (Multimodal Context). This method has achieved significant results in the field of multimodal learning, setting new state-of-the-art (SOTA) records in 37 categories.

Large language models have garnered significant attention in the AI community, showcasing remarkable capabilities. The Apple research team reveals WRAP technology, which utilizes synthetic data for pre-training large models. WRAP enhances model performance, rewrites network documentation, and improves pre-training effectiveness. By using high-quality synthetic data to accelerate training, overall performance is increased, paving new paths.