As AI technology rapidly advances, the challenges of data acquisition are becoming increasingly prominent. Research from institutions like MIT reveals that open-source datasets such as C4, RefineWeb, and Dolma are facing stricter licensing agreements from the websites they scrape, posing significant challenges for AI training and academic research. The study found that even major AI companies like OpenAI are subject to strict limitations due to r....

Recently, researchers at Microsoft have introduced a novel AI framework called Auto Evol-Instruct, which can automatically evolve instructional datasets without any human intervention.In the field of artificial intelligence, the development of large language models (LLMs) is crucial, especially in enhancing their ability to follow detailed instructions. Researchers have been exploring ways to improve the datasets used to train LLMs to enhance their performance and adaptability.Traditional evolut

Nvidia recently announced the launch of two groundbreaking AI technologies: HelpSteer2 and Llama3-70B-SteerLM-RM. These innovations promise to significantly enhance the capabilities of AI systems across a variety of applications, from autonomous driving to natural language processing.

OpenAI and Microsoft are being sued by news agencies for copyright infringement. Multimodal large models are becoming the mainstream trend in the field of large models. OpenAI is reportedly actively negotiating with publishers on AI policies while high-quality dataset copyright issues are receiving attention. CITIC Publishing collaborates with large model companies for language training, and Visual China holds a core advantage in the era of AIGC.

The American social media platform Reddit has reached an agreement with Google, valued at approximately $60 million annually. Springer Publishing Group collaborates with OpenAI, becoming the first publishing entity to deeply integrate journalism with artificial intelligence technology. OpenAI has signed a cooperation agreement with Axel Springer, indicating that future large model training may involve payments to data providers. Publishing industry companies possess a wealth of electronic graphic resources and are expected to become important datasets for large model training.

Shanghai AI Lab, together with Ruijin Hospital affiliated to Shanghai Jiao Tong University School of Medicine, has released the medical multimodal foundational model group 'PuYi 2.0'. PuYi 2.0 adds multi-domain models and incremental language parameters, covering various data modalities such as medical images, medical texts, and bioinformatics. New open-source datasets include the medical image segmentation dataset SA-Med2D-20M and the pathology dataset SNOW. PuYi 2.0 incorporates an evaluation module to provide reference implementations for medical model capabilities, delivering a one-stop open-source solution for large medical models.

OpenAI has launched the Data Partnerships initiative to collaborate with third parties in creating public and private datasets for AI model training. OpenAI seeks data across multiple domains, languages, and formats to enhance the understanding of AI models. The initiative aims to collect various forms of data, including images, audio, and video, to reflect human society and achieve the goal of safer, more beneficial AI for humanity. OpenAI's initiative has sparked discussions about the rights of data owners and transparency.

Researchers, in collaboration with MIT, Cohere for AI, and other institutions, have launched the Data Provenance Platform to tackle the issue of AI data transparency. The platform audits and tracks over 2,000 widely used fine-tuning datasets, emphasizing the importance of data transparency. Insufficient data provenance can lead to data leaks, exposure of personal information, bias, and legal risks. The Data Provenance Platform is expected to enhance AI data transparency and improve the quality and ethical compliance of datasets.