Microsoft announced that its Phi series of generative AI models has a new member - Phi-4. Compared to previous versions, Phi-4 shows improvements in several areas, particularly in solving mathematical problems, thanks to the enhanced quality of training data. As of Thursday evening, access to Phi-4 is relatively limited, available only on Microsoft's newly launched Azure AI Foundry development platform and restricted to research purposes under the Microsoft Research Licensing Agreement. Image source note: Image generated by AI, image licensed.

Harvard University announced on Thursday the release of a high-quality dataset containing nearly one million public domain books, which anyone can use to train large language models and other AI tools. This dataset was created by Harvard's newly established Institutional Data Initiative and is funded by Microsoft and OpenAI. The included books are all works scanned by the Google Books Project that are no longer under copyright protection.

The New York Times and the Daily News encountered an unexpected twist in their copyright lawsuit: an OpenAI engineer inadvertently deleted virtual machine search data that could have been key evidence, adding a dramatic turn to this high-profile legal dispute. According to a letter submitted to the U.S. District Court for the Southern District of New York on Wednesday night, lawyers and technical experts for the two media companies had previously invested over 150 hours searching OpenAI's AI training dataset. However, on November 14, an OpenAI engineer accidentally deleted data stored on the virtual machine.

Recently, OpenAI reached an agreement in a high-profile copyright lawsuit to disclose the data used to train its generative AI models to the plaintiff's lawyers. The copyright lawsuit's plaintiffs include several well-known authors such as Paul Tremblay, Sarah L. J. Elfreth, Michael Chabon, David Henry Huang, and Ta-Nehisi Coates. They filed a lawsuit against OpenAI and its affiliates last year, accusing the AI of using their works without authorization.

In the context of rapid developments in artificial intelligence, Baidu Tieba's 'Stupid Bar' has unexpectedly become an important data source for AI training, challenging traditional perceptions of high-quality data. This community, known for its absurd and humorous content, has outperformed well-known platforms in tests such as Q&A and brainstorming, making it one of the most popular AI training databases. Research shows that the conversational and multi-turn Q&A data from 'Stupid Bar' helps AI come closer to human communication styles. While AI has limitations in understanding human humor, the jokes from 'Stupid Bar' feature tight logic, rich rhetoric, and strong...

LAION launched Re-LAION-5B, the world's first AI training dataset that fully removes links to CSAM, aimed at addressing the issue of Child Sexual Abuse Material (CSAM). This dataset has been significantly improved over LAION-5B and is mainly divided into two versions: Re-LAION-5B Research and Research-Safe. A total of 2,236 CSAM links have been removed, including 1,008 from child protection organizations' lists. The dataset contains 5.5 billion pairs of text and images, designed to help

The Economist article highlights the 'data wall' challenge faced by AI companies due to the depletion of high-quality internet data, predicting that by 2028, all quality text data will be exhausted, hindering machine learning progress. Professor Stuart Russell from UC Berkeley warned that AI like ChatGPT could deplete textual content in the universe. However, Stanford's Professor Fei-Fei Li asserts that there is still a wealth of diverse data ava....

Recently, the 'Diting' seismic wave large model, jointly developed by the National Supercomputing Center in Chengdu, the Institute of Geophysics of the China Earthquake Administration, and Tsinghua University, was officially released in Chengdu, Sichuan. This model is the first seismic wave large model in the country to reach 100 million parameters, marking a significant breakthrough in the integration of seismology research and artificial intelligence technology in China.

AI models are gradually losing access to online training data, with the data blocking ratio increasing from 1% to 5-7%. The study analyzed the robots.txt files and terms of use for 14,000 domains, finding that news websites, forums, and social media platforms are the primary sources of AI data access restrictions, with the blocking ratio for news websites skyrocketing from 3% to 45%. The proportion of high-quality news content in AI training data is decreasing, potentially being replaced by low-quality corporate e-commerce content. This trend poses challenges for AI developers, as high-quality...

IBM opened the Granite13B LLM model for enterprise applications in May. Now, Armand Ruiz, the vice president of IBM's AI platform products, has publicly disclosed the full content of the comprehensive 6.48TB dataset used to train Granite13B.The dataset, after strict preprocessing, was reduced to 2.07TB, a reduction of 68%. Ruiz emphasizes that this step is crucial for ensuring a high-quality, unbiased, ethical, and legally compliant dataset to meet the needs of enterprise applications. The datas

Spawning is committed to giving artists more control over the online use of their work. The company has launched its latest project called Source.Plus, aimed at screening 'non-infringing' media content for AI model training, enabling artists and creators to more precisely manage the usage rights of their works.

Automattic negotiates with Midjourney and OpenAI to share user posts as training data. Automattic plans to allow users to opt out of sharing data with third parties. Other companies are also collaborating with AI firms to provide training data, leading to protests from creators.

The American social media platform Reddit has reached an agreement with Google, valued at approximately $60 million annually. Springer Publishing Group collaborates with OpenAI, becoming the first publishing entity to deeply integrate journalism with artificial intelligence technology. OpenAI has signed a cooperation agreement with Axel Springer, indicating that future large model training may involve payments to data providers. Publishing industry companies possess a wealth of electronic graphic resources and are expected to become important datasets for large model training.

Music publishers have filed a preliminary injunction against Anthropic, which responds that using copyrighted content as LLM training data is reasonable. Anthropic's refusal to grant the injunction request means that obtaining permission to train the system would effectively end general AI tools.

ZhiYuan Research Institute has released a code generation training dataset called TACO, aimed at providing more challenging training data and evaluation benchmarks for code generation models. TACO has advantages in terms of data scale, quality, and evaluation schemes, including a larger training and testing set, diverse problem-solving answers, and fine-grained labels. Experimental results show that current popular code generation models show significant differences compared to GPT-4 in TACO evaluations, indicating that there is still room for improvement in this field. TACO is not just a challenging...

The research found that large language models like ChatGPT retain a considerable amount of original training data during their training process. Attackers can extract large volumes of training data from the model using specific attack methods, threatening the privacy of data owners. Researchers utilized a concept of 'extractable memory' attack techniques and discovered that existing extraction attacks can recover a significant amount of training data from memory even without using the actual training data as prompts. ⚠️ Researchers recommend that data security protective measures should be implemented while developing and using large language models.

Teams from Google DeepMind, the University of Washington, and others have successfully unveiled the sources of ChatGPT's training data through a 'poetry attack'. This attack method involves repeating specific vocabulary, which led ChatGPT to disclose personal information, including phone numbers and email addresses. The vulnerability was reported to OpenAI on August 30, and no further comments have been made. The research calls for environmentally-friendly AI usage, pointing out the significant energy burden that text and image generation imposes on the environment.

AI companies may face the risk of training data exhaustion in the near future, with high-quality text training data potentially running out by 2026. The use of synthetic data may lead to inherent distortion effects in models, as research indicates the risks. A solution may lie in establishing data partnerships to alleviate supply issues through data exchange.