Tencent Youtu Lab has collaborated with the research team at Shanghai Jiao Tong University to introduce a revolutionary knowledge enhancement method, paving new avenues for optimizing large models. This innovative technology discards the limitations of traditional model fine-tuning, directly extracting knowledge from open-source data, significantly simplifying the model optimization process, and achieving outstanding performance that surpasses existing state-of-the-art technologies (SOTA) across multiple tasks. In recent years, while large language models (LLMs) have made significant progress in various fields, they still face numerous challenges in practical applications. Traditional model fine-tuning methods...

The expert team formed by the MIFA Laboratory of the Shanghai Jiao Tong University Qingtuan Research Institute and the Endocrinology Department of Zhongshan Hospital affiliated with Fudan University has recently developed a dedicated large model for diabetes named Diabetica. This model integrates the language processing capabilities of large models with specialized knowledge in diabetes, aiming to provide intelligent support for doctors, patients, and medical education. The team employed a novel framework to establish a comprehensive data processing system, created high-quality datasets and evaluation benchmarks, and fine-tuned the large language model, significantly enhancing its performance on diabetes-related tasks.

At the 2024 WAIC Scientific Frontier Main Forum, the OpenDataLab team from the Shanghai Artificial Intelligence Laboratory (Shanghai AI Lab) launched a new intelligent data extraction tool called MinerU. This tool aims to simplify the AI data processing workflow and assist AI researchers in extracting high-quality data from vast amounts of documents.

Tencent Youtu Lab and Shanghai Jiao Tong University released a comprehensive review focusing on dataset evaluation and selection for instruction tuning to enhance large model performance. The study aims to help large models grasp the essence of natural language processing by analyzing data quality, diversity, and importance to guide the construction of efficient and critical datasets. It highlights that data quality directly impacts tuning effect....

The LSLM full-duplex language model developed in collaboration between Shanghai Jiao Tong University and ByteDance revolutionizes traditional AI assistant interaction by enabling real-time conversation where users can listen while speaking. The core advantage of LSLM lies in its ability to simultaneously process speech input and generate output, significantly enhancing the natural flow of dialogue. By integrating streaming self-supervised learning with a token-based decoder, this model supports real-time speech interaction and performs exceptionally well even in noisy environments. LSLM's mid-term fusion strategy strikes the optimal balance between speech generation and real-time interaction, demonstrating impressive noise resistance.

The AI painter struggles to generate specific scenes like 'Ice Cola in a teacup,' highlighting the issue of text-image misalignment. Zhao Juntu, a PhD student at Shanghai Jiao Tong University, and his team discovered that even the most advanced AI models have difficulty accurately understanding and realizing complex concepts described in text, such as the difference between a transparent glass cup and a traditional teacup. To address this issue, they proposed a Mixture of Concept Experts (MoCE) method, utilizing large language models to help AI grasp hidden concepts, thereby enabling more precise control over text.

The Shanghai Cyberspace Administration recently announced that 11 generative AI services have passed inspection and completed registration, bringing the total number of registered services to 20. This initiative aims to promote the healthy development and standard application of generative AI, implementing relevant interim management measures. The announcement emphasizes that services that directly call registered model capabilities through APIs or other means, especially those with public opinion attributes or social mobilization capabilities, must undergo registration. Existing generative AI applications or features must prominently display their registration number either in a notable position or on their product detail pages. This measure aims to ensure the safe use of the public.

Scientists at ShanghaiTech University have developed CLAY, an AI model that can generate detailed 3D objects from text or 2D images, outperforming earlier technologies in quality, diversity, and generation speed. At its core, CLAY includes a multi-resolution variational autoencoder and a diffusion transformer, allowing direct processing of 3D content without conversion. By training on over 500,000 3D models, CLAY can produce objects ranging from everyday items to complex creatures and achieve precise control through additional inputs, generating urban scenes and reconstructing 3D models from sketches.

From June 26 to 28, 2024, the China venue MWC will take place in Shanghai, focusing on 5G and AI. The event will concentrate on topics such as the transformation brought by 5G, intelligent manufacturing, and the future of AI. Attendees will exchange ideas on IoT, 6G, industry transformation, smart cities, and discuss the impact of generative AI, as well as learn about the latest developments in Industry 4.0. Notable companies such as Huawei, Lenovo, and ZTE have confirmed their participation.

Shanghai AI Lab, together with Ruijin Hospital affiliated to Shanghai Jiao Tong University School of Medicine, has released the medical multimodal foundational model group 'PuYi 2.0'. PuYi 2.0 adds multi-domain models and incremental language parameters, covering various data modalities such as medical images, medical texts, and bioinformatics. New open-source datasets include the medical image segmentation dataset SA-Med2D-20M and the pathology dataset SNOW. PuYi 2.0 incorporates an evaluation module to provide reference implementations for medical model capabilities, delivering a one-stop open-source solution for large medical models.

Shanghai Steel Union has applied to register the trademark 'Steel Union Master Large Language Model'. The Steel Union company has released a vertical large language model for the bulk commodity industry called 'Steel Union Master'. This model possesses various capabilities, including price and data inquiries, as well as industry encyclopedia Q&A. The application falls under the international classification for design research and is currently in the status of 'Registration Application'. This trademark application further consolidates Shanghai Steel Union's position in the large language model field.

Shanghai Jiao Tong University, in collaboration with Baidu, has jointly released the Version 2.0 of the 白玉兰科学大模型. This version includes 'Legal Open Source' and 'Chemical Synthesis 2.0'. The 白玉兰科学大模型 — Legal Open Source surpasses other open-source Chinese general models in comprehensive legal assessment testing. This collaboration promotes advancements in the field of AI for Science. Both parties have made new progress in their cooperation within AI for Science, setting a benchmark for collaboration.

The Shanghai Municipal Commission of Economy and Informatization, the Development and Reform Commission, and three other departments have jointly formulated and officially released several measures to promote the innovation and development of artificial intelligence large models in Shanghai. The document proposes 11 measures focusing on enhancing innovation capabilities of large models, supplying innovative elements, and applying large model innovations. It specifies that Shanghai will implement support plans for large model innovation, establish a testing and evaluation center, and conduct testing and validation focusing on model performance, safety, and ethics to support large model innovation. Shanghai will also build a collaborative ecosystem for intelligent chip hardware and software, support chip companies in large-scale validation, and promote large model innovations.

1. Pu Yu Ling Bi enables fluent bilingual (Chinese and English) interactions with images, accurately understanding image content and demonstrating a profound knowledge of Chinese culture. 2. Pu Yu Ling Bi unlocks a new creative ability for rich text and imagery articles, allowing users to generate articles with images by simply providing a topic. 3. Pu Yu Ling Bi employs a 'three-step' algorithmic process for creating image-text articles: generating text, planning illustrations, and intelligently selecting images. 4. In evaluations across several mainstream multimodal models, Pu Yu Ling Bi consistently performs at a leading level, particularly excelling in Chinese multimodal understanding. 5. Pu Yu

Shanghai AI Lab has released the large model training toolbox XTuner, which supports various hardware adaptations, significantly lowering the training cost barrier. XTuner is compatible with multiple open-source large models and can execute tasks such as incremental pre-training and instruction fine-tuning. XTuner balances usability and configurability with a built-in standardized process for one-click training.

Shanghai AI Lab, in collaboration with the Corpus Data Alliance, has open-sourced the 'Shusheng・Wanjuan' 1.0 multi-modal pre-trained dataset, which includes text, images, and video datasets, totaling over 2TB. The dataset has undergone fine-grained cleaning and deduplication, featuring multi-dimensional integration, meticulous processing, and ease of use. The release of this open-source dataset will help promote the application and innovation of large models and lower the technical barriers associated with large model technologies.