ViTPose is an open-source action estimation model that excels at recognizing human postures, as if it can understand the actions you are performing. The standout feature of this model is its simplicity and efficiency; it does not use complex network structures but directly employs a technique called Vision Transformer. The core of ViTPose uses a pure Vision Transformer, which acts like a powerful 'skeleton' to extract key features from images. Unlike other models, it does not require complexity.

Against the backdrop of the global generative AI wave in 2022, YuanShi Intelligence (RWKV) completed tens of millions of RMB in angel round financing in December 2023, with investment from Tianji Capital. Following this financing, the company’s valuation doubled, and the funds will be used for team expansion, new architecture development, and product commercialization. The emergence of RWKV presents a strong challenge to the traditional Transformer architecture. With the advancement of large language models (LLMs), although the parameter scale of models is becoming increasingly large, issues like hallucination and accuracy persist.

An innovative point cloud compression technology (TSC-PCAC) developed by a joint team from the Chinese Academy of Sciences, Tongji University, and Ningbo University has achieved a significant breakthrough. This technology not only greatly improves the compression efficiency of point cloud data but also significantly reduces processing time, clearing technical obstacles for the development of 3D applications like AR/VR. In the context of the rapid advancement of 3D vision technology, point clouds, as a key data format for virtual and augmented reality, face tremendous transmission and storage challenges. A high-quality point cloud can contain millions of data points, with each point...

In today's digital era, image generation technology is developing at an astonishing pace. Recently, a research team from the National University of Singapore proposed a brand-new framework — OminiControl, aimed at enhancing the flexibility and efficiency of image generation. This framework combines image conditions and fully utilizes the pre-trained Diffusion Transformer (DiT) model, providing unprecedented control capabilities. In simple terms, as long as you provide the source image, OminiControl can utilize it.

Caiyun Technology recently held the 'From Paper to App' themed communication conference in Beijing, officially releasing the general large model Yunjin Tianzhang based on the DCFormer architecture and announcing the upgrade of its AI RPG platform Caiyun Xiaomeng to version V3.5 based on DCFormer. This marks a significant breakthrough in model architecture efficiency in the field of artificial intelligence. In the AI field, the Transformer architecture has always been the core technical support for mainstream large models like ChatGPT and Gemini.

Generating high-quality, temporally continuous videos requires a significant amount of computational resources, especially for longer time spans. The latest Diffusion Transformer model (DiTs) has made significant progress in video generation, but its reliance on larger models and more complex attention mechanisms has resulted in slower inference speeds, exacerbating the challenge. To address this issue, researchers at Meta AI have proposed a training-free method called AdaCache to accelerate video DiTs.

Recent research from Alibaba's Tongyi Laboratory indicates that existing text-to-image Diffusion Transformer models already possess the ability to generate multiple images with specific relational contexts. With just a little 'guidance', they can achieve a synthesis of understanding and produce high-quality multi-image collections. Traditional Diffusion models are more like 'rote learning' students, requiring massive datasets to generate high-quality images. However, with the support of IC-LoRA, they become 'high-achieving students', capable of learning through analogy.

This week, MIT showcased a new model for training robots, aimed at addressing the challenges faced by imitation learning when introduced to small obstacles. Researchers pointed out that imitation learning can fail in conditions such as varying lighting, different environments, or new obstacles because robots simply do not have enough data to adapt. The team sought powerful data methods, like those used in models such as GPT-4, to tackle the problem. They introduced a new architecture called the Heterogeneous Pretrained Transformer (HPT), which aggregates information from different sensors and environments.

Chinese Academy of Engineering academician Wang Jian shared his in-depth thoughts on AI, AI+, and AI infrastructure at the opening ceremony of the 2024 Bund Conference. He emphasized that AI+ is not merely a simple integration of AI and industry, but rather the integration of data, models, and computing power, with cloud computing serving as the infrastructure of the AI era. Wang Jian pointed out that although artificial intelligence has a long history, it has only truly impacted industries for the past 7 years, with the critical turning point being Google’s introduction of the Transformer in 2017. Under the logic of AI+, ChatGPT is not just an application, but a broader application platform.

xAI has released two intelligent models, Grok-2 and Grok-2mini, showcasing significant performance improvements with powerful reasoning, programming, dialogue, and image generation capabilities, surpassing Claude 3.5 and GPT-4 Turbo. The new models excel in solving mathematical problems, news querying, writing cover letters, and game recommendations, and are integrated with FLUX.1 for multitasking. The interface and features on the 𝕏 platform have been updated, offering richer services for Premium and Premium+ users. Enterprise A

The Transformer architecture, a star in the field of artificial intelligence, has led a revolution in natural language processing with its self-attention mechanism at its core. However, when handling long contexts, the resource consumption of self-attention calculations becomes a bottleneck. To address this issue, researchers have proposed the Tree Attention method, which decomposes the calculation tasks through tree reduction to improve efficiency. This method not only reduces communication overhead and memory usage but is also 8 times faster than existing methods in a multi-GPU environment. The introduction of Tree Attention not only enhances performance.

This is an interactive visualization tool specifically designed for non-experts. With this tool, we can directly run the GPT-2 model in the browser and observe in real time how the model 'understands' our text step by step and predicts the next word.

In the field of artificial intelligence, the Transformer model has gained widespread attention for its outstanding performance in language tasks. The recent research paper 'Transformer Layers as Painters' explores the hierarchical structure of the Transformer model from an innovative perspective, comparing each layer to a painter that creates complex and rich texts on the canvas of language. The study reveals the working mechanism of Transformer layers through experiments, particularly how they collaborate with each other.

Zhipu AI announces the open source of its video generation model CogVideoX, aiming to promote the rapid development and application of video generation technology. The CogVideoX model is based on advanced large model technology and is capable of meeting commercial-grade application requirements. The currently open sourced CogVideoX-2B version requires only 18GB of GPU memory for inference at FP-16 precision and 40GB for fine-tuning, allowing inference with a single 4090 GPU and fine-tuning with a single A6000 GPU.

Scientists at ShanghaiTech University have developed CLAY, an AI model that can generate detailed 3D objects from text or 2D images, outperforming earlier technologies in quality, diversity, and generation speed. At its core, CLAY includes a multi-resolution variational autoencoder and a diffusion transformer, allowing direct processing of 3D content without conversion. By training on over 500,000 3D models, CLAY can produce objects ranging from everyday items to complex creatures and achieve precise control through additional inputs, generating urban scenes and reconstructing 3D models from sketches.