Beijing Zhipu Huazhang Technology Co., Ltd. announced a series of significant technological updates on August 29, 2024, including the release of a new generation foundation model and new application services. At the KDD2024 conference, Zhipu introduced new foundation models including the language model GLM-4-Plus, the text-to-image model CogView-3-Plus, the image/video understanding model GLM-4V-Plus, and the video generation model CogVideoX. These models have achieved international leading standards in their respective fields.

The domestic open source video generation model CogVideoX-5B has been officially released in the Mota ModelScope community, significantly improving the quality and visual effects of video generation. Based on the large-scale DiT model, this model utilizes a 3D causal variational autoencoder and expert Transformer technology, achieving spatio-temporal joint modeling through 3D-RoPE positional encoding and a 3D full attention mechanism. The use of progressive training techniques allows the model to generate long videos with distinct motion features, coherence, and high quality.

Zhipu AI announces the open source of its video generation model CogVideoX, aiming to promote the rapid development and application of video generation technology. The CogVideoX model is based on advanced large model technology and is capable of meeting commercial-grade application requirements. The currently open sourced CogVideoX-2B version requires only 18GB of GPU memory for inference at FP-16 precision and 40GB for fine-tuning, allowing inference with a single 4090 GPU and fine-tuning with a single A6000 GPU.

Zhipu AI has launched CogVideoX, a next-generation video generation model that enhances temporal frame relationship capturing through a 3D VAE structure and 3D RoPE module. This model integrates text, time, and space into a transformer architecture, allowing for efficient video generation that meets user needs. It is available on the Zhipu Qingyan platform for free video generation services, featuring fast generation, effective instruction adhere....