Skywork

SkyReels V1: The first and most advanced open-source human-centric video foundation model

Pioneering Multimodal Reasoning with CoT

Skywork series models are pre-trained on 3.2TB of high-quality multilingual (mainly Chinese and English) and code data. We have open-sourced the model, training data, evaluation data, evaluation methods, etc.

MoH: Multi-Head Attention as Mixture-of-Head Attention

[ICLR 2025] MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts