Large Language Models (LLMs) like GPT and Llama have sparked a revolution in the field of artificial intelligence, but efficiently training these massive models while aligning them with human values remains a challenge. Reinforcement Learning with Human Feedback (RLHF) has become an important training method for LLMs in recent years, but traditional RLHF frameworks have limitations in flexibility, efficiency, and scalability. To address these issues, the ByteDance Doubao Large Model Team has open-sourced an RLHF framework called HybridFlow.