HybridFlow (open source project: veRL) is a flexible and efficient large model RL training framework , compatible with a variety of training and reasoning frameworks to support flexible model deployment and a variety of RL algorithms. It adopts a hybrid programming model that combines the advantages of single and multiple controllers to better implement and execute multiple RL algorithms, significantly improve training throughput, and reduce development and maintenance complexity. Experimental results show that HybridFlow can improve training throughput by 1.5x to 20x compared to other frameworks under various model sizes and RL algorithms. The framework is released and open-sourced by the ByteBeanBag Big Model team and the University of Hong Kong.
Paper address:
https://arxiv.org/abs/2409.19256
Open source address:
https://github.com/volcengine/veRL