{"id":22508,"date":"2024-11-04T02:27:11","date_gmt":"2024-11-03T18:27:11","guid":{"rendered":"https:\/\/www.1ai.net\/?p=22508"},"modified":"2024-11-03T20:28:40","modified_gmt":"2024-11-03T12:28:40","slug":"%e6%8f%90%e5%8d%87-1-520-%e5%80%8d%e5%90%9e%e5%90%90%e9%87%8f%ef%bc%8c%e5%ad%97%e8%8a%82%e8%b1%86%e5%8c%85%e5%a4%a7%e6%a8%a1%e5%9e%8b%e5%9b%a2%e9%98%9f%e4%b8%8e%e9%a6%99%e6%b8%af%e5%a4%a7%e5%ad%a6","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/22508.html","title":{"rendered":"1.5~20 times higher throughput, ByteBeanBag Big Model team and HKU release and open source new RLHF framework"},"content":{"rendered":"<p><a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%ad%97%e8%8a%82%e8%b7%b3%e5%8a%a8\" title=\"[View articles tagged with [bytejump]]\" target=\"_blank\" >ByteDance<\/a><a href=\"https:\/\/www.1ai.net\/en\/tag\/%e8%b1%86%e5%8c%85%e5%a4%a7%e6%a8%a1%e5%9e%8b\" title=\"Look at the article that contains the label\" target=\"_blank\" >Bean bag large model<\/a>Teams and<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e9%a6%99%e6%b8%af%e5%a4%a7%e5%ad%a6\" title=\"[Sees articles with labels]\" target=\"_blank\" >The University of Hong Kong<\/a>Publicizing the results of joint research --\u00a0<strong>HybridFlow<\/strong>.<\/p>\n<p>The official claim that HybridFlow (open source project name: veRL) is a flexible and efficient large model RL training framework , compatible with a variety of training and inference frameworks , support for flexible model deployment and a variety of RL algorithm implementation .<\/p>\n<p>The framework adopts a hybrid programming model that combines the flexibility of Single-Controller and the efficiency of Multi-Controller to better implement and execute multiple RL algorithms, significantly improve training throughput, and reduce development and maintenance complexity.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-22509\" title=\"faf3082bj00smdjy80093d000u000cmp\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/11\/faf3082bj00smdjy80093d000u000cmp.jpg\" alt=\"faf3082bj00smdjy80093d000u000cmp\" width=\"1080\" height=\"454\" \/><br \/>\n\u25b2 Flow of one iteration of 3D-HybridEngine (Hybrid Technology for Training Reasoning)<\/p>\n<p>Experimental results show that HybridFlow, under various model sizes and RL algorithms, the<strong>1.5x to 20x increase in training throughput compared to other frameworks<\/strong>.<\/p>\n<p>The paper has now been accepted by EuroSys 2025 and the code repository has been made available to the public with relevant links below:<\/p>\n<ul>\n<li>Link to paper: https:\/\/arxiv.org\/abs\/2409.19256<\/li>\n<li>Link to code: https:\/\/github.com\/volcengine\/veRL<\/li>\n<\/ul>","protected":false},"excerpt":{"rendered":"<p>HybridFlow is the result of a joint research project between ByteDance's Beanbag Big Model team and the University of Hong Kong. Officially, HybridFlow (open source project name: veRL) is a flexible and efficient RL training framework for big models, compatible with a variety of training and inference frameworks, and supporting flexible model deployment and Multiple RL algorithm implementation. The framework adopts a hybrid programming model that combines the flexibility of a single controller (Single-Controller) and multiple controllers (Multi-Controller) efficiency , can better implement and execute a variety of RL algorithms , significantly improve the training throughput , reduce the complexity of development and maintenance . The 3D-HybridEngine is a single iteration of the 3D-HybridEngine.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[548,2610,1704],"collection":[],"class_list":["post-22508","post","type-post","status-publish","format-standard","hentry","category-news","tag-548","tag-2610","tag-1704"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/22508","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=22508"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/22508\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=22508"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=22508"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=22508"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=22508"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}