{"id":9996,"date":"2024-05-10T10:09:32","date_gmt":"2024-05-10T02:09:32","guid":{"rendered":"https:\/\/www.1ai.net\/?p=9996"},"modified":"2024-05-10T10:09:32","modified_gmt":"2024-05-10T02:09:32","slug":"openbuddy%e5%8f%91%e5%b8%83%e6%96%b0%e4%b8%80%e4%bb%a3%e4%b8%ad%e6%96%87%e8%b7%a8%e8%af%ad%e8%a8%80%e5%a4%a7%e6%a8%a1%e5%9e%8b-%e5%9f%ba%e4%ba%8ellama-3-70b%e6%89%93%e9%80%a0","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/9996.html","title":{"rendered":"OpenBuddy releases a new generation of Chinese cross-language large model based on Llama 3 70B"},"content":{"rendered":"<p><a href=\"https:\/\/www.1ai.net\/en\/tag\/openbuddy\" title=\"_Other Organiser\" target=\"_blank\" >OpenBuddy<\/a>\u00a0The team released the Meta Llama370B-based<span class=\"spamTxt\">up to date<\/span>Results:OpenBuddy-Llama3-70B-v21.1-8k, this is their<span class=\"spamTxt\">first<\/span>Chinese cross-language training attempts on the Llama 370B base.<\/p>\n<p>Llama370B is a 15T training set trained on a dense<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%a4%a7%e6%a8%a1%e5%9e%8b\" title=\"[View articles tagged with [large models]]\" target=\"_blank\" >Large Model<\/a>However, due to the lack of Chinese data in the training set, Meta's official Llama3 series of dialog models have weak cognitive and reasoning abilities in the Chinese domain and are prone to answering Chinese questions in English.<\/p>\n<p class=\"article-content__img\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9997\" title=\"202401041559152376_1\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/05\/202401041559152376_1.jpg\" alt=\"202401041559152376_1\" width=\"1000\" height=\"666\" \/><\/p>\n<p>Therefore, the OpenBuddy team chose the Llama370B Base model as the basis for this training, focusing on improving the model's cross-language comprehension and cognitive abilities.<\/p>\n<p>During the test, OpenBuddy-Llama3-70B demonstrated excellent Chinese comprehension with more cognitive potential and post-quantization performance. In this release, they expanded the existing word list of Llama3 by adding more than 7,000 Chinese words to further improve the Chinese coding density of the model.<\/p>\n<p>It is worth mentioning that, in the quantized version of the model capacity decline is more obvious, they only released the full copyright weight of the model, and will taste the optimization of the 70B model quantized performance, and pick the right time to release the quantized version.<\/p>","protected":false},"excerpt":{"rendered":"<p>OpenBuddy team released the latest results based on Meta Llama370B: OpenBuddy-Llama3-70B-v21.1-8k, which is their first attempt of Chinese cross-language training on Llama370B base. Llama370B is a dense model trained on a 15T training set, however, due to the lack of Chinese data in the training set, Meta's official Llama3 series of dialog models have weak cognitive and reasoning abilities in the Chinese domain, and are prone to answering Chinese questions in English. Therefore, the OpenBuddy team chose the Llama370B Base model as the basis for this training, focusing on improving the model's cross-language understanding.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[2532,216],"collection":[],"class_list":["post-9996","post","type-post","status-publish","format-standard","hentry","category-news","tag-openbuddy","tag-216"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/9996","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=9996"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/9996\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=9996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=9996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=9996"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=9996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}