{"id":16382,"date":"2024-07-25T08:37:42","date_gmt":"2024-07-25T00:37:42","guid":{"rendered":"https:\/\/www.1ai.net\/?p=16382"},"modified":"2024-07-25T08:37:42","modified_gmt":"2024-07-25T00:37:42","slug":"openbuddy%e5%bc%80%e6%ba%90%e5%a4%a7%e8%af%ad%e8%a8%80%e6%a8%a1%e5%9e%8b%e5%9b%a2%e9%98%9f%e5%8f%91%e5%b8%83llama3-1-8b%e6%a8%a1%e5%9e%8b%e4%b8%ad%e6%96%87%e7%89%88","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/16382.html","title":{"rendered":"OpenBuddy open source large language model team released the Chinese version of Llama3.1-8B model"},"content":{"rendered":"<p data-track=\"1\" data-pm-slice=\"0 0 []\">Meta recently released a new generation<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%bc%80%e6%ba%90%e6%a8%a1%e5%9e%8b\" title=\"[See articles with [open source model] labels]\" target=\"_blank\" >Open Source Model<\/a>series<a href=\"https:\/\/www.1ai.net\/en\/tag\/llama3-1\" title=\"[See articles with [Llama3.1] labels]\" target=\"_blank\" >Llama3.1<\/a>, including a 405B parameter version that approaches or even surpasses closed-source models such as GPT-4 in some benchmarks. Llama3.1-8B-Instruct is an 8B parameter version in the series, supporting English, German, French, Italian, Portuguese, Spanish, Hindi, and Thai, with a context length of up to 131072 tokens, and the knowledge deadline is updated to December 2023.<\/p>\n<p data-track=\"2\">To enhance the capabilities of Llama3.1-8B-Instruct, Meta used more than 25 million pieces of synthetic data generated by a larger 405B model in training. This enables Llama3.1-8B-Instruct to demonstrate cognitive and reasoning capabilities similar to GPT3.5Turbo in tests such as code and mathematics.<\/p>\n<div class=\"pgc-img\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-16383\" title=\"get-792\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/07\/get-792.jpg\" alt=\"get-792\" width=\"921\" height=\"595\" \/><\/div>\n<p data-track=\"3\"><a href=\"https:\/\/www.1ai.net\/en\/tag\/openbuddy\" title=\"_Other Organiser\" target=\"_blank\" >OpenBuddy<\/a>Using the Llama3.1-8B-Instruct model, OpenBuddy-Llama3.1-8B-v22.1-131K was released by training on a small amount of Chinese data. This is a new generation of open source cross-language model with Chinese question answering and cross-language translation capabilities. Although Llama3.1 itself does not have Chinese capabilities, after training, the model can generate answers that are usually only generated by larger models on some questions that are prone to conceptual confusion, showing stronger cognitive potential.<\/p>\n<p data-track=\"4\">However, due to the limitation of training dataset and time, OpenBuddy-Llama3.1-8B-v22.1 still has limitations in Chinese knowledge, especially traditional cultural knowledge. Nevertheless, the model shows relatively stable performance in tasks such as long text comprehension, which benefits from its original long text capability.<\/p>\n<p data-track=\"5\">In the future, OpenBuddy plans to conduct larger-scale training of the 8B and 70B models to enhance the models\u2019 Chinese knowledge reserves, long-text capabilities, and cognitive abilities, and explore the possibility of fine-tuning the 405B model.<\/p>\n<p data-track=\"6\"><strong>Project address:<\/strong>https:\/\/modelscope.cn\/models\/OpenBuddy\/openbuddy-llama3.1-8b-v22.1-131k<\/p>\n<p>&nbsp;<\/p>","protected":false},"excerpt":{"rendered":"<p>Meta recently released Llama3.1, a next-generation family of open-source models, which includes a 405B-parameter version that comes close to, and even outperforms, closed-source models such as the GPT-4 in some benchmarks.Llama3.1-8B-Instruct is an 8B-parameter version of the family that supports English, German, French, Italian, Portuguese, Spanish, Hindi, and Thai, with a context length of up to 131,072tokens and a knowledge deadline updated to December 2023. To enhance the capabilities of Llama3.1-8B-Instruct, Meta used more than 25 million pieces of synthetic data in training, generated by the larger 405B model. This allows Llama3.1-<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[3670,2532,862],"collection":[],"class_list":["post-16382","post","type-post","status-publish","format-standard","hentry","category-news","tag-llama3-1","tag-openbuddy","tag-862"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/16382","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=16382"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/16382\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=16382"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=16382"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=16382"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=16382"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}