{"id":8481,"date":"2024-04-19T10:06:27","date_gmt":"2024-04-19T02:06:27","guid":{"rendered":"https:\/\/www.1ai.net\/?p=8481"},"modified":"2024-04-19T10:06:27","modified_gmt":"2024-04-19T02:06:27","slug":"%e5%a4%a7%e6%a8%a1%e5%9e%8b%e9%82%a3%e4%b9%88%e7%81%ab%ef%bc%8c%e6%95%99%e4%bd%a0%e4%b8%80%e9%94%ae%e7%8e%a9%e8%bd%ac%e5%bc%80%e6%ba%90llama3%e5%a4%a7%e6%a8%a1%e5%9e%8b","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/8481.html","title":{"rendered":"Big models are so popular, teach you how to play with open source Llama3 big models with one click"},"content":{"rendered":"
\"get-599\"<\/div>\n

Llama 3<\/a>Released today, it provides pre-trained and instruction-fine-tuned language models with 8B and 70B parameters, and these models will soon be available on mainstream platforms such as AWS, Google Cloud, and Microsoft Azure, with strong support from hardware platforms such as AMD and Intel.<\/p>\n

Llama<\/a>3 Direct links:https:\/\/llama.meta.com\/llama3<\/u><\/p>\n

The Llama Chinese Community will take you to learn more about and use Llama3 from the following aspects:<\/p>\n

1. Llama3 introduction, performance and technical analysis<\/p>\n

2. How to experience and download Llama3 models<\/p>\n

3. How to call Llama3<\/p>\n

4. Chat with industry experts about Llama3<\/p>\n

1. Introduction to Llama3<\/strong><\/p>\n

\u201cThe best open sourceLarge Model<\/a>\u201d<\/strong><\/p>\n

The new Llama 3 models, including 8B and 70B parameter versions, are a major upgrade of Llama 2. Llama3 pre-trained models and instruction fine-tuning models perform well in the 8B and 70B parameter scales, becomingThe current open source model is the best<\/strong>The post-training improvements significantly reduced the false rejection rate, improved consistency, and increased the diversity of the model\u2019s responses.<\/p>\n

Llama3 model details at a glance<\/strong><\/p>\n

\"get-601\"<\/div>\n

Performance<\/strong><\/p>\n

Llama 3 inGreat progress has also been made in functions such as reasoning, code generation and instruction tracing<\/strong>, the model is easier to control. The performance and user experience of the model are significantly improved.<\/p>\n

Llama 3 8B outperforms other open source models such as Mistral\u2019s Mistral 7B and Google\u2019s Gemma 7B in at least 9 benchmarks. Both models contain 7 billion parameters. Llama 3 8B performs well in the following benchmarks:<\/p>\n

\n

MMLU: A Multi-Task Language Understanding Benchmark.<\/p>\n

ARC: A test of complex reading comprehension.<\/p>\n

DROP: A digital reading comprehension test.<\/p>\n

GPQA: A set of questions covering biology, physics, and chemistry related issues.<\/p>\n

HumanEval: Code generation testing.<\/p>\n

GSM-8K: Math word problems.<\/p>\n

MATH: Mathematics benchmark test.<\/p>\n

AGIEval: Problem Solving Test Set.<\/p>\n

BIG-Bench Hard: An assessment of common sense reasoning.<\/p>\n

\n<\/blockquote>\n

Llama 3 70B outperforms the weaker Claude 3 Sonnet in the Claude 3 family on five benchmarks, including MMLU, GPQA, HumanEval, GSM-8K, and MATH. These results highlight the superior performance of the Llama 3 70B model across a wide range of application domains.<\/p>\n

\"get-602\"<\/div>\n

During the development of Llama 3, we not only focused on model performance, but also focused on optimizing performance in actual application scenarios.<\/p>\n

The team created a new set of high-quality human evaluations<\/strong>, covers 1,800 prompts across 12 key use cases: suggestion consultation, brainstorming, classification, closed question and answer, programming, creative writing, information extraction, character creation, open question and answer, reasoning, rewriting, and summarizing.<\/p>\n

The figure below shows the human evaluation results for Claude Sonnet, Mistral Medium, and GPT-3.5 on these categories and prompts.<\/p>\n

\"get-600\"<\/div>\n

\n

andThe Llama3-8B model performed better than the Llama2-70B model in the test results.<\/strong><\/p>\n

\"get-603\"<\/div>\n

Technical Details<\/strong><\/p>\n

The development of Llama 3 emphasizes excellent language model design, focusing on innovation, expansion, and optimization. The project revolves around four key elements: model architecture, pre-training data, expansion of pre-training scale, and instruction fine-tuning.<\/p>\n

Model Architecture<\/h1>\n

Llama 3 uses a relatively standard decoder-only Transformer architecture and makes key improvements over Llama 2. The model uses a 128K token vocabulary, which improves the efficiency of language encoding and significantly improves performance.<\/p>\n

In the 8B and 70B scale models,Llama 3 introduces Grouped Query Attention (GQA)<\/strong>, and trained the model on 8,192 labeled sequences, using masks to ensure that self-attention does not cross document boundaries, thereby improving the inference efficiency of the model.<\/p>\n

Training Data<\/h1>\n

In order to build excellent language models, it is essential to manage large, high-quality training datasets. Llama 3 is pre-trained with more than 15T tokens, all of which are from public sources.Seven times the size of the Llama 2 training dataset and contains four times more code<\/strong>.<\/p>\n

also,Over 5%'s pre-training dataset consists of high-quality non-English data covering more than 30 languages<\/strong>.<\/p>\n

To ensure that the model acceptsHighest quality training data<\/strong>,Llama 3 developed a series of data filtering pipelines, including heuristic filters, NSFW filters, semantic deduplication methods, and text quality predictors.<\/p>\n

Scaling up pre-training<\/h1>\n

During the development of Llama 3, a lot of effort was put into scaling up pre-training. By developing detailed scaling rules, the team was able to optimize the data combination to ensure optimal use of training compute. 15T tokens training<\/strong>After that, the 8B and 70B parameter models continue to improve in a log-linear manner.<\/p>\n

In addition, by combining data parallelism, model parallelism, and pipeline parallelismThree types of parallel training methods<\/strong>, Llama 3 achieves efficient training on two custom 24K GPU clusters.<\/p>\n

The combined application of these technologies and methods ensuresThe training efficiency of Llama 3 is about three times higher than that of Llama 2.<\/strong>, providing users with a better experience and more powerful model performance.<\/p>\n

Instruction fine-tuning<\/h1>\n

To fully realize the potential of pre-trained models for conversational use cases, the Llama 3 team used a combination of techniques including supervised fine-tuning (SFT), rejection sampling, proximal policy optimization (PPO), and direct policy optimization (DPO).<\/p>\n

The quality of the hints used in SFT and the preference ranking used in PPO and DPO have a huge impact on the performance of the alignment model.<\/strong>By carefully curating the data and performing quality assurance on the annotations, Llama 3 achieves significant improvements on both reasoning and encoding tasks.<\/p>\n

Computing power consumption and carbon emissions<\/h1>\n

Llama3 pre-training uses H100-80GB (thermal design power consumption TDP is 700W),Training required 7.7 million GPU hours<\/strong>Total carbon emissions were 2,290 tonnes of carbon dioxide equivalent (tCO2eq), all of which were offset through Meta\u2019s sustainability program.<\/p>\n

\"get-604\"<\/div>\n

2.Llama3 model experience and download<\/strong><\/p>\n

\"get-605\"<\/div>\n

Hugging face experience link:<\/p>\n

https:\/\/huggingface.co\/chat\/<\/p>\n

Meta.ai experience link:<\/p>\n

https:\/\/www.meta.ai\/<\/p>\n

Model download application:<\/p>\n

https:\/\/llama.meta.com\/llama-downloads<\/p>\n

It is recommended that you experience it on Hugging face first. The community\u2019s official website https:\/\/llama.family is also launching links and model downloads for domestic experience.<\/p>\n

3. How to call Llama3<\/strong><\/p>\n

Llama 3 uses several special tags:<\/p>\n

<|begin_of_text|><\/strong>: Equivalent to the BOS tag, marking the beginning of a sentence.<\/p>\n

<|eot_id|><\/strong>: Equivalent to the EOS marker, marking the end of a sentence.<\/p>\n

<|start_header_id|>{role}<|end_header_id|><\/strong>: Identifies the role corresponding to a message, which can be "system", "user" and "assistant".<\/p>\n

Basic model call<\/strong><\/p>\n

Llama 3 basic model call is relatively simple, at the start marker<|begin_of_text|><\/strong>Just add the user information at the end, and the model will generate subsequent text based on the {{ user_message }} information.<\/p>\n

<|begin_of_text|>{{ user_message }}<\/code><\/pre>\n
Dialogue model call<\/strong><\/p>\n
For a single-round conversation, first you need to use part 1 <|begin_of_text|> to mark the beginning of the prompt, then part 2 to mark the role (for example, "user"), part 3 to contain the specific conversation information, and part 4 <|eot_id|> to mark the end of the text. Then part 5 to mark the next role (for example, "assistant"). The model will generate a conversation reply message after the prompt, that is, {{assistant_message}}.<\/p>\n
<|begin_of_text|>1<|start_header_id|>user<|end_header_id|>2 {{user_message }}3<|eot_id|>4<|start_header_id|>assistant<|end_header_id|>5<\/code><\/pre>\n
In addition, you can also add system information to the prompt, for example, add {{ system_prompt }} after the system logo.<\/p>\n
<|begin_of_text|><|start_header_id|>system<|end_header_id|> {{system_prompt }} <|start_header_id|>user<|end_header_id] {{user_message }}<|eot_id|><|start_header_id|>assistant<\/code><\/pre>\n
The same is true for multi-round conversations. By representing multiple pieces of user and assistant information, the model can generate multi-round conversations.<\/p>\n
<|begin_of_text|><|start_header_id|>system<|end_headder_id {{system_prompt }}<|eot_id|><|start_header_id|>user<|end_header_id]> {{user_message_1 }}<|eot_id|><|start_header_id|> assistant<|end_header_id|> {{ model_answer_1}}<|eot_id|><|start_header_id|>user {{user_message_2}}<|eot_id|><|start_header_id|>assisttant<|end_header_id|><\/code><\/pre>\n
4. Next steps<\/strong><\/p>\n
The Llama 3 400B model is in training..<\/strong><\/p>\n
\"get-606\"<\/div>\n
 <\/p>","protected":false},"excerpt":{"rendered":"
Llama 3 is released today, providing pre-trained and instruction fine-tuned language models with 8B and 70B parameters, and these models will soon be available on mainstream platforms such as AWS, Google Cloud, Microsoft Azure, etc., with strong support from hardware platforms such as AMD and Intel. Llama3 link direct: https:\/\/llama.meta.com\/llama3 Llama Chinese community will take you from the following aspects of in-depth understanding and use of Llama3: 1, Llama3 introduction, performance and technical analysis 2, how to experience and download the Llama3 model 3, Llama3 invocation method 4, and industry experts Talk about Llama3 together 1.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[149,144],"tags":[184,1671,216],"collection":[],"class_list":["post-8481","post","type-post","status-publish","format-standard","hentry","category-jiaocheng","category-baike","tag-llama","tag-llama-3","tag-216"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/8481","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=8481"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/8481\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=8481"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=8481"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=8481"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=8481"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}