{"id":22783,"date":"2024-11-09T03:22:43","date_gmt":"2024-11-08T19:22:43","guid":{"rendered":"https:\/\/www.1ai.net\/?p=22783"},"modified":"2024-11-08T21:24:12","modified_gmt":"2024-11-08T13:24:12","slug":"meta-%e5%bc%80%e6%ba%90%e5%b0%8f%e8%af%ad%e8%a8%80-ai%e6%a8%a1%e5%9e%8b-mobilellm-%e5%ae%b6%e6%97%8f%ef%bc%9a%e9%80%82%e7%94%a8%e6%99%ba%e8%83%bd%e6%89%8b%e6%9c%ba%e3%80%81%e6%8f%90%e4%be%9b-125m-1b","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/22783.html","title":{"rendered":"Meta Open Source Small-Language AI Models MobileLLM Family: Smartphone Friendly, 125M-1B Version Available"},"content":{"rendered":"<p><a href=\"https:\/\/www.1ai.net\/en\/tag\/meta\" title=\"[View articles tagged with [Meta]]\" target=\"_blank\" >Meta<\/a> issued a press release last week announcing the official<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%bc%80%e6%ba%90\" title=\"[View articles tagged with [open source]]\" target=\"_blank\" >Open Source<\/a>The MobileLLM family of small language models that run on smartphones, and the addition of three new parameterized versions of the family, 600M, 1B, and 1.5B, are available on the project's GitHub project page (click here to visit).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-22784\" title=\"e6d4a8bbj00smmvux001gd000pr00jyp\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/11\/e6d4a8bbj00smmvux001gd000pr00jyp.jpg\" alt=\"e6d4a8bbj00smmvux001gd000pr00jyp\" width=\"927\" height=\"718\" \/><\/p>\n<p>According to Meta researchers, the MobileLLM family of models, built specifically for smartphones, is claimed to have a streamlined architecture and introduces a \"SwiGLU activation function,\" \"grouped-query attention,\" and a \"grouped-query attention\" mechanism to balance efficiency and performance outcomes. The model family is designed for smartphones and claims to use a streamlined architecture and introduces the \"SwiGLU activation function\" and \"grouped-query attention\" mechanism, which can balance efficiency and performance results.<\/p>\n<p>Additionally, MobileLLM models are claimed to be faster to train, with Meta researchers claiming that when they trained MobileLLM models with varying number of covariates on 1 trillion words (tokens) in a server environment with 32 Nvidia A100 80G GPUs, they<strong>The 1.5B version takes only 18 days and the 125M version takes only 3 days.<\/strong>.<\/p>\n<p>And from the results, the MobileLLM 125M and 350M models are 2.7% and 4.3% more accurate than the State of the Art (SOTA) models such as Cerebras, OPT, and BLOOM, respectively, in the zero-sample general knowledge comprehension task.<\/p>\n<p>The Meta researchers also compared MobileLLM-1.5B to other models in the industry with larger parameter counts, and claimed to be ahead of models such as GPT-neo-2.7B, OPT-2.7B, BLOOM-3B, and Qwen 1.5-1.8B in terms of outcome testing.<\/p>","protected":false},"excerpt":{"rendered":"<p>Meta last week issued a press release announcing that it has officially open sourced the MobileLLM family of small language models that run on smartphones, and has added three new parameterized versions of the family, 600M, 1B, and 1.5B to the project's GitHub project page (click here to visit). According to Meta researchers, the MobileLLM family of models is designed for smartphones, and it claims to use a lean architecture and introduces \"SwiGLU activation function\", \"grouped-query attention\" mechanism, which can be used to improve the performance of smartphones in a more efficient way, and to improve the performance of smartphones. The model is claimed to adopt a streamlined architecture and introduce \"SwiGLU activation function\" and \"grouped-query attention\" mechanism, which can balance efficiency and performance results. In addition, the MobileLLM model is said to be faster to train, with Meta researchers claiming that<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[167,297,219],"collection":[],"class_list":["post-22783","post","type-post","status-publish","format-standard","hentry","category-news","tag-ai","tag-meta","tag-219"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/22783","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=22783"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/22783\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=22783"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=22783"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=22783"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=22783"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}