{"id":15184,"date":"2024-07-09T10:13:05","date_gmt":"2024-07-09T02:13:05","guid":{"rendered":"https:\/\/www.1ai.net\/?p=15184"},"modified":"2024-07-09T10:13:05","modified_gmt":"2024-07-09T02:13:05","slug":"meta-ai%e4%b8%ba%e7%a7%bb%e5%8a%a8%e8%ae%be%e5%a4%87%e5%bc%80%e5%8f%91%e7%b4%a7%e5%87%91%e5%9e%8b%e8%af%ad%e8%a8%80%e6%a8%a1%e5%9e%8bmobilellm%ef%bc%8c%e4%bb%853-5%e4%ba%bf%e5%8f%82%e6%95%b0","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/15184.html","title":{"rendered":"Meta AI develops a compact language model MobileLLM for mobile devices with only 350 million parameters"},"content":{"rendered":"

Meta<\/a> AI researchers have introduced MobileLLM<\/a>, a language designed for high-performance computing on smartphones and other resource-constrained devicesModel<\/a>The study, published on June 27, 2024, challenges the prevailing view that effective AI Models<\/a>Assumptions of necessary size.<\/p>\n

The research team, which includes members of Meta Reality Labs, PyTorch, and Meta AI Research (FAIR), focused on optimizing models with fewer than a billion parameters, a fraction of the parameters of models like GPT-4, which are estimated to have more than a trillion.<\/p>\n

The main innovations of MobileLLM include:<\/p>\n

    \n
  1. Prioritize model depth over width<\/li>\n
  2. Implementing embedded sharing and grouping query notes<\/li>\n
  3. Utilizes a novel direct block weight sharing technique<\/li>\n<\/ol>\n

    These design choices allow MobileLLM to outperform previous models of similar size by 2.7% to 4.3% on common benchmark tasks. While these single-digit improvements may seem small, they represent significant progress in the highly competitive field of language model development.<\/p>\n

    Notably, on certain API call tasks, the 350 million parameter version of MobileLLM demonstrated comparable accuracy to the larger 7 billion parameter LLaMA-2 model, suggesting that for certain specific applications, more compact models may provide similar functionality while using less computational resources.<\/p>\n

    \"\"<\/p>\n

    The development of MobileLLM coincides with growing interest in more efficient AI models. As progress on very large language models shows signs of slowing, researchers are increasingly exploring the potential of more compact, specialized designs. Despite the \u201cLLM\u201d in the name, the focus on efficiency and on-device deployment puts MobileLLM in the same category as what some researchers call small language models (SLMs).<\/p>\n

    While MobileLLM is not yet available to the public, Meta has open-sourced the pre-trained code, allowing other researchers to build on its work. As this technology develops, it could bring more advanced AI capabilities to personal devices, although the timeline and specific features remain uncertain.<\/p>","protected":false},"excerpt":{"rendered":"

    Meta AI researchers have introduced MobileLLM, a new approach to designing efficient language models for smartphones and other resource-constrained devices. The study, published on June 27, 2024, challenges assumptions about the necessary scale of effective AI models. The research team, comprised of members of Meta Reality Labs, PyTorch, and Meta AI Research (FAIR), focused on optimizing models with fewer than a billion parameters. This is only a small fraction of models such as GPT-4, which is estimated to have over a trillion parameters. Key innovations of MobileLLM include: Prioritizing model depth over width Enabling embedding sharing and grouping<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[167,297,3413,1489],"collection":[],"class_list":["post-15184","post","type-post","status-publish","format-standard","hentry","category-news","tag-ai","tag-meta","tag-mobilellm","tag-1489"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/15184","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=15184"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/15184\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=15184"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=15184"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=15184"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=15184"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}