{"id":12140,"date":"2024-06-04T09:31:59","date_gmt":"2024-06-04T01:31:59","guid":{"rendered":"https:\/\/www.1ai.net\/?p=12140"},"modified":"2024-06-04T09:31:59","modified_gmt":"2024-06-04T01:31:59","slug":"%e5%85%a8%e7%90%83%e9%a6%96%e5%88%9b%e5%8d%95%e5%8f%b0-rtx-4090-%e6%9c%8d%e5%8a%a1%e5%99%a8%e6%8e%a8%e7%90%86%ef%bc%8c%e6%98%86%e4%bb%91%e4%b8%87%e7%bb%b4%e5%bc%80%e6%ba%90-2","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/12140.html","title":{"rendered":"&quot;World&#039;s First&quot; Single RTX 4090 Server Inference, Kunlun Wanwei Open Source 200 Billion Sparse Large Model Tiangong MoE"},"content":{"rendered":"<p><a href=\"https:\/\/www.1ai.net\/en\/tag\/%e6%98%86%e4%bb%91%e4%b8%87%e7%bb%b4\" title=\"[Sees articles with [Konlen] tags]\" target=\"_blank\" >Kunlun Wanwei<\/a>Announced today<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%bc%80%e6%ba%90\" title=\"[View articles tagged with [open source]]\" target=\"_blank\" >Open Source<\/a> 2 Hundred Billion Sparse<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%a4%a7%e6%a8%a1%e5%9e%8b\" title=\"[View articles tagged with [large models]]\" target=\"_blank\" >Large Model<\/a> Skywork-MoE is an expansion of the intermediate checkpoint of the Skywork-13B model previously open sourced by Kunlun Wanwei. It is said to be the first open source trillion-dollar MoE large model that fully applies and implements the MoE Upcycling technology. It is also the first open source trillion-dollar MoE large model that supports reasoning using a single RTX 4090 server (8 RTX 4090 graphics cards).<\/p>\n<p>According to reports, the open source Skywork-MoE model belongs to the Tiangong 3.0 R&amp;D model series and is a medium-sized model (Skywork-MoE-Medium). The total number of parameters in the model is 146B, the number of activated parameters is 22B, there are 16 experts in total, each expert is 13B in size, and 2 experts are activated each time.<\/p>\n<p>Tiangong 3.0 also trained two MoE models, 75B (Skywork-MoE-Small) and 400B (Skywork-MoE-Large), which are not included in this open source release.<\/p>\n<p>According to official tests, with the same activation parameter size of 20B (inference calculation amount), Skywork-MoE&#039;s capability is close to that of a 70B Dense model, which reduces the model&#039;s inference cost by nearly 3 times. At the same time, the total parameter size of Skywork-MoE is 1\/3 smaller than that of DeepSeekV2, achieving similar capabilities with a smaller parameter size.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-12141\" title=\"d7abed8c-9c9c-4f93-9d25-9d67b94e0d7a\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/06\/d7abed8c-9c9c-4f93-9d25-9d67b94e0d7a.png\" alt=\"d7abed8c-9c9c-4f93-9d25-9d67b94e0d7a\" width=\"1080\" height=\"491\" \/><\/p>\n<p>Skywork-MoE&#039;s model weights and technical reports are completely open source and free for commercial use without application. The links are as follows:<\/p>\n<p>Model weights download:<\/p>\n<p>https:\/\/huggingface.co\/Skywork\/Skywork-MoE-base<\/p>\n<p>https:\/\/huggingface.co\/Skywork\/Skywork-MoE-Base-FP8<\/p>\n<p>Model open source repository: https:\/\/github.com\/SkyworkAI\/Skywork-MoE<\/p>\n<p>Model technical report: https:\/\/github.com\/SkyworkAI\/Skywork-MoE\/blob\/main\/skywork-moe-tech-report.pdf<\/p>\n<p>Model inference code: (supports 8 bit quantized load inference on 8 x 4090 servers) https:\/\/github.com\/SkyworkAI\/vllm<\/p>","protected":false},"excerpt":{"rendered":"<p>Kunlun World Wide today announced the open source 200 billion sparse model Skywork-MoE, based on the previous Kunlun World Wide's open source Skywork-13B model middle checkpoint extension, said to be the first complete MoE Upcycling technology application and landing of the open source 100 billion MoE model, but also the first to support the use of a single RTX 4090 server (8 RTX 4090 graphics cards) reasoning. It is also the first open source MoE model that supports reasoning with a single RTX 4090 server (8 RTX 4090 graphics cards). According to the introduction, the open source Skywork-MoE model belongs to the R&amp;D model series of Skywork 3.0, which is a medium-sized model (Skywork-MoE-Medium), with the total number of parameters of the model as follows<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[216,219,1050],"collection":[],"class_list":["post-12140","post","type-post","status-publish","format-standard","hentry","category-news","tag-216","tag-219","tag-1050"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/12140","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=12140"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/12140\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=12140"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=12140"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=12140"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=12140"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}