{"id":24035,"date":"2024-11-29T00:23:22","date_gmt":"2024-11-28T16:23:22","guid":{"rendered":"https:\/\/www.1ai.net\/?p=24035"},"modified":"2024-11-28T21:25:07","modified_gmt":"2024-11-28T13:25:07","slug":"%e6%9c%88%e4%b9%8b%e6%9a%97%e9%9d%a2-kimi-%e8%81%94%e5%90%88%e6%b8%85%e5%8d%8e%e5%a4%a7%e5%ad%a6%e7%ad%89%e5%bc%80%e6%ba%90%e5%a4%a7%e6%a8%a1%e5%9e%8b%e6%8e%a8%e7%90%86%e6%9e%b6%e6%9e%84-mooncake","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/24035.html","title":{"rendered":"Dark Side of the Moon Kimi Open Source Big Model Reasoning Architecture Mooncake with Tsinghua University and others"},"content":{"rendered":"<p><a href=\"https:\/\/www.1ai.net\/en\/tag\/%e6%9c%88%e4%b9%8b%e6%9a%97%e9%9d%a2\" title=\"[Sees articles with labels]\" target=\"_blank\" >Dark Side of the Moon<\/a> <a href=\"https:\/\/www.1ai.net\/en\/tag\/kimi\" title=\"[View articles tagged with [Kimi]]\" target=\"_blank\" >Kimi<\/a> and<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e6%b8%85%e5%8d%8e%e5%a4%a7%e5%ad%a6\" title=\"[Sees articles with tags]\" target=\"_blank\" >Tsinghua University<\/a> MADSys Labs 2024 co-published a design for the Mooncake inference system underlying Kimi. The system is based on a KVCache-centered PD separation and store-for-store conversion architecture.<strong>Improved inference throughput<\/strong>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-24036\" title=\"ac1615c8j00snnx8a000gd000u000b4p\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/11\/ac1615c8j00snnx8a000gd000u000b4p.jpg\" alt=\"ac1615c8j00snnx8a000gd000u000b4p\" width=\"1080\" height=\"400\" \/><\/p>\n<p>Recently, in order to further accelerate the application and promotion of this technology framework, Kimi of the Dark Side of the Moon and MADSys Lab of Tsinghua University have joined hands with 9#AISoft, AliCloud, Huawei Storage, Noodle Intelligence, and Tendency Technology.<strong>Co-launch of the open source project Mooncake<\/strong>The KVCache-centered<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%a4%a7%e6%a8%a1%e5%9e%8b\" title=\"[View articles tagged with [large models]]\" target=\"_blank\" >Large Model<\/a>Reasoning Architecture.<\/p>\n<p>November 28, Mooncake technology framework has been open source online, 1AI attached address is as follows:<\/p>\n<p><a href=\"https:\/\/github.com\/kvcache-ai\/Mooncake\">https:\/\/github.com\/kvcache-ai\/Mooncake<\/a><\/p>\n<p>According to the introduction, Mooncake open source project extends from the paper, centered on the ultra-large-scale KVCache cache pool, and improves the inference throughput by drastically reducing the arithmetic overhead through the innovative concept of storage-for-computation.<\/p>\n<p><strong>This open source will use a phased approach<\/strong>KVCache is an open source implementation of Mooncake Store, a high-performance KVCache multi-level cache, compatible with various inference engines and underlying storage\/transfer resources. The Transfer Engine part of the Transfer Engine is now open-sourced globally on GitHub.<\/p>\n<p>The ultimate goal of the Mooncake open source project is to create a standard interface for a new type of high-performance in-memory semantic storage for the era of big models, and to provide a reference implementation.<\/p>","protected":false},"excerpt":{"rendered":"<p>Dark Side of the Moon Kimi and MADSys Lab at Tsinghua University 2024 jointly released a design for the Mooncake inference system underlying Kimi. The system is based on the KVCache-centered PD separation and store-for-store architecture, which improves the inference throughput. Recently, in order to further accelerate the application and promotion of this technical framework, Kimi of the dark side of the moon and MADSys Laboratory of Tsinghua University, together with 9#AISoft, Ali Cloud, Huawei Storage, Facade Intelligence, and Convergence Technology, jointly released the open source project Mooncake, to build a large model inference architecture centered on KVCache. On November 28th, the Mooncake technical framework has been opened.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[1814,216,1168,1712],"collection":[],"class_list":["post-24035","post","type-post","status-publish","format-standard","hentry","category-news","tag-kimi","tag-216","tag-1168","tag-1712"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/24035","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=24035"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/24035\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=24035"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=24035"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=24035"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=24035"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}