{"id":18916,"date":"2024-08-30T09:28:43","date_gmt":"2024-08-30T01:28:43","guid":{"rendered":"https:\/\/www.1ai.net\/?p=18916"},"modified":"2024-08-30T09:28:43","modified_gmt":"2024-08-30T01:28:43","slug":"%e6%9c%80%e5%bc%ba%e7%ab%af%e4%be%a7%e5%bc%80%e6%ba%90-ai%e6%a8%a1%e5%9e%8b-zamba2-mini-%e7%99%bb%e5%9c%ba%ef%bc%9a12-%e4%ba%bf%e5%8f%82%e6%95%b0%ef%bc%8c4bit-%e9%87%8f%e5%8c%96%e4%b8%8b%e5%86%85","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/18916.html","title":{"rendered":"The most powerful open source AI model Zamba2-mini is released: 1.2 billion parameters, less than 700MB memory usage at 4-bit quantization"},"content":{"rendered":"

Zyphra published a blog post on August 27, announcing the release of the Zamba2-mini 1.2B model.It has a total of 1.2 billion parameters and is claimed to be an end-side SOTA small language model. Its memory usage is less than 700MB under 4-bit quantization.<\/strong><\/p>\n

SOTA stands for state-of-the-art. It does not refer to a specific model, but to the best\/most advanced model currently available in this research task.<\/p>\n

\"4924b6b8j00sj0c270023d0018g00sqm\"<\/p>\n

Although small in size, the Zamba2-mini 1.2B is comparable to larger models including Google's Gemma-2B, Huggingface's SmolLM-1.7B, Apple's OpenELM-1.1B, and Microsoft's Phi-1.5.<\/p>\n

In the reasoning task, the outstanding performance of Zamba2-mini is particularly remarkable. Compared with models such as Phi3-3.8B, the first token time of Zamba2-mini (the delay from input to output of the first token) is half of the previous one, and the memory usage is reduced by 27%.<\/p>\n

Zamba2-mini 1.2B is mainly achieved through a highly optimized architecture that combines the advantages of different neural network designs, which can not only maintain the high-quality output of large and dense transformers, but also run with the computational and memory efficiency of smaller models.<\/p>\n

One of the key advances of Zamba2-mini compared to its predecessor Zamba1 is the integration of two shared attention layers.<\/p>\n

This two-layer approach enhances the model\u2019s ability to preserve information at different depths, improving overall performance. Including rotational position embeddings in the shared attention layer also slightly improves performance, demonstrating that Zyphra is committed to making incremental yet impactful improvements in model design.<\/p>\n

Zamba2-mini is pre-trained on a massive dataset of three trillion tokens from Zyda and other public sources.<\/p>\n

\"23591b31j00sj0c27002zd0018g00ogm\"<\/p>\n

This massive dataset is rigorously filtered and iterated to ensure the highest quality training data, and is further refined in an annealing phase, which includes training on 100 billion very high quality tokens.<\/p>\n

Zyphra has committed to making Zamba2-mini available under the Apache 2.0 license.Open Source<\/a>Model.<\/p>\n

Attach reference address<\/p>\n