June 27th.TencentThe Hybrid Grand Model family announced today the arrival of a new member -- the The Hybrid-A13B model was released andOpen SourceThe first MoE open-source software to be released at the 13B level, it is claimed to be "the industry's first 13B level MoE open-source software".mixed inference model”.
Hybrid-A13B As a large model based on the Mixing of Experts (MoE) architecture.Total parameters 80 billion, activation parameters 13 billionThe company claims that it "dramatically reduces inference latency and computational overhead while delivering results comparable to top open source models".
This is undoubtedly good news for individual developers and small and medium-sized enterprises (SMEs), said Tencent Mixed Elements.Deployable with only 1 low to mid-range GPU card in extreme conditionsUsers can download and use the model API from Github, HuggingFace and other technical communities. Users can download and use it from Github, HuggingFace, and other technical communities, and the model API is available on the Tencent Cloud website.
Hybrid-A13B model passed MoE ArchitectureThe model, which selectively activates relevant model components for each input, is claimed to be "fast and economical" compared to dense models of the same size, and provides a "scalable and efficient alternative" for individual developers and SMEs.
In the pre-training, the model uses a 20 trillion high-quality web lexical meta-corpus, which raises the upper limit of the model's inference ability; the Scaling Law (i.e., the law of scaling) theoretical system of the MoE architecture is improved, which provides quantifiable engineering guidance for the design of the MoE architecture, and enhances the pre-training effect of the model.
Users can choose their thinking mode on demand, with the fast thinking mode providing concise, efficient outputs suitable for simple tasks where speed and minimal computational overhead are sought; the slow thinking mode involves deeper, more comprehensive reasoning steps. This optimizes the allocation of computational resources, balancing efficiency and accuracy.
Hybrid has also open-sourced two new datasets to fill the gaps in the industry's relevant assessment standardsThe ArtifactsBench is mainly used for code evaluation. Among them, ArtifactsBench is mainly used for code evaluation, constructing a new benchmark with 1825 tasks; C3-Bench is designed for Agent scenario model evaluation, with 1024 test data to find out the shortcomings of model capability.
In terms of concrete results, in terms of mathematical reasoning, for example, by inputting "Who is bigger, 9.11 or 9.9?", the model can accurately complete decimal comparisons and demonstrate step-by-step parsing capabilities.
For the popular intelligent body (Agent) application, the model can call tools to generate complex command responses such as travel tips and data file analysis.
Let's look at the data and results. The model demonstrated "leading results" on math, science, and logical reasoning tasks on multiple public data test sets.
1AI Attached open source address:
https://github.com/Tencent-Hunyuan/Hunyuan-A13B