{"id":30909,"date":"2025-03-17T14:24:12","date_gmt":"2025-03-17T06:24:12","guid":{"rendered":"https:\/\/www.1ai.net\/?p=30909"},"modified":"2025-03-17T14:24:25","modified_gmt":"2025-03-17T06:24:25","slug":"%e5%8f%97-deepseek-r1-%e5%90%af%e5%8f%91%ef%bc%8c%e5%b0%8f%e7%b1%b3%e5%a4%a7%e6%a8%a1%e5%9e%8b%e5%9b%a2%e9%98%9f%e7%99%bb%e9%a1%b6%e9%9f%b3%e9%a2%91%e6%8e%a8%e7%90%86-mmau-%e6%a6%9c","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/30909.html","title":{"rendered":"Inspired by DeepSeek-R1, Xiaomi's Big Modeling Team Tops Audio Inference MMAU Chart"},"content":{"rendered":"

March 17, @Millet<\/a>Technology official microblogging today said that XiaomiLarge Model<\/a>Team inAudio Reasoning<\/a>The team has made breakthrough progress in the field of multimodal audio understanding. Inspired by DeepSeek-R1, the team is the first to apply reinforcement learning algorithms to multimodal audio comprehension tasks, and in just one week, the team has topped the international authoritative MMAU audio comprehension review with a SOTA accuracy of 64.5%, which is now open-sourced.<\/p>\n

\"3265c835j00st98fa001od000ho00d5p\"<\/p>\n

1AI attached the full official text below:<\/p>\n

Reinforcement Learning Demonstrates \"Counterintuitive\" Advantage -- Xiaomi's Big Model Team Tops Audio Reasoning MMAU Chart<\/p>\n

Faced with a recording of a car's cockpit in motion, can AI determine if there is a potential malfunction in the car? At a symphony performance, can AI predict the mood of the composer when creating the music? In the chaos of footsteps at a subway station during the morning rush hour, can AI predict the risk of a potential collision at the gate? In the era of big models, people are no longer satisfied with machines just recognizing the content of speech and the type of voice, but expect them to have the ability of complex reasoning.<\/p>\n

The MMAU (Massive Multi-Task Audio Understanding and Reasoning) review set (https:\/\/arxiv.org\/ abs \/ 2410.19168) is a quantitative scale of this audio reasoning ability, which is measured by 10,000 audio samples covering speech, ambient sound, and music, combined with 10,000 audio samples covering speech and ambient sound and music, combined with Q&A pairs labeled by human experts, to test the model's performance on 27 skills, such as cross-scene reasoning, expertise, and other applications, with the expectation that the model achieves a level of logical analysis close to that of a human expert.<\/p>\n

As a benchmark ceiling, human experts on MMAU achieved an accuracy of 82.231 TP3T. This is a difficult set of reviews, and the best performing model on the current MMAU official list is GPT-4o from OpenAI, with an accuracy of 57.31 TP3T. A close second is Gemini 2.0 Flash from Google DeepMind, with an accuracy of 55.61 TP3T. Gemini 2.0 Flash from Google DeepMind, with an accuracy of 55.61 TP3T.<\/p>\n

The Qwen2-Audio-7B model from Ali has an accuracy of 49.21 TP3T on this review set. Due to its open-source nature, we attempted to fine-tune this model using a smaller dataset, the AVQA dataset released by Tsinghua University (https:\/\/mn.cs.tsinghua.edu.cn\/ avqa\/). The AVQA The AVQA dataset contains only 38,000 training samples, and with full supervised fine-tuning (SFT), the accuracy of the model on MMAU is improved to 51.8%. This is not a particularly significant improvement.<\/p>\n

The release of DeepSeek-R1 inspired our research on this task. DeepSeek-R1\u2019s Group Relative Policy Optimization (GRPO) approach allows models to evolve autonomously by simply \u201ctick-and-reward\u201d mechanisms, with human-like reasoning such as reflection, multi-step validation, etc. At the same time, a pre-print of the paper published by Carnegie Mellon University, \u201cAll Roads Lead to Likelihood: The Value of Renewal Learning in Fine-Tunning\u201d (https:\/\/arxiv.org\/abs\/25003.01067), drew an interesting conclusion from a delicate experiment: when the mission has a clear generation-validation Gap, that is, when the task's results are much more difficult than the validation of their validity, strengthening learning has a unique advantage over oversight fine-tuningAnd AQA tasks happen to be perfect for tasks with significant generation-validation gaps.<\/strong><\/p>\n

As an analogy, offline fine-tuning methods, such as SFT, are a bit like memorizing a question bank, where you can only train based on existing questions and answers, but may not be able to do the new questions you encounter; whereas reinforcement learning methods, such as GRPO, are like the teacher asking you to think of a few more answers, and then the teacher tells you which answer is good, so that you can actively think and stimulate your own ability, rather than being \" Fill-in-the-blank\" teaching. Of course, if the amount of training is sufficient, for example, some students are willing to spend many years to memorize the questions, may eventually achieve good results, but the efficiency is too low, wasting too much time. Active thinking, on the other hand, is more likely to achieve the effect of learning by example quickly. Reinforcement learning's real-time feedback may help the model target areas of the distribution of high-quality answers more quickly, whereas offline methods require traversing the entire possibility space, which is much less efficient.<\/p>\n

Based on the above insights.We try to migrate the GRPO algorithm of DeepSeek-R1 to the Qwen2-Audio-7B model<\/strong>. Surprisingly, with only 38,000 training samples from AVQA, theThe reinforcement learning fine-tuned model achieves an accuracy of 64.5% on the MMAU review set, a result that is nearly 10 percentage points better than GPT-4o, the commercial closed-source model that is currently number one on the list.<\/strong><\/p>\n

Interestingly, when we force the model to output the reasoning process during training (similar to the traditional chain-of-consciousness approach), the accuracy instead drops to 61.11 TP3T. this suggests that explicit chain-of-consciousness result output may not be conducive to model training.<\/p>\n

Our experiments reveal several conclusions that are different from traditional perceptions:<\/p>\n