{"id":16485,"date":"2024-07-26T09:36:32","date_gmt":"2024-07-26T01:36:32","guid":{"rendered":"https:\/\/www.1ai.net\/?p=16485"},"modified":"2024-07-26T09:36:32","modified_gmt":"2024-07-26T01:36:32","slug":"%e6%ad%a6%e6%b1%89%e5%a4%a7%e5%ad%a6%e8%81%94%e5%90%88%e4%b8%ad%e5%9b%bd%e7%a7%bb%e5%8a%a8%e4%b9%9d%e5%a4%a9%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd%e5%9b%a2%e9%98%9f%e5%bc%80%e6%ba%90%e9%9f%b3%e8%a7%86","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/16485.html","title":{"rendered":"Wuhan University and China Mobile's Jiutian AI team jointly open-sourced the audio and video speaker recognition dataset VoxBlink2"},"content":{"rendered":"

Wuhan University<\/a>jointChina Mobile<\/a>The Jiutian AI team and Duke Kunshan University used YouTube data toOpen Source<\/a>More than 110,000 hours of audio and video speaker recognitionDataset<\/a>VoxBlink2<\/a>The dataset contains 9,904,382 high-quality audio clips and their corresponding video clips from 111,284 users on YouTube. It is currently the largest publicly available audio and video speaker recognition dataset. The release of the dataset aims to enrich the open source speech corpus and support the training of large voiceprint models.<\/p>\n

\"get-823\"<\/div>\n

The VoxBlink2 dataset is mined through the following steps:<\/p>\n

\n

Candidate preparation: Collect multilingual keyword lists, retrieve user videos, and select the first minute of video for processing.<\/p>\n

Face Extraction & Detection: Extract video frames at a high frame rate and use MobileNet to detect faces, ensuring that the video track contains only a single speaker.<\/p>\n

Face recognition: Pre-trained face recognizer recognizes each frame to ensure that the audio and video clips are from the same person.<\/p>\n

Active speaker detection: Using lip movement sequences and audio, a multimodal active speaker detector outputs the utterance segment, and aliasing detection removes multi-speaker segments.<\/p>\n<\/blockquote>\n

In order to improve the data accuracy, a bypass step of the in-set face recognizer was also introduced. Through rough face extraction, face verification, face sampling and training, the accuracy was improved from 72% to 92%.<\/p>\n

VoxBlink2 also open-sources voiceprint models of different sizes, including a 2D convolutional model based on ResNet, a time series model based on ECAPA-TDNN, and a super-large model ResNet293 based on Simple Attention Module. After post-processing on the Vox1-O dataset, these models can achieve an EER of 0.17% and a minDCF of 0.006%.<\/p>\n

Dataset website:<\/strong>https:\/\/VoxBlink2.github.io<\/p>\n

How to download the dataset:<\/strong>https:\/\/github.com\/VoxBlink2\/ScriptsForVoxBlink2<\/p>\n

Meta files and models:<\/strong>https:\/\/drive.google.com\/drive\/folders\/1lzumPsnl5yEaMP9g2bFbSKINLZ-QRJVP<\/p>\n

Paper address:<\/strong>https:\/\/arxiv.org\/abs\/2407.11510<\/p>\n

 <\/p>","protected":false},"excerpt":{"rendered":"

Wuhan University, together with China Mobile's JiuTian AI team and Duke Kunshan University, have open-sourced over 110,000 hours of audio\/video speaker recognition dataset VoxBlink2 based on YouTube data.The dataset consists of 9,904,382 high-quality audio clips and their corresponding video clips from 112,284 YouTube users, and it is the largest publicly available speaker recognition dataset in the world. It is the largest publicly available audio and video speaker recognition dataset. The release of the dataset is intended to enrich the open source speech corpus and support the training of large models of voiceprints. The VoxBlink2 dataset is mined by the following steps: Candidate Preparation: Collect multilingual keyword lists, retrieve user videos, and select the first minute of video for processing. Face Extraction & Detection:Extract video frames at high frame rate and use M<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[3713,2072,219,3355,3712],"collection":[],"class_list":["post-16485","post","type-post","status-publish","format-standard","hentry","category-news","tag-voxblink2","tag-2072","tag-219","tag-3355","tag-3712"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/16485","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=16485"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/16485\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=16485"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=16485"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=16485"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=16485"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}