{"id":15469,"date":"2024-07-13T12:09:53","date_gmt":"2024-07-13T04:09:53","guid":{"rendered":"https:\/\/www.1ai.net\/?p=15469"},"modified":"2024-07-13T12:09:53","modified_gmt":"2024-07-13T04:09:53","slug":"%e6%99%ba%e8%b0%b1-ai-%e5%bc%80%e6%ba%90%e8%a7%86%e9%a2%91%e7%90%86%e8%a7%a3%e6%a8%a1%e5%9e%8b-cogvlm2-video%ef%bc%8c%e5%8f%af%e5%9b%9e%e7%ad%94%e6%97%b6%e9%97%b4%e7%9b%b8%e5%85%b3%e9%97%ae%e9%a2%98","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/15469.html","title":{"rendered":"Zhipu AI open-sources video understanding model CogVLM2-Video, which can answer time-related questions"},"content":{"rendered":"

Zhipu AI<\/a> Announced that a new video understanding model has been trained CogVLM2-Video<\/a>, andOpen Source<\/a>.<\/p>\n

It is reported that most current video understanding models use frame averaging and video tag compression methods, which results in the loss of temporal information and cannot accurately answer time-related questions. Some models that focus on temporal question-answering datasets are too limited to specific formats and applicable fields, causing the models to lose broader question-answering capabilities.<\/p>\n

\"3b1da7b1-3a10-408d-929e-2a13414106da\"<\/p>\n

\u25b2 Official effect demonstration<\/p>\n

Zhipu AI proposed aAutomatic time positioning data construction method based on visual model<\/strong>, generating 30,000 time-related video question-answering data. Based on this new dataset and existing open-domain question-answering data, we introduced multi-frame video images and timestamps as encoder inputs and trained the CogVLM2-Video model.<\/p>\n

Zhipu AI said that CogVLM2-Video not only achieved state-of-the-art performance on public video understanding benchmarks, but also excelled in video subtitle generation and temporal positioning.<\/p>\n

Attached related links:<\/p>\n