{"id":34044,"date":"2025-04-25T14:50:17","date_gmt":"2025-04-25T06:50:17","guid":{"rendered":"https:\/\/www.1ai.net\/?p=34044"},"modified":"2025-04-25T14:50:17","modified_gmt":"2025-04-25T06:50:17","slug":"meta-%e6%8e%a8-webssl-%e6%a8%a1%e5%9e%8b%ef%bc%9a%e6%8e%a2%e7%b4%a2-ai-%e6%97%a0%e8%af%ad%e8%a8%80%e8%a7%86%e8%a7%89%e5%ad%a6%e4%b9%a0%ef%bc%8c%e7%ba%af%e5%9b%be%e8%ae%ad%e7%bb%83%e5%aa%b2%e7%be%8e-op","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/34044.html","title":{"rendered":"Meta Launches WebSSL Models: Exploring AI Languageless Visual Learning, Pure Graph Training Comparable to OpenAI CLIP"},"content":{"rendered":"

April 25, 2011 - Technology media outlet marktechpost published a blog post yesterday (April 24) reporting that Meta<\/a> Company Releases WebSSL<\/a> series of models with parameter sizes ranging from 300 million to 7 billion, trained on pure image data.aims to explore the potential of visual self-supervised learning (SSL) without verbal supervision.<\/strong><\/p>\n

\"6f736d4cj00sv9hmc0030d000sg00l8p\"<\/p>\n

OpenAI<\/a> represented by CLIP, the contrastive language-image model has become the default choice for learning visual representations, with outstanding performance in multimodal tasks such as visual question and answer (VQA) and document understanding. However, limited by the complexity of dataset acquisition and data size, language dependency faces many challenges.<\/p>\n

Meta has addressed these pain points by releasing a family of WebSSL models on the Hugging Face platform, covering DINO and Vision Transformer (ViT) architectures, with parameter sizes ranging from 300 million to 7 billion.<\/p>\n

These models were trained using only a subset of 2 billion images from the MetaCLIP dataset (MC-2B), excluding the influence of linguistic supervision.Meta's goal is not to replace CLIP, but rather to provide an in-depth evaluation of the performance potential of purely visual Self-Supervised Learning (SSL) without the constraints of data and model sizes by controlling for variables.<\/p>\n

The WebSSL model uses two visual self-supervised learning paradigms: joint embedding learning (DINOv2) and mask modeling (MAE). Training is performed uniformly using 224\u00d7224 resolution images, and the visual coder is frozen to ensure that differences in results stem only from the pre-training strategy.<\/p>\n

The model is trained on five capacity tiers (ViT-1B to ViT-7B) and evaluated based on the Cambrian-1 benchmark, covering 16 VQA tasks such as general visual understanding, knowledge reasoning, OCR, and diagram interpretation. In addition, the model is seamlessly integrated into Hugging Face's transformers library for easy research and application.<\/p>\n

The experiments reveal several key findings: the WebSSL model's performance on the VQA task improves close to log-linearly as the parameter size increases, while the CLIP saturates in performance after exceeding 3 billion parameters.<\/p>\n

WebSSL outperforms CLIP in the OCR and Chart tasks, especially after data filtering, with only 1.3% of rich text image training, and boosts up to 13.6% in the OCRBench and ChartQA tasks.<\/p>\n

In addition, high-resolution (518px) fine-tuning further closes the gap with high-resolution models such as SigLIP, which performs particularly well in document tasks.<\/p>\n

WebSSL models show good alignment with pre-trained language models (e.g., LLaMA-3) even without linguistic supervision, indicating that large-scale visual models can implicitly learn features related to textual semantics.<\/p>\n

At the same time, WebSSL maintains strong performance on traditional benchmarks (e.g., ImageNet-1k classification, ADE20K segmentation) and even outperforms MetaCLIP and DINOv2 in some scenarios.<\/p>\n

1AI Attach reference address<\/span><\/p>\n