Multimodality

On September 26th, the LongCat API Open Platform was launched on September 25th with a big new generation model LongCat-2.5-Preview, which plays "long-range missions" with multi-modular capabilities and opens the API and web-end experience portal simultaneously. 1AI Attach the following core features of the official update log: LongCat-2.5-Preview: Multi-modular Understanding: Adding Photo Understanding Capable of Parsing Image Content, Supporting Cross-modular Questions and Answers, Content Abstracts and Complex Visual Logs. Excellent..

On September 2nd, the "AI godmother" Lee Feife's founder, World Labs, released today the "first multi-modular world model of the world" -- Atlas, claiming that the model could generate images and video frames with pixel-level and accurate camera control and rebuild it in 3D. World Labs official presentation, where Atlas is pre-trained from scratch so that it can receive multi-modular input, including camera moves, and turn it into a 3D view. Atlas allows to use..

On June 16, MiniMax announced yesterday that the model weight of MiniMax M3 has been officially launched under the open-source flag, and that the MSA (MiniMax Sparse Act) technical paper has been published simultaneously. Mini Max M3 Total Parameter 428B, Activating Parameter 23B, officially positioned as the first open-source model to introduce multi-modular hybrid training from Step 0. The training phase was heavily integrated with text, images and multi-modular staggered data for pre-training levels..

On September 4, Shanghai Artificial Intelligence Laboratory (Shanghai AI Lab) announced the open source of general multimodal large model Shusheng Wanxiang 3.5 (InternVL3.5), which upgraded its reasoning ability, deployment efficiency and generalization ability. InternVL3.5 has nine sizes of models with parameters ranging from 1 billion to 241 billion, which can meet the needs of various scenarios. Among them, the flagship model InternVL3.5-241B-A28B scored 77.7 points in the multidisciplinary reasoning benchmark MMMU, which is the most ...

May 17, 2011 - Tech media outlet WinBuzzer published a blog post yesterday (May 16) reporting that Ollama, an open source large language modeling service, has launched a self-developed multimodal AI customization engine, moving away from its direct reliance on the llama.cpp framework. The llama.cpp project recently integrated full visual support through the libmtmd library, and Ollama's relationship with it has sparked community discussion. Ollama team members clarified on Hacker News that Ollama...

March 10 news, according to the people's financial news reported, in order to further accelerate the intelligent body to enable industrial applications, and promote the high quality development of multimodal intelligent body, the China Information and Communication Technology Institute officially launched the multimodal intelligent body technical specification preparation. In order to fully improve the content of the technical specifications and enhance industrial exchanges, the Artificial Intelligence Research Institute of the China Academy of Information and Communications Technology (AI) will convene a multimodal intelligent body technology salon and technical specifications seminar on March 13, inviting experts in the industry to discuss the dynamics of the development of multimodal intelligent body technology, application landing, and introducing the framework of the technical specifications of multimodal intelligent body. 1AI noted that in May last year, the...

February 27, 2011 - Microsoft released Phi-4 in December 2024, a small language model (SLM) that excels in its class. Today, Microsoft extends the Phi-4 family with two new models: Phi-4-multimodal and Phi-4-mini. The Phi-4 multimodal model is Microsoft's first unified architecture multimodal language model that integrates speech, vision, and text processing with 5.6 billion parameters. In multiple benchmarks...

February 26th news, Beijing time this morning, Microsoft in the official website of the open source multimodal AI Agent base model - Magma. compared with the traditional Agent, Magma has across the digital and physical world of multimodal capabilities, can automatically deal with different types of data, such as images, video, text, etc. In addition, Magma can also be built-in psychological prediction capabilities to enhance the understanding of the spatial and temporal dynamics of future video frames, can accurately infer the intentions and future behavior of characters or objects in the video. In addition, Magma has built-in psychological prediction capabilities that enhance its ability to understand the spatial and temporal dynamics of future video frames, allowing it to accurately predict the intentions and future behaviors of people or objects in the video. Users can use Magma to automatically place e-commerce orders, check sky...

January 23 news, Beijing Zhi Spectrum Huazhang Technology Co., Ltd. today announced that its Zhi Spectrum GLM-PC open experience, claiming that "autonomous operation of the computer's multimodal Agent upgraded". According to the introduction, GLM-PC is the world's first public-oriented, back to the computer intelligence body (agent) based on CogAgent, a large multimodal model of the Smart Spectrum. GLM-PC v1.0 will be released on November 29, 2024 and open to the public...

Dark Side of the Moon announced on January 20 the launch of Kimi's new SOTA model -- k1.5 multimodal thinking model, which realizes SOTA (state-of-the-art) level multimodal reasoning and generalized reasoning capabilities. Officially, in short-CoT mode, Kimi k1.5's mathematical, coding, and visual multimodal and generalized capabilities dramatically outperform those of the global short-thinking SOTA models GPT-4o and Claude 3.5 Sonnet, leading by as much as 5...

January 15, 2011 - The Dark Side of the Moon today released the Kimi Multimodal Image Understanding Model API, a new multimodal image understanding model, moonshot-v1-vision-preview (hereinafter referred to as the "Vision model"), which completes the moonshot-v1 model series' multimodal capabilities. model series. Model Capability Description Image Recognition The Vision model has the ability to recognize complex details and nuances in an image, whether it is a food or an animal, and to distinguish between objects that are similar but not identical. ...

January 15, 2011 - A research team from Stanford Medical School has developed an AI model called MUSK, which combines medical images and text data to accurately predict the prognosis and treatment response of cancer patients. Note: Prognosis is a medical term that refers to the estimation of the possible outcome after treatment based on the patient's current condition, combined with the understanding of the disease, such as clinical manifestations, laboratory results, imaging tests, etiology, pathology, and pattern of the disease, as well as the timing of the treatment, the method, and the new situation that emerges during the process. The highlight of the MUSK model is the breakthrough...

Google released Gemini 2.0 yesterday along with a new Multimodal Live API to help developers build apps with real-time audio and video streaming capabilities. The API enables low-latency, bi-directional text, audio and video interactions with audio and text output for a more natural, smooth, human-like interactive experience. Users can interrupt the model at any time and interact with it by sharing camera input or screen recordings to ask questions about the content. The model's video comprehension feature extends the communication model...

In a blog post today (October 21), Samsung announced the launch of its second-generation generative AI model, Samsung Gauss2, at a developer conference in South Korea. The multimodal language model is capable of processing multiple data types such as text, code, and images simultaneously, providing a significant boost in performance and efficiency. Gauss2 is available in three different sizes - Compact, Balanced and Supreme - to meet the needs of different computational requirements. Gauss2 offers three different models with different specifications to meet the needs of different computing environments and application scenarios, which are briefly summarized by IT Home as follows...

Nov. 19 - Mistral AI announced yesterday, Nov. 18, a new multimodal AI model, Pixtral Large, with 124 billion parameters, based on Mistral Large 2, and designed primarily for processing text and images. Pixtral Large is now available under the Mistral Research License and Commercial License for research, education, and commercial use. Pixtral Large is a Mistral ...

Meta is about to make something big again! They are about to release an open source language model called Llama3405B, which is not only their largest model to date, but also the largest open source language model in history. This behemoth, with a staggering 405 billion parameters, can shuttle between images and text, completely subverting the old calendar that can only process text. Key points: Meta will release Llama3405B, a multimodal model with 405 billion parameters, on July 23. The decision to open source Llama3405B and its weights may completely…

Google recently published a blog post on its GitHub page, introducing the VLOGGER AI model. Users only need to input a portrait photo and an audio content, and the model can make these characters "animate" and read the audio content with facial expressions. VLOGGER AI is a multimodal diffusion model suitable for virtual portraits. It is trained using the MENTOR database, which contains more than 800,000 portraits and more than 2,200 hours of videos, allowing VLOGGER to generate different...

The recent Gemini-Pro evaluation report shows that it has made significant progress in the multimodal field, comparable to GPT-4V, and even better in some aspects. First, in the comprehensive performance on the multimodal proprietary benchmark MME, Gemini-Pro surpassed GPT-4V with a high score of 1933.4, showing its comprehensive advantages in perception and cognition. Among the 37 visual understanding tasks, Gemini-Pro performed outstandingly in tasks such as text translation, color/landmark/person recognition, and OCR, showing its excellent ability in the basic perception field. …

Gemini is a new generation of artificial intelligence system launched by Google DeepMind. It is capable of multimodal reasoning and supports seamless interaction between text, images, video, audio, and code.Gemini has surpassed its previous state in many areas such as language comprehension, reasoning, math, and programming, making it one of the most powerful AI systems to date. Gemini is available in three different sizes to meet a wide range of needs, from edge computing to the cloud, and can be used in a wide range of applications, including creative design, writing assistance, question answering, code generation, and more. Gemini Features Writing Assistant: Ge...