{"id":3809,"date":"2024-02-09T12:03:53","date_gmt":"2024-02-09T04:03:53","guid":{"rendered":"https:\/\/www.1ai.net\/?p=3809"},"modified":"2024-02-09T12:03:53","modified_gmt":"2024-02-09T04:03:53","slug":"%e8%8b%b9%e6%9e%9c%e5%b1%95%e7%a4%ba-ai-%e6%96%b0%e6%a8%a1%e5%9e%8b-mgie%ef%bc%8c%e5%8f%af%e4%b8%80%e5%8f%a5%e8%af%9d%e7%b2%be%e4%bf%ae%e5%9b%be%e7%89%87","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/3809.html","title":{"rendered":"Apple shows off new AI model MGIE, which can retouch photos with just one sentence"},"content":{"rendered":"

Compared with Microsoft's booming business,Apple<\/a>Apple's layout in the field of AI seems to be much more low-key, but this does not mean that Apple has no achievements in this field.Apple recently released a newMGIE<\/a>\u201d is a new open source AI model that can edit images based on natural language instructions.<\/strong><\/p>\n

\"fc3ac60b-3b50-4723-81e7-6d43bc2df108\"<\/p>\n

Image source: VentureBeat and Midjourney<\/span><\/p>\n

The full name of MGIE is MLLM<\/a>-Guided Image Editing, usingMultimodal Large Language Models<\/a>(MLLM) interprets user instructions and performs pixel-level operations. MGIE can understand natural language commands issued by users and perform operations such as Photoshop-style modification, global photo optimization, and local editing.<\/p>\n

Apple and researchers from the University of California, Santa Barbara are collaborating to present MGIE-related research results at the 2024 International Conference on Learning Representations (ICLR), one of the top conferences for artificial intelligence research.<\/p>\n

Before introducing MGIE, let's first introduce MLLM. MLLM is a powerful AI model that can process text and images simultaneously to enhance instruction-based image editing capabilities. MLLMs have shown excellent capabilities in cross-modal understanding and visual perceptual response generation, but have not yet been widely used in image editing tasks.<\/p>\n

MGIE integrates MLLMs into the image editing process in two ways: First, it uses MLLMs to derive expressive instructions from user inputs. These instructions are concise and provide clear guidance for the editing process.<\/p>\n

For example, when you enter "Make the sky bluer<\/strong>\u201d, MGIE can generate \u201cIncreased the saturation of the sky area by 20%<\/strong>\u201d instruction.<\/p>\n

Second, it uses MLLM to generate visual imagination, a latent representation of the desired edits. This representation captures the essence of the edits and can be used to guide pixel-level operations. MGIE adopts a novel end-to-end training scheme that jointly optimizes instruction derivation, visual imagination, and image editing modules.<\/p>\n

MGIE can handle a wide range of editing situations, from simple color adjustments to complex object manipulations. The model can also perform global and local editing based on the user's preferences. Some of the features and capabilities of MGIE include:<\/p>\n