August 26, 2011 - Technology media outlet bleepingcomputer released a blog post yesterday (August 25) reporting that Trail of Bits researchers have developed a new type of AI Attack Maneuvers.Hiding malicious prompt words in high-resolution images and revealing them after automatic downsampling and processing by AI systems, and then executing them as commands by large language models, can steal user data.
The method, proposed by Kikimora Morozova and Suha Sabi Hussain of Trail of Bits, was inspired by the theory of image scaling attacks at the Technical University of Braunschweig, Germany, in 2020. Attackers first embed commands invisible to the naked eye in high-resolution images, and then utilize an AI system downsampling algorithm to make them visible.
1AI cites a blog post that describes how AI platforms usually automatically downscaled user-uploaded images to save performance and costs, using image resampling algorithms such as nearest neighbor, bilinear, and bicubic interpolation.
An attacker can design an image for a specific algorithm so that hidden blocks of color form recognizable text after downsampling. For example, in the case of Trail of Bits, darker areas of the image turn red and reveal black text after double or triple downsampling.
Once the text is visible, the AI model treats it as part of the user's input and executes it in conjunction with normal commands, which can lead to a data breach or other risky actions. The researchers exploited this vulnerability in the Gemini CLI environment in Zapier MCP "trust=True" mode to extract Google Calendar data and send it to a specified email address without user confirmation.
The approach has been tested and validated on multiple platforms, including Google Gemini CLI, Vertex AI Studio (Gemini backend), Gemini web and API interfaces, Google Assistant on Android phones, and Genspark.
The research team also released the open-source tool Anamorpher (in beta), which generates images of attacks against different downsampling methods, demonstrating that the range of potential threats far exceeds validated tools.