Recent findings from Trail of Bits highlight an unexpected vulnerability in many AI systems that process images. The core issue? When these systems downscale large images to save resources, attackers can craft images that hide malicious prompts or data, which only reveal themselves after the resizing—potentially leading to data leaks or system manipulation.
Understanding the Hidden Mechanics
In simple terms, attackers create images that look completely normal at full resolution but contain subtle modifications designed to activate once the image is scaled down. This works because image resizing algorithms—such as bilinear or bicubic interpolation—don’t simply shrink images; they mathematically blend neighboring pixel values to produce a smaller version. Attackers exploit these mathematical processes by embedding carefully engineered patterns within the high-resolution image. When the image is resized, these patterns can emerge as specific signals or prompts, effectively hiding malicious instructions in what appears to be a benign picture.
For example, an attacker might manipulate pixel values so that, after downscaling, the resulting image contains a prompt that triggers data exfiltration or commands. The resizing process, intended to improve performance, becomes a tool for embedding or revealing hidden signals. Because many AI systems don’t display or verify the scaled image before processing, these malicious payloads can activate unnoticed, leading to serious security risks.
Implications for AI Security and Trust
Trail of Bits demonstrated this attack on real-world AI platforms such as Google’s Gemini CLI, Vertex AI, and mobile assistants. In one case, they uploaded an ordinary-looking image that, after resizing, contained a hidden prompt. Since many systems don’t preview the scaled image or check what the AI actually sees, the attack could silently cause the system to perform actions like exfiltrating sensitive data or executing commands without user approval.
This discovery underscores that even routine image processing steps can be exploited if not carefully managed. As AI becomes more embedded in critical systems, recognizing and addressing these hidden risks is essential for safeguarding data and maintaining trust. The researchers also built Anamorpher, a tool to help visualize and craft such manipulated images, aiding developers and security teams in understanding and detecting these vulnerabilities.
Read the full security report on their official blog post here.
Leave a Reply