Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks

Summary

Researchers introduce Multimodal-SEA, an interactive benchmark with 608 expert-designed tasks based on real-world fraud cases, covering 14 domains and 152 attack scenarios with multimodal content. Evaluating seven multimodal models, they find all are vulnerable, with attack success rates ranging from 7.1% to 34.5%. Adding multimodal content increases attack success from 19.9% to 29.2%, while stronger safety instructions reduce success but do not improve recognition of deceptive content.

This is a reviewed summary. Read the complete article at the original source.

Why it matters by industry

• Artificial intelligence

  This benchmark directly addresses the security of AI agents under multimodal social engineering attacks, a critical concern for deploying agents in real-world applications.