University of Oulu

J. Li, R. Ji, H. Liu, X. Hong, Y. Gao and Q. Tian, "Universal Perturbation Attack Against Image Retrieval," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, pp. 4898-4907, doi: 10.1109/ICCV.2019.00500

Universal perturbation attack against image retrieval

Saved in:
Author: Li, Jie1; Ji, Rongrong1,2; Liu, Hong1;
Organizations: 1Department of Artificial Intelligence, School of Informatics, Xiamen University, China
2Peng Cheng Lab, Shenzhen, China
3MOE Key Lab. for Intelligent Networks and Network Security/Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, PRC
4University of Oulu, Finland
5Tsinghua University
6Huawei Noah’s Ark Lab
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 4.5 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2020-06-03


Universal adversarial perturbations (UAPs), a.k.a. input-agnostic perturbations, has been proved to exist and be able to fool cutting-edge deep learning models on most of the data samples. Existing UAP methods mainly focus on attacking image classification models. Nevertheless, little attention has been paid to attacking image retrieval systems. In this paper, we make the first attempt in attacking image retrieval systems. Concretely, image retrieval attack is to make the retrieval system return irrelevant images to the query at the top ranking list. It plays an important role to corrupt the neighbourhood relationships among features in image retrieval attack. To this end, we propose a novel method to generate retrieval-against UAP to break the neighbourhood relationships of image features via degrading the corresponding ranking metric. To expand the attack method to scenarios with varying input sizes or untouchable network parameters, a multi-scale random resizing scheme and a ranking distillation strategy are proposed. We evaluate the proposed method on four widely-used image retrieval datasets, and report a significant performance drop in terms of different metrics, such as mAP and mP@10. Finally, we test our attack methods on the real-world visual search engine, i.e., Google Images, which demonstrates the practical potentials of our methods.

see all

Series: IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops
ISSN: 2160-7508
ISSN-E: 2160-7516
ISSN-L: 2160-7508
ISBN: 978-1-7281-4803-8
ISBN Print: 978-1-7281-4804-5
Pages: 4898 - 4907
Article number: 9010035
DOI: 10.1109/ICCV.2019.00500
Host publication: 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
Conference: IEEE/CVF International Conference on Computer Vision
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
213 Electronic, automation and communications engineering, electronics
Funding: This work is supported by the National Key R&D Program (No.2017YFC0113000 and No.2016YFB1001503), Nature Science Foundation of China (No.U1705262, No.61772443, and No.61572410), Scientific Research Project of National Language Committee of China (No.YB135-49), and Nature Science Foundation of Fujian Province, China (No.2017J01125 and No.2018J01106).
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.