University of Oulu

Y. Xu, X. Hong, F. Porikli, X. Liu, J. Chen and G. Zhao, "Saliency Integration: An Arbitrator Model," in IEEE Transactions on Multimedia, vol. 21, no. 1, pp. 98-113, Jan. 2019. doi: 10.1109/TMM.2018.2856126

Saliency integration : an arbitrator model

Saved in:
Author: Xu, Yingyue1; Hong, Xiaopeng1; Porikli, Fatih2;
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, 90014, Oulu, Finland
2Research School of Engineering, Australian National University, Canberra, ACT 0200, Australia
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.5 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-06-03


Saliency integration has attracted much attention on unifying saliency maps from multiple saliency models. Previous offline integration methods usually face two challenges: 1) if most of the candidate saliency models misjudge the saliency on an image, the integration result will lean heavily on those inferior candidate models; and 2) an unawareness of the ground truth saliency labels brings difficulty in estimating the expertise of each candidate model. To address these problems, in this paper, we propose an arbitrator model (AM) for saliency integration. First, we incorporate the consensus of multiple saliency models and the external knowledge into a reference map to effectively rectify the misleading by candidate models. Second, our quest for ways of estimating the expertise of the saliency models without ground truth labels gives rise to two distinct online model-expertise estimation methods. Finally, we derive a Bayesian integration framework to reconcile the saliency models of varying expertise and the reference map. To extensively evaluate the proposed AM model, we test 27 state-of-the-art saliency models, covering both traditional and deep learning ones, on various combinations over four datasets. The evaluation results show that the AM model improves the performance substantially compared to the existing state-of-the-art integration methods, regardless of the chosen candidate saliency models.

see all

Series: IEEE transactions on multimedia
ISSN: 1520-9210
ISSN-E: 1941-0077
ISSN-L: 1520-9210
Volume: 21
Issue: 1
Pages: 98 - 113
DOI: 10.1109/TMM.2018.2856126
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work was supported in part by Tekes Fidipro Program under Grant 1849/31/2015, in part by the Tekes project under Grant 3116/31/2017, in part by Academy of Finland, in part by Infotech, and in part by the Natural Science Foundation of China under Grants 61772419/61572205/61601362. The computational resources were supported by CSC-IT Center for Science, Finland.
Copyright information: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.