University of Oulu

Y. Xu et al., "Structured Modeling of Joint Deep Feature and Prediction Refinement for Salient Object Detection," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, pp. 3788-3797, doi: 10.1109/ICCV.2019.00389

Structured modeling of joint deep feature and prediction refinement for salient object detection

Saved in:
Author: Xu, Yingyue1; Xu, Dan2; Hong, Xiaopeng3,4,1;
Organizations: 1University of Oulu
2University of Oxford
3Xi’an Jiaotong University
4Peng Cheng Laborotory
5SenseTime Computer Vision Group, The University of Sydney
6Xiamen University
7University of Technology Sydney
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.5 MB)
Persistent link:
Language: English
Published: IEEE Computer Society, 2020
Publish Date: 2020-06-10


Recent saliency models extensively explore to incorporate multi-scale contextual information from Convolutional Neural Networks (CNNs). Besides direct fusion strategies, many approaches introduce message-passing to enhance CNN features or predictions. However, the messages are mainly transmitted in two ways, by feature-to-feature passing, and by prediction-to-prediction passing. In this paper, we add message-passing between features and predictions and propose a deep unified CRF saliency model. We design a novel cascade CRFs architecture with CNN to jointly refine deep features and predictions at each scale and progressively compute a final refined saliency map. We formulate the CRF graphical model that involves message-passing of feature-feature, feature-prediction, and prediction-prediction, from the coarse scale to the finer scale, to update the features and the corresponding predictions. Also, we formulate the mean-field updates for joint end-to-end model training with CNN through back propagation. The proposed deep unified CRF saliency model is evaluated over six datasets and shows highly competitive performance among the state of the arts.

see all

Series: IEEE International Conference on Computer Vision
ISSN: 1550-5499
ISSN-E: 2380-7504
ISSN-L: 1550-5499
ISBN: 978-1-7281-4803-8
ISBN Print: 978-1-7281-4804-5
Pages: 3788 - 3797
DOI: 10.1109/ICCV.2019.00389
Host publication: 2019 International Conference on Computer Vision, 27 October - 2 November 2019, Seoul, Korea : proceedings
Conference: IEEE International Conference on Computer Vision
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work is supported by the Academy of Finland ICT 2023 project (313600), Tekes Fidipro program (Grant No.1849/31/2015), Business Finland project (Grant No.3116/31/2017), Infotech Oulu, and National Natural Science Foundation of China (Grant No.61772419). Computational resources are supported by CSC-IT Center for Science, Finland and Nvidia.
Academy of Finland Grant Number: 313600
Detailed Information: 313600 (Academy of Finland Funding decision)
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.