University of Oulu

J. Shi and G. Zhao, "Face Hallucination via Coarse-to-Fine Recursive Kernel Regression Structure," in IEEE Transactions on Multimedia, vol. 21, no. 9, pp. 2223-2236, Sept. 2019. doi: 10.1109/TMM.2019.2898752

Face hallucination via coarse-to-fine recursive kernel regression structure

Saved in:
Author: Shi, Jingang1; Zhao, Guoying2,1
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, FI-90014 Oulu, Finland
2School of Information and Technology, Northwest University, Xi’an 710069, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.3 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-12-10


In recent years, patch-based face hallucination algorithms have attracted considerable interest due to their effectiveness. These approaches produce a high-resolution (HR) face image according to the corresponding low-resolution (LR) input by learning a reconstruction model from the given training image set. The critical problem in these algorithms is establishing the underlying relationship between LR and HR patch pairs. Most previous methods aim to denote each input LR patch by the linear combination of the training set in the LR space while utilizing the combination weights to reconstruct the target HR patch. However, this assumes that the same combination weights should be shared between various resolution spaces, which is truly difficult to satisfy because of the one-to-many mapping relation between LR and HR patches. In this paper, we directly train a series of adaptive kernel regression mappings for predicting the lost high-frequency information from the LR patch, which avoids dealing with the above difficult problem. During the training process, we first establish a local optimization function on each LR/HR training pair according to the geometric structure of neighboring patches. The objective of local optimization can be presented in two aspects: 1) ensure the reconstruction consistency between each LR patch and the corresponding HR patch and 2) preserve the intrinsic geometry between each HR training patch and its original neighbors after the reconstruction process. The local optimizations are finally incorporated as the global optimization for calculating the optimal kernel regression function. To better approximate the target HR patch, we further propose a recursive structure to compensate for the residual reconstruction error of high-frequency details by a series of regression mappings. The proposed method is rather fast yet very effective in producing HR face images. Experimental results show that the proposed approach achieves superior performance with reasonable computational time compared with the state-of-the-art methods.

see all

Series: IEEE transactions on multimedia
ISSN: 1520-9210
ISSN-E: 1941-0077
ISSN-L: 1520-9210
Volume: 21
Issue: 9
Pages: 2223 - 2236
DOI: 10.1109/TMM.2019.2898752
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
113 Computer and information sciences
Funding: This work was funded by the National Natural Science Foundation of China (No. 61772419), Academy of Finland, Bussiness Finland, Tekniikan Edistämissäätiö and Infotech Oulu
Copyright information: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.