University of Oulu

J. Shi, X. Liu, Y. Zong, C. Qi and G. Zhao, "Hallucinating Face Image by Regularization Models in High-Resolution Feature Space," in IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2980-2995, June 2018. doi: 10.1109/TIP.2018.2813163

Hallucinating face image by regularization models in high-resolution feature space

Saved in:
Author: Shi, Jingang1; Liu, Xin1; Zong, Yuan1,2;
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, FI-90014 Oulu, Finland
2Key Laboratory of Child Development and Learning Science of Ministry of Education, Research Center for Learning Science, Southeast University, Nanjing 210096, China
3School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China
4School of Information and Technology, Northwest University, Xi’an 710069, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 3.9 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2018
Publish Date: 2019-02-25


In this paper, we propose two novel regularization models in patch-wise and pixel-wise, respectively, which are efficient to reconstruct high-resolution (HR) face image from low-resolution (LR) input. Unlike the conventional patch-based models which depend on the assumption of local geometry consistency in LR and HR spaces, the proposed method directly regularizes the relationship between the target patch and corresponding training set in the HR space. It avoids dealing with the tough problem of preserving local geometry in various resolutions. Taking advantage of kernel function in efficiently describing intrinsic features, we further conduct the patch-based reconstruction model in the high-dimensional kernel space for capturing nonlinear characteristics. Meanwhile, a pixel-based model is proposed to regularize the relationship of pixels in the local neighborhood, which can be employed to enhance the fuzzy details in the target HR face image. It privileges the reconstruction of pixels along the dominant orientation of structure, which is useful for preserving high-frequency information on complex edges. Finally, we combine the two reconstruction models into a unified framework. The output HR face image can be finally optimized by performing an iterative procedure. Experimental results demonstrate that the proposed face hallucination method produces superior performance than the state-of-the-art methods.

see all

Series: IEEE transactions on image processing
ISSN: 1057-7149
ISSN-E: 1941-0042
ISSN-L: 1057-7149
Volume: 27
Issue: 6
Pages: 2980 - 2995
DOI: 10.1109/TIP.2018.2813163
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
113 Computer and information sciences
Funding: This work was supported by the Academy of Finland, Tekes Fidipro Program, under Grant 1849/31/2015, in part by the Tekes Project under Grant 3116/31/2017, in part by Infotech, in part by the National Natural Science Foundation of China under Grant 61772419 and Grant 61572395, and in part by the Tekniikan Edistamissaatio Foundation.
Copyright information: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.