University of Oulu

J. Zhang, Z. Su, Y. Feng, X. Lu, M. Pietikäinen and L. Liu, "Dynamic Binary Neural Network by Learning Channel-Wise Thresholds," ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, Singapore, 2022, pp. 1885-1889, doi: 10.1109/ICASSP43922.2022.9747328.

Dynamic binary neural network by learning channel-wise thresholds

Saved in:
Author: Zhang, Jiehua1; Su, Zhuo1; Feng, Yanghe2;
Organizations: 1Cmvs, University of Oulu
2National University of Defense Technology
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.6 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-04-03


Binary neural networks (BNNs) constrain weights and activations to +1 or -1 with limited storage and computational cost, which is hardware-friendly for portable devices. Recently, BNNs have achieved remarkable progress and been adopted into various fields. However, the performance of BNNs is sensitive to activation distribution. The existing BNNs utilized the Sign function with predefined or learned static thresholds to binarize activations. This process limits representation capacity of BNNs since different samples may adapt to unequal thresholds. To address this problem, we propose a dynamic BNN (DyBNN) incorporating dynamic learnable channel-wise thresholds of Sign function and shift parameters of PReLU. The method aggregates the global information into the hyper function and effectively increases the feature expression ability. The experimental results prove that our method is an effective and straightforward way to reduce information loss and enhance performance of BNNs. The DyBNN based on two backbones of ReActNet (MobileNetV1 and ResNet18) achieve 71.2% and 67.4% top1-accuracy on ImageNet dataset, outperforming baselines by a large margin (i.e., 1.8% and 1.5% respectively).

see all

Series: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing
ISSN: 1520-6149
ISSN-E: 2379-190X
ISSN-L: 1520-6149
ISBN: 978-1-6654-0540-9
ISBN Print: 978-1-6654-0541-6
Pages: 1885 - 1889
DOI: 10.1109/icassp43922.2022.9747328
Host publication: ICASSP 2022 : 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Conference: IEEE International Conference on Acoustics, Speech and Signal Processing
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
213 Electronic, automation and communications engineering, electronics
Funding: This work was partially supported by the Academy of Finland under grant 331883 and the National Natural Science Foundation of China under Grant 61872379. The authors also wish to acknowledge CSC IT Center for Science, Finland, for computational resources.
Academy of Finland Grant Number: 331883
Detailed Information: 331883 (Academy of Finland Funding decision)
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.