University of Oulu

T. AlShammari, A. Elgabli and M. Bennis, "MetaBayes: A Meta-Learning Framework from a Bayesian Perspective," 2021 55th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, 2021, pp. 351-355, doi: 10.1109/IEEECONF53345.2021.9723290

MetaBayes : a meta-learning framework from a Bayesian perspective

Saved in:
Author: AlShammari, Tamara1; Elgabli, Anis1; Bennis, Mehdi1
Organizations: 1Centre for Wireless Communication, University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-04-05


Meta-learning is a powerful learning paradigm in which solving a new task can benefit from similar tasks for faster adaption (few shot learning). Stochastic gradient descent (SGD) based meta learning has emerged as an attractive solution in the few-shot learning. However, this approach suffers from significant computational complexity due to the double loop and matrix inversion operations which incurs a significant amount of uncertainty and poor generalization. To achieve lower complexity and better generalization, in this paper, we propose MetaBayes, a novel framework that views the original meta learning problem from a Bayesian perspective where the meta-model is cast as the prior distribution and the task-specific models are viewed as task-specific posterior distributions. The objective amounts to jointly optimizing the prior and the posterior distributions. With this, we obtain a closed-form expression to update the distributions at every iteration, to avoid the high computation cost issue of SGD based meta learning, and produce a more robust and generalized meta-model. Our simulations show that tasks with few training samples achieves higher accuracy when MetaBayes prior distribution is used as an initializer compared to the commonly-used Gaussian prior distribution.

see all

Series: Asilomar Conference on on Signals, Systems & Computers
ISSN: 2576-2303
ISSN-L: 2576-2303
ISBN: 978-1-6654-5828-3
ISBN Print: 978-1-6654-5829-0
Article number: 9723290
DOI: 10.1109/ieeeconf53345.2021.9723290
Host publication: 2021 55th Asilomar Conference on Signals, Systems, and Computers
Conference: Asilomar Conference on Signals, Systems, and Computers
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.