University of Oulu

J. Nyyssälä, M. Mäntylä and M. Varela, "How to Configure Masked Event Anomaly Detection on Software Logs?," 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), Limassol, Cyprus, 2022, pp. 414-418, doi: 10.1109/ICSME55016.2022.00050.

How to configure masked event anomaly detection on software logs?

Saved in:
Author: Nyyssölä, Jesse1; Mäntylä, Mika1; Varela, Martín2
Organizations: 1M3S, ITEE, University of Oulu Oulu, Finland
2Profilence Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.1 MB)
Persistent link:
Language: English
Published: IEEE Computer Society Press, 2022
Publish Date: 2023-01-16


Software Log anomaly event detection with masked event prediction has various technical approaches with countless configurations and parameters. Our objective is to provide a baseline of settings for similar studies in the future. The models we use are the N-Gram model, which is a classic approach in the field of natural language processing (NLP), and two deep learning (DL) models long short-term memory (LSTM) and convolutional neural network (CNN). For datasets we used four datasets Profilence, BlueGene/L (BGL), Hadoop Distributed File System (HDFS) and Hadoop. Other settings are the size of the sliding window which determines how many surrounding events we are using to predict a given event, mask position (the position within the window we are predicting), the usage of only unique sequences, and the portion of data that is used for training. The results show clear indications of settings that can be generalized across datasets. The performance of the DL models does not deteriorate as the window size increases while the N-Gram model shows worse performance with large window sizes on the BGL and Profilence datasets. Despite the popularity of Next Event Prediction, the results show that in this context it is better not to predict events at the edges of the subsequence, i.e., first or last event, with the best result coming from predicting the fourth event when the window size is five. Regarding the amount of data used for training, the results show differences across datasets and models. For example, the N-Gram model appears to be more sensitive toward the lack of data than the DL models. Overall, for similar experimental setups we suggest the following general baseline: Window size 10, mask position second to last, do not filter out non-unique sequences, and use a half of the total data for training.

see all

Series: Proceedings. IEEE International Conference on Software Maintenance
ISSN: 1063-6773
ISSN-E: 2576-3148
ISSN-L: 1063-6773
ISBN: 978-1-6654-7956-1
ISBN Print: 978-1-6654-7957-8
Pages: 414 - 418
DOI: 10.1109/icsme55016.2022.00050
Host publication: 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)
Conference: IEEE International Conference on Software Maintenance and Evolution
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.