Article
Version 1
Preserved in Portico This version is not peer-reviewed
LogBD: A Log Anomaly Detection Method Based on Pre-trained Models and Domain Adaptation
Version 1
: Received: 2 May 2023 / Approved: 16 May 2023 / Online: 16 May 2023 (04:49:53 CEST)
A peer-reviewed article of this Preprint also exists.
Liu, S.; Deng, L.; Xu, H.; Wang, W. LogBD: A Log Anomaly Detection Method Based on Pretrained Models and Domain Adaptation. Appl. Sci. 2023, 13, 7739. Liu, S.; Deng, L.; Xu, H.; Wang, W. LogBD: A Log Anomaly Detection Method Based on Pretrained Models and Domain Adaptation. Appl. Sci. 2023, 13, 7739.
Abstract
The log data generated during the operation of a software system contains information about the system, and using logs for anomaly detection can detect system failures in a timely manner. Most of the existing log anomaly detection methods are specific to a particular system, have cold start problems, and are sensitive to updates in log format. In this paper, we propose a log anomaly detection method LogBD based on pre-training model and domain adaptation, which uses the pre-training model BERT to learn the semantic information of logs, and can solve the problems caused by multiple meaning words and log statement updates. An anomaly detection method LogBD based on distance to determine anomalies is constructed based on domain adaptation, using TCN networks to extract common features of different system logs and map them to the same hypersphere space. Finally, experiments are conducted on two publicly available datasets to evaluate the method, and the experimental results show that the method can better solve the log instability problem and has some improvement in the cross-system log anomaly detection effect.
Keywords
log anomaly detection; pre-trained models; domain adaptation
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (0)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment