1. Introduction

2. Related Work

3. Preliminary

4. Methodology

4.2. Uncertainty in Log Sequence Detection

In this paper, we apply EDL combined with MLM method to construct anomaly log sequence detection.

First, valuable semantic information is mainly contained in log sequence variations such as trends, period, and peaks and valleys, which may correspond to the system state. For log sequences, there is a correlation between the logs at each location in the log sequence and the nearby logs. Anomalous log sequences are often the result of unexpected logs appearing at log locations where they should be.

Second, recent models using the MLM approach in BERT do not have the ability to detect anomalies autonomously. However, researchers have been able to detect anomalies through some phenomena such as low prediction probability of anomalous logs and large loss values [20]. Because the logs on the mask can be in multiple situations, normal logs can be mistaken for anomalous logs by simple presentation methods.

We propose a log anomaly detection system, LogEDL, that exploits the fact that EDL can be characterized by high uncertainty values for unseen patterns in order to overcome the limitations of existing methods. Uncertainty modeling is performed for each [mask] location, and the uncertainty of the [mask] location responds to the likelihood of an anomaly occurring here.In the training phase, by applying EDL to the MLM learning of normal logs, we are essentially trying to collect evidence of the classes reflected by the context at each mask. In the testing phase, the assumption of contextual uncertainty at each mask allows the model to “know the unknown”. Therefore, uncertainty is utilized to detect anomalies in the logs.

Figure 2. Examples of preprocessing system log (HDFS dataset).

Figure 2. Examples of preprocessing system log (HDFS dataset).

4.4. Problem definition

Log anomaly detection in this study can be defined as follows. Given an unstructured sequence of log messages, we aim to detect whether this sequence is normal or abnormal. We can then define the log sequence as an ordered sequence of log keys $S=\left\{k_1,\dots ,k_t,\dots ,k_T\right\}$, where $k_t\in V$ represents the log key at position t, and V denotes the set of unique log keys extracted from the log messages. The goal of this task is to predict whether a new log sequence S is abnormal or not based on a training dataset $Datase{t}_{train}={\left\{S_i\right\}}_{i=1}^N$ that contains only normal log sequences. To achieve this goal, LogEDL models normal log sequences and further derives an anomaly detection criterion to identify anomalous sequences. In the training phase, normal log key sequence keys are replaced by [MASK] tokens with a certain probability for network optimization; in the testing phase, the replacement is performed with the same probability to determine whether a log sequence is normal or not by the number of uncertainty log keys.

Figure 3. The framework of logEDL

Figure 3. The framework of logEDL

4.5. LogEDL

In this section, we present LogEDL, a semi-supervised log representation framework for anomaly detection. An overview of the proposed LogEDL framework is shown in Figure 3. It consists of three main components, Transformer Encoder, ENN head, and Uncertainty. Transformer Encoder is used after the log sequence is tokenized to extract the contextual information of the log sequence and transform it into semantic vectors. In the second step, ENN Head is used to convert the semantic vectors into evidential of the logs, which is the core of LogEDL, and finally Uncertainty is applied to make the judgment of anomaly detection based on the computed uncertainty. For this purpose, we build an evidential deep learning anomaly detection model. It mainly solves the problem of how to extract evidence of separable features with contextual information, which means that normal and abnormal sequences have boundaries in the feature space. Finally, the uncertainty of the extracted evidence is computed for anomalous log sequence detection. We describe each component in detail in the following subsections.

4.5.1. Transformer Encoder

We treat log sequences as sentences in natural language and use Transformer Encoder to extract the semantic vectors of log keys and capture the input log sequence contextual semantic information through a multi-head attention mechanism.Transformer Encoder consists of multiple transformer layers. Each transformer layer consists of a multi-head self-attention and a position-wise feed forward sub-layer using residual connection around the two sub-layers followed by layer normalization [8]. The sequence of logs $S=\left\{k_1,\dots ,k_t,\dots ,k_T\right\}$, where T is the sequence length. then represents each log key $k_t$ as an input representation $x_t$, where the representation $x_t$ are obtained by using the tokenizer, resulting in $X=\left\{x_1,\dots ,x_t,\dots ,x_T\right\}$ Secondly it is used as input to transformer encoder to obtain a d dimendional semantic vetor $h_t$, where d is set to 256. Finally we obtain the sequence of semantic vectors, written as $H=\left\{h_1,h_2,\cdots ,h_t\right\}$. To summarize, the transformer encoder represents the log keys with semantic vectors, which preserves the semantic information of the events similar log keys also have similar semantic vectors, mitigating the effect of unstable logs and noise.

Figure 4. ENN head

Figure 4. ENN head

4.5.2. ENN head

After the Transformer Encoder, each log key is represented as a semantic vector. To detect anomalous sequences, we propose an evidential neural network head (ENN head) based on an uncertainty mechanism, as shown in Figure 4. This module includes two fully connected (FC) layers, which are used to learn the evidence of contextual patterns across the entire sequence.Given a sequence of semantic vectors, $H=\left\{h_1,h_2,\cdots ,h_t\right\}$, the evidence $E=\left\{e_1,e_2,\cdots ,e_t\right\}$ for this sequence is learned in two layers of FC after ENN head. Note that we have the last layer of hidden states as output, denoted as:

$$e_t={\sigma (w}_2\sigma \left(w_1{h}_t+b_1\right)+b_2)$$

(1)

where $w_1,w_2,b_1,b_2$ are the trainable parameters of the input $h_t$, and $\sigma \left( \right)$ is the Relu function.

When we obtain the evidence $e_t$, it can efficiently learn the information about the mask position of the input log sequence in relation to the context and use the last step of the hidden state of the output as a representation of the context pattern of this log key position. To achieve this goal, we propose MMSM to model normal sequences, followed by log sequences-wise contrastive learning for representation learning, and further utilize an evidential neural network module to derive an anomaly detection criterion for identifying anomalous sequences.

4.5.3. Uncertainty

After obtaining the evidence, the Dirichlet prior distribution is utilized to compute the uncertainty, which is then used for subsequent anomaly detection. The Dirichlet distribution is a probability density function (pdf) that defines the possible values of a probability mass function (pmf) p. It is characterized by K parameters $\alpha =\left[{\alpha }_1,\dots ,{\alpha }_k,\dots ,{\alpha }_K\right]$ and is expressed as follows:

$$Dir\left(p|\alpha \right)=\left\{\begin{array}{cc}\frac{1}{B\left(\alpha \right)}\prod _{k=1}^K{p}_k^{{\alpha }_k-1},\hfill & forp\in P_K\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.$$

(2)

where $P_K$ is the K-dimensional unit simplex, $P_K=\left\{p\mid {\sum }_{i=1}^K{p}_i=1and0\le p_1,\cdots ,p_K\le 1\right\}$${\alpha }_k$ represents the concentration parameter of the Dirichlet distribution, K denotes the number of log key classes.

According to belief theory [29], subjective logic provides a belief mass $b_k$ for each singleton $k=1,\cdots ,K$, along with an overall uncertainty mass u. These $K+1$ mass values are non-negative and sum to 1, i.e.

$${\displaystyle u+\sum _{k=1}^K{b}_k=1}$$

(3)

where $u\ge 0$ and $b_k\ge 0$, for $k=1,\dots ,K$.

A belief mass $b_k$ for a singleton k is computed using evidence for singleton. Let $e_k\ge 0$ be the evidence derived for the $k^{th}$ Singleton, then the belif $b_k$ and the uncertainty u are computed as

$$b_k=\frac{e_k}{\mathcal{S}}andu=\frac{K}{\mathcal{S}}$$

(4)

where $\mathcal{S}={\sum }_{k=1}^K{\alpha }_k$ is referred to as the Dirichlet strength, and a belief mass assignment, i.e., subjective opinion, corresponds to a Dirichlet distribution with parameters ${\alpha }_k=e_k+1$.

The probability is given by

$$p_k=\frac{{\alpha }_k}{\mathcal{S}}$$

(5)

We optimized the network by setting a suitable loss function, enabling it to effectively learn normal log sequences. By leveraging uncertainty, we identified unknown anomalous logs, thus determining whether a log sequence is anomalous.We use the K-simplex (illustrated as a triangular heatmap in Figure 5) to jointly represent the token classification probability and uncertainty at the masked positions. The uncertainty detection at [MASK] is particularly suitable for log anomaly detection. Using a 3-class classification as an example and assuming the first class is the correct label, a well-calibrated model should give Accurate and Certain (AC) predictions (Figure 5(a)), indicating that the context for the [MASK] token is very certain. Accurate and Uncertain (AU) predictions (Figure 5(b)) indicate that the model is somewhat confused by the context and cannot classify it well. Inaccurate and Certain (IC) predictions (Figure 5(c)) suggest that the [MASK] token could have multiple possible values, which aligns well with the contextual variability in log sequences. Inaccurate and Uncertain (IU) predictions (Figure 5(d)) indicate that the context for the [MASK] token is completely unfamiliar, suggesting a high likelihood of the log sequence being anomalous.

4.5.4. anomaly detection

After training, we can deploy logEDL for detecting anomalous log sequences. The idea behind applying logEDL for log anomaly detection is that "there are differences in the context between normal system logs and anomalous system logs."In other words, when a language model trained solely on normal log data encounters the context of anomalous logs during testing, it is expected to exhibit higher uncertainty.

In this study, the defined uncertainty refers to the logit values produced by the model when predicting the [MASK] token, indicating the network’s inability to make a judgment on this contextual pattern. Furthermore, when the uncertainty is high, it suggests that the corresponding context is difficult to find in normal contexts, thus identifying it as an anomaly.

Therefore, we can derive the anomaly score of a log sequence based on the predicted logits for the [MASK] token. To achieve this, given a test log sequence, we first randomly replace a certain proportion of log keys with the [MASK] token, similar to the training process, and use the randomly masked log sequence as input to logEDL. The uncertainty of the $k^{th}$ log key refers to the degree of unknown represented by $u_k$ when the log key at the [MASK] position is considered as the answer. For each [MASK] token, we compute $u_k$, which reflects the uncertainty at this masked position. For n [MASK] tokens, we select the $top-k$$u_k$ values and compute the final anomaly score to detect anomalies. The formula for calculating the anomaly score of a test log sequence is shown in Equation 6.

$$abnormal=\frac{1}{n}\sum _{k\in Top-nindices}{u}_k$$

(6)

4.5.5. Evidential Uncertainty Loss

To train the logEDL model, we follow the MLM paradigm to capture patterns in normal log sequences. The neural network architecture of logEDL is very similar to that of a classical neural network. The only difference is that we replace the softmax layer with an activation layer, such as ReLU, to ensure non-negative outputs, which serve as the evidence vector for predicting the Dirichlet distribution.

Given a sample $k_i$, let $f\left(k_i\right|\Theta )$ represent the evidence vector $e_i$ predicted by the network for classification, where $\Theta$ denotes the network parameters. The corresponding Dirichlet distribution has parameters ${\alpha }_i=f\left(k_i\right|\Theta )+1$. Once these distribution parameters are computed, their mean, $k_i/\mathcal{S}$, can be used as an estimate of the class probabilities. Let the ground truth $y_i$ be the one-hot vector encoding the observation $k_i$, with $y_{ij}=1$ and $y_{ij}=0$ for all $i\ne j$. Here, ${\alpha }_i$ is the parameter of the Dirichlet density for the predictor. Assuming that class probability follows a prior Dirichlet distribution, the evidential mean squared error loss (EMSE) to be minimized for learning eventually reduces to the following form

$$\begin{array}{cc}\hfill {\mathcal{L}}_{EMSE}^i\left(\Theta \right)& {\displaystyle =\int {∥y_i-{\mathrm{p}}_i∥}_2^2\frac{1}{B\left({\alpha }_i\right)}\prod _{i=1}^K{p}_{ij}^{{\alpha }_{ij-1}}d{\mathrm{p}}_i}\hfill \\ \hfill & {\displaystyle =\sum _{j=1}^K\mathbb{E}[y_{ij}^2-2y_{ij}{p}_{ij}+p_{ij}^2]}\hfill \\ \hfill & {\displaystyle =\sum _{j=1}^K(y_{ij}^2-2y_{ij}\mathbb{E}\left[p_{ij}\right]+\mathbb{E}\left[p_{ij}^2\right])}\hfill \\ \hfill & {\displaystyle =\sum _{j=1}^K{(y_{ij}-{\alpha }_{ij}/{\mathcal{S}}_i)}^2+\frac{{\alpha }_{ij}({\mathcal{S}}_i-{\alpha }_{ij})}{{\mathcal{S}}_i^2({\mathcal{S}}_i+1)}}\hfill \\ \hfill & {\displaystyle =\sum _{j=1}^K{(y_{ij}-p_{ij})}^2+\frac{p_{ij}(1-p_{ij})}{({\mathcal{S}}_i+1)}}\hfill \end{array}$$

(7)

By decomposing the first and second moments, the loss function aims to achieve a joint objective of minimizing both the prediction error and the variance of the Dirichlet distribution generated by the neural network for each sample in the training set. In doing so, it prioritizes data fitting over variance estimation. Given that the patterns of normal log sequences and anomalous log sequences are different, we expect that once logEDL can correctly predict the masked log keys, it will be able to distinguish between normal and anomalous log sequences.

we can obtain the negative log marginal likelihood by integrating out the class probability, the evidential negative log likelihood loss (ENL) takes

$$\begin{array}{cc}\hfill {\mathcal{L}}_{ENL}^i\left(\Theta \right)& {\displaystyle =-log(\int \prod _{j=1}^K{p}_{ij}^{y_{ij}}\frac{1}{B\left({\alpha }_i\right)}\prod _{i=1}^K{p}_{ij}^{{\alpha }_{ij-1}}d{\mathrm{p}}_i)}\hfill \\ \hfill & {\displaystyle =\sum _{j=1}^K{y}_{ij}(\psi \left(S_i\right)-\psi \left({\alpha }_{ij}\right))}\hfill \end{array}$$

(8)

The same approach can be applied also to the cross-entropy loss, the evidential cross-entropy loss (ECE) will read

$$\begin{array}{cc}\hfill {\mathcal{L}}_{ECE}^i\left(\Theta \right)& {\displaystyle =\int \left[{\displaystyle \sum _{j=1}^K-y_{ij}log\left(p_{ij}\right)}\right]\frac{1}{B\left({\alpha }_i\right)}\prod _{i=1}^K{p}_{ij}^{{\alpha }_{ij-1}}d{\mathrm{p}}_i}\hfill \\ \hfill & {\displaystyle =\sum _{j=1}^K{y}_{ij}(\psi \left(S_i\right)-\psi \left({\alpha }_{ij}\right))}\hfill \end{array}$$

(9)

5. Experiment and Analysis

5.2. Results and Analysis

The comparative experiments is shown in Table 3, traditional anomaly detection methods, such as PCA, Isolation Forest, and OCSVM, have demonstrated suboptimal performance across the three datasets. Although these methods sometimes achieve high precision or recall, for instance, PCA’s recall reaches $100\%$ in all datasets, their F1 scores remain exceedingly low. This indicates an inability to balance precision and recall effectively, likely due to their failure to capture the temporal sequence information inherent in log data, resulting in inadequate anomaly detection capabilities.

LogCluster, a method specifically designed for log anomaly detection, shows improvements over traditional methods, particularly in the HDFS and BGL datasets. However, its performance in the Thunderbird dataset remains unsatisfactory, with F1 scores failing to reach high levels.

Table 3. Performance Comparison of LogEDL and Baseline methods.

Table 3. Performance Comparison of LogEDL and Baseline methods.

Method	HDFS			BGL			Thunderbird
	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
PCA	5.89	100.00	11.12	9.07	98.23	16.61	37.35	100.00	54.39
iForest	53.60	69.41	60.49	99.70	18.11	30.65	34.45	1.68	3.20
OCSVM	2.54	100.00	4.95	1.06	12.24	1.96	18.89	39.11	25.48
LogCluster	99.26	37.08	53.99	95.46	64.01	76.63	98.28	42.78	59.61
DeepLog	88.44	69.49	77.34	89.74	82.78	86.12	87.34	99.61	93.08
LogAnomaly	94.15	40.47	56.19	73.12	76.09	74.08	86.72	99.63	92.73
LogBERT	87.02	78.10	82.32	89.40	92.32	90.83	96.75	96.52	96.64
LogEDL (Ours)	90.06	92.80	91.41	99.84	97.26	98.53	97.80	98.08	97.91

Table 4. Ablation experiments on different evidence loss functions

Table 4. Ablation experiments on different evidence loss functions

Method	HDFS			BGL			Thunderbird
	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
logbert	87.02	78.10	82.32	89.40	92.32	90.83	96.75	96.52	96.64
LogEDL (w. ENL)	89.14	84.59	86.81	97.43	92.54	94.92	96.70	96.96	96.83
LogEDL (w. EMSE)	89.93	84.62	87.19	97.46	92.34	94.83	96.99	97.80	97.39
LogEDL (w. ECE)	90.06	92.80	91.41	99.84	97.26	98.53	97.80	98.08	97.91

DeepLog and LogAnomaly, two deep learning-based methods, significantly outperform traditional methods and LogCluster, showcasing the advantages of deep learning in capturing log sequence patterns. Both methods achieve high F1 scores across the three datasets, especially excelling in the HDFS and Thunderbird datasets.

LogBERT, a method based on pre-trained models and self-supervised learning tasks, demonstrates superior performance across all three datasets, with F1 scores markedly higher than those of traditional methods and other deep learning approaches. LogBERT successfully models normal log sequences and accurately identifies anomalous sequences through self-supervised pre-training tasks.

Our proposed method, LogEDL, surpasses all other methods across the three datasets, exhibiting exceptional performance. In the HDFS dataset, LogEDL achieves an F1 score of 91.41, significantly higher than LogBERT’s 82.32. In the BGL dataset, LogEDL’s F1 score of 98.53 substantially exceeds LogBERT’s 90.83. In the Thunderbird dataset, LogEDL’s F1 score of 97.91 outperforms LogBERT. This demonstrates that LogEDL maintains high precision and recall while achieving the best F1 scores.

The comparative analysis indicates that LogEDL outperforms other methods in log anomaly detection tasks. Traditional methods (PCA, Isolation Forest, OCSVM) exhibit significant shortcomings in balancing precision and recall, resulting in low F1 scores. The specialized method LogCluster shows some improvements but still lags behind deep learning methods. DeepLog and LogAnomaly demonstrate the strengths of deep learning, while LogBERT further enhances performance through self-supervised learning. Ultimately, LogEDL consistently achieves the best overall performance across the three datasets, proving its efficiency and reliability in log anomaly detection tasks.

5.2.1. Ablation Studies

The ablation experiments is shown in Table 4 Across all three datasets, the LogEDL model optimized with the ECE loss function demonstrates the best performance. ECE outperforms other methods in terms of precision, recall, and F1 score. Notably, in the BGL dataset, its precision reaches an almost perfect $99.84\%$. Compared to LogBERT, the LogEDL model shows improvements in all cases, especially when using the $ECE$ loss function, resulting in significant enhancements.Therefore, for similar log anomaly detection tasks, it is recommended to use the LogEDL model optimized with the evidence cross-entropy loss function to achieve the best performance.

6. Conclusion

7. Patents

Abbreviations

References

1. He, S.; He, P.; Chen, Z.; Yang, T.; Su, Y.; Lyu, M.R. A survey on automated log analysis for reliability engineering. ACM computing surveys (CSUR) 2021, 54, 1–37. [Google Scholar] [CrossRef]
2. Liu, Y.; Zhang, X.; He, S.; Zhang, H.; Li, L.; Kang, Y.; Xu, Y.; Ma, M.; Lin, Q.; Dang, Y.; et al. Uniparser: A unified log parser for heterogeneous log data. In Proceedings of the ACM Web Conference; 2022; pp. 1893–1901. [Google Scholar]
3. Ma, L.; Yang, W.; Xu, B.; Jiang, S.; Fei, B.; Liang, J.; Zhou, M.; Xiao, Y. KnowLog: Knowledge Enhanced Pre-trained Language Model for Log Understanding. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering; 2024; pp. 1–13. [Google Scholar]
4. Lin, Q.; Zhang, H.; Lou, J.G.; Zhang, Y.; Chen, X. Log clustering based problem identification for online service systems. In Proceedings of the 38th International Conference on Software Engineering Companion; 2016; pp. 102–111. [Google Scholar]
5. Xie, Y.; Zhang, H.; Zhang, B.; Babar, M.A.; Lu, S. Logdp: Combining dependency and proximity for log-based anomaly detection. In Proceedings of the Service-Oriented Computing: 19th International Conference, ICSOC 2021, Virtual Event, 22–25 November 2021; Proceedings 19. Springer, 2021; pp. 708–716. [Google Scholar]
6. Zhang, X.; Xu, Y.; Qin, S.; He, S.; Qiao, B.; Li, Z.; Zhang, H.; Li, X.; Dang, Y.; Lin, Q.; et al. Onion: identifying incident-indicating logs for cloud systems. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering; 2021; pp. 1253–1263. [Google Scholar]
7. Du, M.; Li, F.; Zheng, G.; Srikumar, V. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security; 2017; pp. 1285–1298. [Google Scholar]
8. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Advances in neural information processing systems 2017, 30. [Google Scholar]
9. Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv 2018, arXiv:1810.04805. [Google Scholar]
10. Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; Stoyanov, V. Roberta: A robustly optimized bert pretraining approach. arXiv 2019, arXiv:1907.11692. [Google Scholar]
11. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language models are few-shot learners. Advances in neural information processing systems 2020, 33, 1877–1901. [Google Scholar]
12. Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; Zettlemoyer, L. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv 2019, arXiv:1910.13461. [Google Scholar]
13. Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P.J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 2020, 21, 1–67. [Google Scholar]
14. Geng, C.; Huang, S.j.; Chen, S. Recent advances in open set recognition: A survey. IEEE transactions on pattern analysis and machine intelligence 2020, 43, 3614–3631. [Google Scholar] [CrossRef] [PubMed]
15. Liu, Z.; Miao, Z.; Zhan, X.; Wang, J.; Gong, B.; Yu, S.X. Large-scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2019; pp. 2537–2546. [Google Scholar]
16. Jafarzadeh, M.; Dhamija, A.R.; Cruz, S.; Li, C.; Ahmad, T.; Boult, T.E. A Review of Open-World Learning and Steps Toward Open-World Learning Without Labels. arXiv 2020, arXiv:2011.12906. [Google Scholar]
17. Amini, A.; Schwarting, W.; Soleimany, A.; Rus, D. Deep evidential regression. Advances in neural information processing systems 2020, 33, 14927–14937. [Google Scholar]
18. Sensoy, M.; Kaplan, L.; Kandemir, M. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems 2018, 31. [Google Scholar]
19. Guo, H.; Yuan, S.; Wu, X. Logbert: Log anomaly detection via bert. In Proceedings of the 2021 international joint conference on neural networks (IJCNN); IEEE, 2021; pp. 1–8. [Google Scholar]
20. Lee, Y.; Kim, J.; Kang, P. Lanobert: System log anomaly detection based on bert masked language model. Applied Soft Computing 2023, 146, 110689. [Google Scholar] [CrossRef]
21. Almodovar, C.; Sabrina, F.; Karimi, S.; Azad, S. LogFiT: Log anomaly detection using fine-tuned language models. IEEE Transactions on Network and Service Management 2024. [Google Scholar] [CrossRef]
22. Shafer, G. Dempster-shafer theory. Encyclopedia of artificial intelligence 1992, 1, 330–331. [Google Scholar]
23. Du, M.; Li, F. Spell: Streaming parsing of system event logs. In Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM); IEEE, 2016; pp. 859–864. [Google Scholar]
24. Makanju, A.A.; Zincir-Heywood, A.N.; Milios, E.E. Clustering event logs using iterative partitioning. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining; 2009; pp. 1255–1264. [Google Scholar]
25. He, P.; Zhu, J.; Zheng, Z.; Lyu, M.R. Drain: An online log parsing approach with fixed depth tree. In Proceedings of the 2017 IEEE international conference on web services (ICWS); IEEE, 2017; pp. 33–40. [Google Scholar]
26. Huang, S.; Liu, Y.; Fung, C.; He, R.; Zhao, Y.; Yang, H.; Luan, Z. Paddy: An event log parsing approach using dynamic dictionary. In Proceedings of the NOMS 2020-2020 IEEE/IFIP Network Operations and Management Symposium; IEEE, 2020; pp. 1–8. [Google Scholar]
27. Zhu, J.; He, S.; Liu, J.; He, P.; Xie, Q.; Zheng, Z.; Lyu, M.R. Tools and benchmarks for automated log parsing. In Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP); IEEE, 2019; pp. 121–130. [Google Scholar]
28. Qi, J.; Luan, Z.; Huang, S.; Fung, C.; Yang, H.; Li, H.; Zhu, D.; Qian, D. Logencoder: Log-based contrastive representation learning for anomaly detection. IEEE Transactions on Network and Service Management 2023. [Google Scholar] [CrossRef]
29. Yager, R.R.; Liu, L. Classic works of the Dempster-Shafer theory of belief functions; Springer, 2008; Volume 219. [Google Scholar]
30. Xu, W.; Huang, L.; Fox, A.; Patterson, D.; Jordan, M.I. Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles; 2009; pp. 117–132. [Google Scholar]
31. Oliner, A.; Stearley, J. What supercomputers say: A study of five system logs. In Proceedings of the 37th annual IEEE/IFIP international conference on dependable systems and networks (DSN’07); IEEE, 2007; pp. 575–584. [Google Scholar]
32. Zhu, J.; He, S.; He, P.; Liu, J.; Lyu, M.R. Loghub: A large collection of system log datasets for ai-driven log analytics. In Proceedings of the 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE); IEEE, 2023; pp. 355–366. [Google Scholar]
33. Liu, F.T.; Ting, K.M.; Zhou, Z.H. 2008 eighth ieee international conference on data mining. In Proceedings of the Eighth IEEE International Conference on Data Mining; 2008; pp. 413–422. [Google Scholar]
34. Schölkopf, B.; Platt, J.C.; Shawe-Taylor, J.; Smola, A.J.; Williamson, R.C. Estimating the support of a high-dimensional distribution. Neural computation 2001, 13, 1443–1471. [Google Scholar] [CrossRef] [PubMed]
35. Meng, W.; Liu, Y.; Zhu, Y.; Zhang, S.; Pei, D.; Liu, Y.; Chen, Y.; Zhang, R.; Tao, S.; Sun, P.; et al. Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. IJCAI 2019, 19, 4739–4745. [Google Scholar]