Preprint
Article

Combining Log Files and Monitoring Data to Detect Anomaly Patterns in a Data Center

Altmetrics

Downloads

298

Views

153

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

10 June 2022

Posted:

14 June 2022

You are already at the latest version

Alerts
Abstract
Context: Anomaly detection in a data center is a challenging task, having to consider different services on various resources. Current literature shows the application of artificial intelligence techniques to either log files or monitoring data: the former created by services at run time, while the latter produced by specific sensors directly on the physical or virtual machine. Objectives: We propose a model that exploits information both in log files and monitoring data to identify patterns and detect anomalies over time. Methods: The key idea is to use on one side natural language processing solutions to detect problems at service level, extracting words that represent anomalies. Clustering and topic modeling techniques have been used to identify patterns and group them with respect to topics. On the other side time series anomaly detection technique has been applied to sensors data in order to combine problems found in the log files with problems stored in the monitoring data. Results: We have tested our approach on a real data center equipped with log files and monitoring data that can characterize the behaviour of physical and virtual resources in production. We have observed a correspondence between anomalies in log files and monitoring data, e.g. an increase in memory usage or machine load. The results are extremely promising. Conclusion: Our model requires to integrate site administrators' expertise in order to consider all critical scenario in the data center and understand results properly.
Keywords: 
Subject: Computer Science and Mathematics  -   Information Systems
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated