ModuleLog : module based approach to anomaly detection in parallel file system logs

Search for this publication on Google Scholar
Egersdoerfer, C., Dai, D., & Zhang, D. (2022). ModuleLog : module based approach to anomaly detection in parallel file system logs. Unc Charlotte Undergraduate Research Conference.
Download PDF

Analytics

84 views ◎
74 downloads ⇓


Description

With the increasing prevalence of Parallel File Systems (PFSes) in the context of vast and complex server networks, the importance of accurate anomaly detection on runtime logs of parallel file systems is increasing. But as it currently stands, many state-of-the-art methods for log-based anomaly detection, such as DeepLog, have encountered numerous challenges when applied to parallel file system logs due to their irregularity and a lack of identifying characteristics. Although a previous work, SentiLog has shown promising results, the sentiment based model lacks analysis of temporal dependencies within a log sequence, and hence misses important sequence-based anomalies. To circumvent these problems, this study proposes ModuleLog, a log anomaly detection solution which analyzes the temporal sequence of logging modules to detect irregularity. The key distinction from existing sequence-based anomaly detection solutions is the attempt to reduce the granularity of using individual log keys by grouping these keys by the module they reside in, based on the PFS source code. We apply an RNN architecture with regular LSTM cells to the sequence of modules. This method allows ModuleLog to be able to detect transition points between normal and abnormal logs in a given sequence, as well as detect sequences of abnormal logs. Presented at the 2022 UNC Charlotte Undergraduate Research Conference.

Details
Title

ModuleLog : module based approach to anomaly detection in parallel file system logs

Author(s)

Egersdoerfer, Chris
Dai, Dong
Zhang, Di

UNC Charlotte Affiliate Type

Undergraduate

Extent

1 online resource

Date Created

2022

Material Type

Conference proceedings

Genre

conference proceedings

Resource Type

text

Abstract

With the increasing prevalence of Parallel File Systems (PFSes) in the context of vast and complex server networks, the importance of accurate anomaly detection on runtime logs of parallel file systems is increasing. But as it currently stands, many state-of-the-art methods for log-based anomaly detection, such as DeepLog, have encountered numerous challenges when applied to parallel file system logs due to their irregularity and a lack of identifying characteristics. Although a previous work, SentiLog has shown promising results, the sentiment based model lacks analysis of temporal dependencies within a log sequence, and hence misses important sequence-based anomalies. To circumvent these problems, this study proposes ModuleLog, a log anomaly detection solution which analyzes the temporal sequence of logging modules to detect irregularity. The key distinction from existing sequence-based anomaly detection solutions is the attempt to reduce the granularity of using individual log keys by grouping these keys by the module they reside in, based on the PFS source code. We apply an RNN architecture with regular LSTM cells to the sequence of modules. This method allows ModuleLog to be able to detect transition points between normal and abnormal logs in a given sequence, as well as detect sequences of abnormal logs. Presented at the 2022 UNC Charlotte Undergraduate Research Conference.

Subjects--Topics

Computer networks
Anomaly detection (Computer security)
Computer science

Is Part Of

UNC Charlotte undergraduate research conference

Language

English

Permalink

http://hdl.handle.net/20.500.13093/work:292

Rights

This item may be protected by copyright and other related rights. Atkins Library provides access to this item for educational and research purposes only; other uses require the permission of the copyright holder.