You are here

Dr. Yali Yuan and Sripriya Adhatarao's paper "ADA: Adaptive Deep Log Anomaly Detector" accepted by IEEE INFOCOM 2020 program

Dr. Yali Yuan and Sripriya Adhatarao's paper "ADA: Adaptive Deep Log Anomaly Detector" accepted by IEEE INFOCOM 2020 program
December 5, 2019 - 11:19pm

The paper "ADA: Adaptive Deep Log Anomaly Detector" cowritten by Yali Yuan*, Sripriya Srikant Adhatarao*, Mingkai Lin, Yachao Yuan (University of Göttingen), Zheli Liu (Nankai University), and Xiaoming Fu (University of Göttingen), has been accepted by IEEE INFOCOM 2020, Beijing, China, April 2020 (totally, 268 papers were accepted out of 1354 submissions). *Both Yali Yuan and Sripriya Adhatarao are first authors with equal contributions . The  paper has the following abstract:

Large private and government networks are often subjected to attacks like data extrusion and service disruption. Existing anomaly detection systems use offline supervised learning and hence cannot detect anomalies in real-time. Even though unsupervised algorithms are increasingly used, they cannot readily adapt to newer threats. Moreover, such systems also suffer from high cost of storage and require extensive computational resources in addition to employing experts for labeling. In this paper, we propose ADA: Adaptive Deep Log Anomaly Detector, an unsupervised online deep neural network framework that leverages LSTM networks. We regularly adapt to new log patterns to ensure accurate anomaly detection. We also design an adaptive model selection strategy to choose Pareto-optimal configurations and thereby utilize resources efficiently. Further, we propose a dynamic threshold algorithm to dictate the optimal threshold based on recently detected events to improve the detection accuracy. We then use the predictions to guide storage of abnormal data and effectively reduce the overall storage cost. We compare ADA with the state-of-the-art using the Los Alamos National Laboratory cyber security dataset and show that ADA accurately detects anomalies with high F1-score ~95% and it is 97 times faster than existing approaches and incurs very low storage cost.

Another paper related to the NET group, in collaboration with Dr. Haisheng Tan‘s group at USTC in China, was also accepted by IEEE INFOCOM 2020: Zhenhua Han, Haisheng Tan, Shaofeng Jiang, Xiaoming Fu, Wanli Cao, Francis C.M. Lau, "Scheduling Placement-Sensitive BSP Jobs with Inaccurate Execution Time Estimation".