|Table of Contents|

Ensemble forecasting algorithm for anomaly detection onelectric-power big data log analysis platform(PDF)

《南京理工大学学报》(自然科学版)[ISSN:1005-9830/CN:32-1397/N]

Issue:
2017年05期
Page:
634-
Research Field:
Publishing date:

Info

Title:
Ensemble forecasting algorithm for anomaly detection onelectric-power big data log analysis platform
Author(s):
Ni Zhen12Li Qianmu1Guo Yajuan3
1.School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China; 2.School of Information Engineering,Nanjing Xiaozhuang University,Nanjing 210017,China; 3.Electric Power Research Institute,Jiangsu Electric Power Company,Nanjing 211100,China
Keywords:
log analysis anomaly detection big data platform ensemble forecasting algorithm
PACS:
TP393.08
DOI:
10.14177/j.cnki.32-1397n.2017.41.05.016
Abstract:
In view of that the traditional or the new log processing system can not meet the requirements of the log analysis in the current situation of big data entirely with the development of power enterprise network technology,an algorithm for estimating the number of systems based on time series and the evaluation system are presented to realize the system for the algorithm.The algorithm integrates multiple classification prediction algorithms to classify the collected log data,and then realize the purpose of forecasting the number of anomaly systems with the best accuracy.The evaluation system also supports that the algorithm can increase the security of the log analysis platform.

References:

[1] 付晓毓,任睿,詹剑锋,等.一种基于日志结合分析的集群系统失效预测方法[J].高技术通讯,2016,26(6):519-527.
Fu Xiaoyu,Ren Rui,Zhan Jianfeng,et al.A log co-analysis based failure prediction method for large-scale cluster systems[J].Chinese High Technology Letters,2016,26(6):519-527.
[2]刘燕.基于Map/Reduce框架的分布式日志分析系统的研究及应用[D].长春:东北师范大学信息科学与技术学院,2011.
[3]田曲波.基于日志分析的超级计算机错误预测方法研究[D].武汉:华中科技大学软件学院,2010.
[4]李涛.数据挖掘的应用与实践[M].厦门:厦门大学出版社,2013.
[5]Han Jiawei,KamberMicheline,Pei Jian,等.数据挖掘:概念与技术[M].范明,孟小峰 译.北京:机械工业出版社,2012.
[6]Agrawal R,Srikant R.Fast algorithms for mining association rules[M]//Readings in database systems(3rd ed.).San Mateo,US:Morgan Kaufmann Publishers Inc,1998.
[7]Ning P,Cui Y,Reeves D S.Analyzing intensive intrusion alerts via correlation[J].Lecture Notes in Computer Science,2003,2516:74-94.
[8]Tan Pangning,Steinbach Michael,Kumar Vipin.数据挖掘导论[M].范明,范宏建,译.北京:人民邮电出版社,2011.
[9]Jiang Y,Perng C S,Li T,et al.Cloud analytics for capacity planning and instant VM provisioning[J].IEEE Transactions on Network & Service Management,2013,10(3):312-325.
[10]Oliner A J,Aiken A,Stearley J.Alert detection in system logs[C]//Eighth IEEE International Conference on Data Mining.Pisa,Italy:IEEE Computer Society,2008:959-964.
[11]Liang Y,Zhang Y,Jette M,et al.BlueGene/L failure analysis and prediction models[C]//International Conference on Dependable Systems and Networks.Philadelphia,US:IEEE,2006:425-434.
[12]Liang Y,Zhang Y,Xiong H,et al.Failure prediction in IBM blueGene/L event logs[C]//IEEE International Conference on Data Mining.Washington DC,US:IEEE,2007:583-588.
[13]黄伟,陈昊,郭雅娟,等.基于集成分类的恶意应用检测方法[J].南京理工大学学报,2016,40(1):35-40.
Huang Wei,Chen Hao,Guo Yajuan,et al.Mobile malware detection approach using ensemble classification[J].Journal of Nanjing University of Science and Technology,2016,40(1):35-40.
[14]Yule G U.On a method of investigating periodicities in disturbed series with special reference to Wolfer’s sunspot numbers[J].Philosophical Transactions of the Royal Society B Biological Sciences,1927,226(226):267-298.
[15]Taylor H M,Karlin S.An introduction to stochastic modeling(Third Edition)[M].San Diego,US:Academic Press,1998:6-10.
[16]Percival D B,Walden A T.Wavelet methods for time series analysis[M].Cambridge,UK:Cambridge University Press,2016.
[17]Box G E P,Jenkins G M.Time series analysis:Forecasting and control[J].Journal of the Operational Research Society,1971,22(2):199-201.
[18]Engle R F.Autoregressive conditional heterosce-dasticity with estimates of the variance of United Kingdom inflation[J].Econometrica,1982,50(4):987-1007.
[19]Bottou L.Large-scale machine learning with stochastic gradient descent[J].Proceedings of COMPSTAT,2010:177-186.
[20]Cortes C,Vapnik V.Support-vector networks[J].Machine Learning,1995,20(3):273-297.

Memo

Memo:
-
Last Update: 2017-09-30