|Table of Contents|

Ensemble forecasting algorithm for anomaly detection onelectric-power big data log analysis platform(PDF)


Research Field:
Publishing date:


Ensemble forecasting algorithm for anomaly detection onelectric-power big data log analysis platform
Ni Zhen12Li Qianmu1Guo Yajuan3
1.School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China; 2.School of Information Engineering,Nanjing Xiaozhuang University,Nanjing 210017,China; 3.Electric Power Research Institute,Jiangsu Electric Power Company,Nanjing 211100,China
log analysis anomaly detection big data platform ensemble forecasting algorithm
In view of that the traditional or the new log processing system can not meet the requirements of the log analysis in the current situation of big data entirely with the development of power enterprise network technology,an algorithm for estimating the number of systems based on time series and the evaluation system are presented to realize the system for the algorithm.The algorithm integrates multiple classification prediction algorithms to classify the collected log data,and then realize the purpose of forecasting the number of anomaly systems with the best accuracy.The evaluation system also supports that the algorithm can increase the security of the log analysis platform.


[1] 付晓毓,任睿,詹剑锋,等.一种基于日志结合分析的集群系统失效预测方法[J].高技术通讯,2016,26(6):519-527.
Fu Xiaoyu,Ren Rui,Zhan Jianfeng,et al.A log co-analysis based failure prediction method for large-scale cluster systems[J].Chinese High Technology Letters,2016,26(6):519-527.
[5]Han Jiawei,KamberMicheline,Pei Jian,等.数据挖掘:概念与技术[M].范明,孟小峰 译.北京:机械工业出版社,2012.
[6]Agrawal R,Srikant R.Fast algorithms for mining association rules[M]//Readings in database systems(3rd ed.).San Mateo,US:Morgan Kaufmann Publishers Inc,1998.
[7]Ning P,Cui Y,Reeves D S.Analyzing intensive intrusion alerts via correlation[J].Lecture Notes in Computer Science,2003,2516:74-94.
[8]Tan Pangning,Steinbach Michael,Kumar Vipin.数据挖掘导论[M].范明,范宏建,译.北京:人民邮电出版社,2011.
[9]Jiang Y,Perng C S,Li T,et al.Cloud analytics for capacity planning and instant VM provisioning[J].IEEE Transactions on Network & Service Management,2013,10(3):312-325.
[10]Oliner A J,Aiken A,Stearley J.Alert detection in system logs[C]//Eighth IEEE International Conference on Data Mining.Pisa,Italy:IEEE Computer Society,2008:959-964.
[11]Liang Y,Zhang Y,Jette M,et al.BlueGene/L failure analysis and prediction models[C]//International Conference on Dependable Systems and Networks.Philadelphia,US:IEEE,2006:425-434.
[12]Liang Y,Zhang Y,Xiong H,et al.Failure prediction in IBM blueGene/L event logs[C]//IEEE International Conference on Data Mining.Washington DC,US:IEEE,2007:583-588.
Huang Wei,Chen Hao,Guo Yajuan,et al.Mobile malware detection approach using ensemble classification[J].Journal of Nanjing University of Science and Technology,2016,40(1):35-40.
[14]Yule G U.On a method of investigating periodicities in disturbed series with special reference to Wolfer’s sunspot numbers[J].Philosophical Transactions of the Royal Society B Biological Sciences,1927,226(226):267-298.
[15]Taylor H M,Karlin S.An introduction to stochastic modeling(Third Edition)[M].San Diego,US:Academic Press,1998:6-10.
[16]Percival D B,Walden A T.Wavelet methods for time series analysis[M].Cambridge,UK:Cambridge University Press,2016.
[17]Box G E P,Jenkins G M.Time series analysis:Forecasting and control[J].Journal of the Operational Research Society,1971,22(2):199-201.
[18]Engle R F.Autoregressive conditional heterosce-dasticity with estimates of the variance of United Kingdom inflation[J].Econometrica,1982,50(4):987-1007.
[19]Bottou L.Large-scale machine learning with stochastic gradient descent[J].Proceedings of COMPSTAT,2010:177-186.
[20]Cortes C,Vapnik V.Support-vector networks[J].Machine Learning,1995,20(3):273-297.


Last Update: 2017-09-30