Within our research about statistical modeling for anomaly detection in big datasets we developed a software module for unsupervised learning (OnAIR Anomaly Score, OAS). OAS includes state-of-the-art algorithms working both in batch mode, with data coming from several devices of the same type, and online in real-time connection with a single device.
With OAS we created a concept dashboard monitoring a set of hard disks in a laboratory/datacenter. Data is publicly avaiable at BackBlaze, OAS had been trained for a particular HITACHI model in use between 2013/2014, more than two million records have been studied. The model can catch all failures ahead of time with a very low false alarm rate.
The dashboard for a subset of the data is avaiable at this page.
The same tool, in online learning mode, allows to analyze in real-time data coming from a device, detetcing suspected concept drifts. This approach has been tested on several datasets concerning vibrations of bearings subjected to a mechanical stress (PHM2012). In every observed case, OAS succeeds in detecting the problems before they become unrecoverable.
The PHM2012 dashboard is avaiable at this page.