What is Big Data?

Big Data is the term by which large-volume data sets are known, that traditional computer applications are not able of processing or touching up. Likewise, the term Big Data is often used to refer to the necessary techniques to process and analyze these large volumes of data.

The challenges of Big Data are dealing with a large volume of information, processing it at high speed and dealing with a wide variety of sources and types of data.

To manage and understand such a large amount of data, it is essential to have adequate tools.

IDbox: a tool for big data

IDbox collects data from different sources, being able to store and process large volumes of data.

The data are processed, then statistics are generated and transformed to obtain a common structure, making easier their use and bringing data from different origins back together. Thus, data are prepared to be visualized and analyzed, using different statistical techniques and of Machine Learning.

In IDbox we find different techniques to make this analysis:

Correlations:

• Correlation
• Cross correlation
• Convolution

Predictions:

• Classical statistical techniques
• Machine Learning methods
• Detection of anomalies
• Pattern recognition

To obtain a great performance and efficiency in results, IDbox combines the power of the most modern techniques of Machine Learning, joined with the reliability of the classic statistical techniques.

For this, IDbox uses different environments and graphics.

Display of the correlations of a group of every 15 minute signals over a period of 6 months.

Display of combined scatter plot (For a better visualization, a classic scatter plot has been combined with a HeatMap). Thus it is possible to observe in which areas there is a greater concentration of points, their values and the dates between which they have been given.

Using the BPM, predictive models can be generated easily using Machine Learning algorithms such as:

• SVM
• Neural Networks
• Random Forest

Predictive maintenance: getting ahead of the problems

We define therefore predictive maintenance as the set of techniques that allow reducing costs to ensure availability and performance. It is a set of instrumented techniques for measuring and analyzing variables to predict the future point of breakage or failure of a component or system in such a way that said component can be used just before it fails.

These techniques require indicators or parameters to predict the life of the component to be analyzed within the platform, as well as the possibility that these will be monitored and measured during their lifetime.

This theoretically allows the repair, supply and labor time to be scheduled well in advance and has important implications:

• Reduces the possibility of downtimes.
• Extends the intervals between downtimes.
• It allows optimizing the service life of the components.

For this, the BPM training environment is used.

In the following image we can observe a prediction generated based on one of these algorithms. In this case, the training was conducted in the BPM based on a series of historical traffic data. In this way, we can give a prediction of how it will evolve next week. In this case, thanks to the quality of the data, the prediction is quite good.

Using IDbox Machine Learning techniques we can detect anomalies in real time, as well as observe the prediction in real time of the next value (taking into account the latest data).

IDbox Machine Learning also performs pattern recognition on historical data and data in real time. In this case, the user marks a pattern to search and through statistical techniques and Machine Learning, it is located where the pattern is reproduced.

In conclusion, IDbox is an ideal tool for Big Data, is able to store and work with large volumes of data from various sources, and apply the most cutting-edge Big Data analysis algorithms.