Tag Archives: real time

Alarm Flooding Control with Event Clustering Using Spark Streaming

You show up at work in the morning and open your email to find 100 alarm emails in your inbox for the same error from an application running on some server within a short time window of 1 minute. You … Continue reading

Posted in Anomaly Detection, Big Data, Real Time Processing, Spark, stream processing | Tagged , , , | 1 Comment

Making Recommendations in Real Time

Making recommendations based on an user’s current behavior in a small time window is a powerful feature that has been added to sifarish recently. In this post I will go over the details of this feature. The real time feature … Continue reading

Posted in Big Data, Collaborative Filtering, Data Mining, Data Science, Hadoop and Map Reduce, Real Time Processing, Recommendation Engine, Redis, Storm | Tagged , | 2 Comments

Real Time Fraud Detection with Sequence Mining

Real time fraud detection  is one of the use cases, where multiple components of the Big Data eco system come into play in a significant way, Hadoop batch processing  for building the predictive model and Storm for predicting fraud from … Continue reading

Posted in Big Data, Data Mining, Data Science, Fraud Detection, Hadoop and Map Reduce, Redis, Storm | Tagged , , , | 25 Comments

Big Data Caught in Storm

Hadoop is great for batch processing. However depending on the  incoming data throughput and the cluster characteristic, there is a minimum latency threshold for processing data. My blog post is based on a simple performance model for Hadoop that allows … Continue reading

Posted in Big Data, Predictive Analytic, Real Time Processing | Tagged , | 12 Comments