Wednesday, March 18, 2015

Real-Time Event Stream Processing – What are your choices?

A search on Google with “define real-time processing” yields “Batch processing requires separate programs for input, process and output. An example is payroll and billing systems. In contrast, real time data processing involves a continual input, process and output of data. Data must be processed in a small time period (or near real time)”. Emphasis is by Google Search itself. It’s interesting that Google decides to contrast the real-time with batch processing but (even though tangent to today’s topic) I think Google got it wrong when it cited Data Science Central’s article to define the properties of batch processing. Batch processing simply means that you periodically process the data because it brings in the optimization of the resources through amortization. How you do it is completely orthogonal to what it is. Contrary to Data Science Central’s definition of batch processing, I can use the same program to collect, process, and store (sounds very similar to ETL?), yet it can be batch processing.

Read more here

Leave a Reply

All Tech News IN © 2011 & Main Blogger .