There is a lot to consider when setting up a big data streaming application: How much data will we need to handle? How important are “real time” results? What about constraints on data quality? And how can we deal with various failure scenarios? The open-source world offers numerous big data frameworks that can help process unbounded data, each with its own mechanisms to tackle these problems.

In this "recipe", I want to show you these frameworks and explain their mechanisms in order to give you some insights on which ingredients you should add to build a big data streaming application that suits your needs.