Credits - https://www.linkedin.com/in/scgupta
Lambda Architecture is a popular data processing architecture that includes both batch and stream processing. You can balance your throughput and latency requirements.
Just like the CAP theorem for distributed systems, We have this for big data analytics. You can have only two of the following: high throughput, low latency, and low cost.
Often leaders ask for Real-Time Analytics. You shouldn't comply blindly. Ask them what they would do differently if they had real-time analytics, which of their decision-making workflows will change if analytics moves from, say, daily, to real-time.
Then ask them to quantify the business value of the benefits. You should implement real-time analytics only if that value is substantially higher than the dev and OpEx cost.
Posts
