Friday, April 4, 2014

What We Learned About Scaling with Apache Storm: Pushing the Performance Envelope

Log management isn’t easy to do at scale. We designed Loggly Gen2 using the latest social-media-scale technologies—including ElasticSearch, Kafka from LinkedIn, and Apache Storm—as the backbone of ingestion processing for our multi-tenant, geo-distributed, and real-time log management system. (If you want to learn how we did it and why, watch our presentation at last year’s AWS re:Invent or check out the slides. There’s a reason that our talk has become the most watched non-keynote talk by far.)

Since we launched Gen2, we’ve learned a lot more about these technologies. We regularly contribute back to the open source community, so I decided that it’s time to give an update on our experience with Storm and explain why we have dropped it from our platform, at least for now.

Read more here

Leave a Reply

All Tech News IN © 2011 & Main Blogger .