Loading…
This event has ended. Visit the official site or create your own event on Sched.
June 25 - 27 - Beijing, China
Click Here For Information & Registration

Tuesday, June 26 • 16:00 - 17:00
Streaming Computation in Baidu

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
As the big data era comes, computing endless data in real-time has become a necessity in many scenarios. Take Baidu as an example, trillions of data comes to real-time computation platform everyday. From the year of 2011, DStream, a true streaming computation engine with its own scheduler system have been proposed, implemented and put into practice. It supports low-level but flexible API and configuration. Moreover, it support logging / monitoring / paging / tracing / releasing / dictionary / etc., which are crucial in production. Along the time, as DStream are for developers and needs learning curve, Spark Streaming are introduced for data scientists. Our team follows the Spark community. Moreover, best practices with DStream in production complexity are contributed back to SparkStreaming. We adapt Baidu home-brewed storage, messaging system, PaaS, etc., to Spark Streaming. In this session, we’d like to share our experience with DStream and Spark Streaming in Baidu.

Tuesday June 26, 2018 16:00 - 17:00
207