Big Data in Real Time at Twitter. Big Data in Real-Time at Twitter. What is the best storage solution for building a "news feed" - MongoDB or MySQL. How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog. Background We use MySQL for storing all of the data in FriendFeed.
Our database has grown a lot as our user base has grown. We now store over 250 million entries and a bunch of other data, from comments and "likes" to friend lists. As our database has grown, we have tried to iteratively deal with the scaling issues that come with rapid growth. We did the typical things, like using read slaves and memcache to increase read throughput and sharding our database to improve write throughput.
What are the scaling issues to keep in mind while developing a social network feed. What are best practices for building something like a News Feed. Etsy Activity Feeds Architecture.