lzo
Update 6: Some interesting changes from Twitter's Evan Weaver : everything in RAM now, database is a backup; peaks at 300 tweets/second; every tweet followed by average 126 people; vector cache of tweet IDs; row cache; fragment cache; page cache; keep separate caches; GC makes Ruby optimization resistant so went with Scala; Thrift and HTTP are used internally; 100s internal requests for every external request; rewrote MQ but kept interface the same; 3 queues are used to load balance requests; extensive A/B testing for backwards capability; switched to C memcached client for speed; optimize critical path; faster to get the cached results from the network memory than recompute them locally.
Scaling Twitter: Making Twitter 10000 Percent Faster | High Scal
When it comes to Twitter , everyone’s a critic. The irony is, the majority of the technical criticism written about Twitter reveals more about the lack of understanding of the author than anything about Twitter.
Scaling a Microblogging Service - Part I