PlentyOfFish Architecture. Update 5: PlentyOfFish Update - 6 Billion Pageviews And 32 Billion Images A MonthUpdate 4: Jeff Atwood costs out Markus' scale up approach against a scale out approach and finds scale up wanting.
The discussion in the comments is as interesting as the article. My guess is Markus doesn't want to rewrite his software to work across a scale out cluster so even if it's more expensive scale up works better for his needs. Update 3: POF now has 200 million images and serves 10,000 images served per second. They'll be moving to a 250,000 IOPS RamSan to handle the load.
Also upgraded to a core database machine with 512 GB of RAM, 32 CPU’s, SQLServer 2008 and Windows 2008. PlentyOfFish is a hugely popular on-line dating system slammed by over 45 million visitors a month and 30+ million hits a day (500 - 600 pages per second). Site: GQL - Google App Engine. How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale.
Update 3: ReadWriteWeb says Google App Engine Announces New Pricing Plans, APIs, Open Access.
Pricing is specified but I'm not sure what to make of it yet. An image manipulation library is added (thus the need to pay for more CPU :-) and memcached support has been added. Memcached will help resolve the can't write for every read problem that pops up when keeping counters.Update 2: onGWT.com threw a GAE load party and a lot of people came. The results at Load test : Google App Engine = 1, Community = 0. GAE handled a peak of 35 requests/second and a sustained 10 requests/second. How do you structure your database using a distributed hash table like BigTable? Flickr anticipated this design in their architecture when they chose to duplicate comments in both the commentor and the commentee user shards rather than create a separate comment relation.
But Flickr’s reasoning was genius. From one world view comments logically belong to a relation binding comments and users together. YouTube Architecture. Update 3: 7 Years Of YouTube Scalability Lessons In 30 Minutes and YouTube Strategy: Adding Jitter Isn't A Bug Update 2: YouTube Reaches One Billion Views Per Day.
That’s at least 11,574 views per second, 694,444 views per minute, and 41,666,667 views per hour. Update: YouTube: The Platform. YouTube adds a new rich set of APIs in order to become your video platform leader--all for free. Upload, edit, watch, search, and comment on video from your own site without visiting YouTube. YouTube grew incredibly fast, to over 100 million video views per day, with only a handful of people responsible for scaling the site. Information Sources Google Video Platform Apache Python Linux (SuSe) MySQL psyco, a dynamic python->C compiler lighttpd for video instead of Apache What's Inside? The Stats Supports the delivery of over 100 million videos per day. Recipe for handling rapid growth.