Scalability
Get flash to fully experience Pearltrees
Why are Facebook, Digg, an
Real-time social graphs (connectivity between people, places, and things). That's why scaling Facebook is hard says Jeff Rothschild , Vice President of Technology at Facebook. Social networking sites like Facebook, Digg, and Twitter are simply harder than traditional websites to scale. Why is that? Why would social networking sites be any more difficult to scale than traditional web sites? Let's find out.
Bytepawn - Scalable Web Architectures and Application State
In this article we follow a hypothetical programmer, Damian, on his quest to make his web application scalable. In the early 2000s Damian built a website for fellow gamers interested in the magic game dungeons and dragons. The site was simple, as it listed various properties of magic items and featured a messageboard. Damian used the LAMP stack : data about items was stored in Mysql, and various PHP scripts let the visitors of the site view items and statistics. Now fast forward to 2009.
Against all the odds | High Scalability
First let's describe what means by odds: In my social network, I found 93% of the mainstream developers sanctify the database, or at least consider it in any data persistence challenge as the ultimate, superhero, and undefeatable solution. I think this problem come from the education, personally, and some companies also I think it's involved in this. To start to fix this bad thinking, we all should agree in the following points: Every challenge have its own solutions, so whatever you want to save/persistent, there are always many solutions.» Scalable Web Applications Programming the new world: Programmi
Purpose of the entry On Saturday June 13th 2009 I attended a talk by Eli White on Scalable web applications. Eli White previously worked at digg.com and now holds the position PHP Community Manager & DevZone Editor-in-Chief at Zend Technologies .
Over the past few weeks, I’ve been taking whatever spare moments I can find to think about what technologies we’re going to use to build the initial release of BankSimple . Many people would probably assume that I’d immediately reach for Scala, what with having co-authored a book on the language , but that’s not how I approach engineering problems.
Node and Scaling in the Small vs Scaling in the Large
Update 2: Velocity 09: John Allspaw, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr . Insightful talk. Some highlights: Change is good if you can build tools and culture to lower the risk of change.
How to Succeed at Capacity Planning Without Really Trying : An I
examples
Digg: 4000% Performance In
O'Reilly Radar's James Turner conducted a very informative interview with Joe Stump, current CTO of SimpleGeo and former lead architect at Digg , in which Joe makes some of his usually insightful comments on his experience using Cassandra vs MySQL. As Digg started out with a MySQL oriented architecture and has recently been moving full speed to Cassandra, his observations on some of their lessons learned and the motivation for the move are especially valuable. Here are some of the key takeaways you find useful: Precompute on writes, make reads fast . This is an oldie as a scaling strategy, but it's valuable to see how SimpleGeo is applying it to their problem of finding entities within a certain geographical region. Using Cassandra they've built two clusters: one for indexes and one for records.Update 4: Why you don’t want to shard. by Morgon on the MySQL Performance Blog. Optimize everything else first, and then if performance still isn’t good enough, it’s time to take a very bitter medicine. Update 3: Building Scalable Databases: Pros and Cons of Various Database Sharding Schemes by Dare Obasanjo. Excellent discussion of why and when you would choose a sharding architecture, how to shard, and problems with sharding. Update 2: Mr. Moore gets to punt on sharding by Alan Rimm-Kaufman of 37signals.
An Unorthodox Approach to Database Design : The Coming of the Sh
This is a guest post by Steffen Konerow, author of the High Performance Blog . Learning how to scale isn’t easy without any prior experience. Nowadays you have plenty of websites like highscalability.com to get some inspiration, but unfortunately there is no solution that fits all websites and needs.
6 Ways to Kill Your Servers - Learning How to Scale the Hard Way
The biggest thing in web apps since “rails can’t scale” is this idea that “your rdbms doesn’t scale.”
NoSQL: If Only It Was That Easy « Marked As Pertinent
Eric Lai published a provoking article on Computerworld magazine titled “ No to SQL? Anti-database movement gains steam ” where he pointed to many references in which different Internet-based companies chose an alternative approach to the traditional SQL database. The write-up was driven from the the inaugural get-together of the burgeoning NoSQL community who seem to represent a growing Anti-SQL database movement. Quoting Jon Travis from this article: Relational databases give you too much.
No to SQL? Anti-database movement gains stea
Advice from Google on large distributed syste
Google Fellow Jeff Dean gave a keynote talk at LADIS 2009 on "Designs, Lessons and Advice from Building Large Distributed Systems". Slides ( PDF ) are available. Some of this talk is similar to Jeff's past talks but with updated numbers. Let me highlight a few things that stood out: A standard Google server appears to have about 16G RAM and 2T of disk.Put that database in memory
An upcoming paper, "The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM" ( PDF ), makes some interesting new arguments for shifting most databases to serving entirely out of memory rather than off disk. The paper looks at Facebook as an example and points out that, due to aggressive use of memcached and caches in mysql, the memory they use already is about "75% of the total size of the data (excluding images)." They go on to argue that a system designed around in-memory storage with disk just used for archival purposes would be much simpler, more efficient, and faster. They also look at examples of smaller databases and note that, with servers getting to 64G of RAM and higher and most databases just a couple terabytes, it doesn't take that many servers to get everything in memory.You don't even have to make a bid, Randy Shoup, an eBay Distinguished Architect, gives this presentation on how eBay scales, for free.