background preloader

Cluster

Facebook Twitter

Sharding Pinterest: How we scaled our MySQL fleet. “Shard.

Sharding Pinterest: How we scaled our MySQL fleet

Or do not shard. There is no try.” - Yoda This is a technical dive into how we split our data across many MySQL servers. We finished launching this sharding approach in early 2012, and it’s still the system we use today to store our core data. Before we discuss how to split the data, let’s be intimate with our data. Pinterest is a discovery engine for everything that interests you. Growing pains In 2011, we hit traction. Requirements Our overall system needed to be very stable, easy to operate and scale to the moon.

Design philosophies and notes Since we wanted this data to span multiple databases, we couldn’t use the database’s joins, foreign keys or indexes to gather all data, though they can be used for subqueries that don’t span databases. Application Clustering For Scalability and High Availability. Application clustering is a sub-topic of “Parallel Computing”.

Application Clustering For Scalability and High Availability

Today many types of software support parallel computing in some form. Why do we need parallel, clustered applications? For 2 reasons; scalability and high availability. Does every application need clustering? Guide - How to configure an apache load balancer - Community Wiki - Magnolia. At a certain amount of traffic, or a certain need on availability, you might consider using multiple public instances.

Guide - How to configure an apache load balancer - Community Wiki - Magnolia

Most likely those instances are on different servers as well. This guide will illustrate how to setup a load-balanced system using three different servers, where one acts as the load-balancer (using apache for splitting the requests) and the two remaining servers host the Magnolia public instances. Sticky session will ensure that one visitor generally will be handled by the same server over the lifetime of a session. If you want to access the public instances' adminCentral (or have public user registration or any other User Generated Content module installed) you want to use sticky sessions. Server setup The layout may look something like this (we will refer to these names through the rest of the guide).

Load-balancer This server will handle all HTTP requests from site visitors. First, create a virtual host handling the requests for your domain: www.yourcompany.com Done. Tomcat Clustering Series Part 2 : Session Affinity Load Balancer. Hi this second part of the Tomcat Clustering Series.

Tomcat Clustering Series Part 2 : Session Affinity Load Balancer

In my first part we discuss about how to setup simple load balancer. and we seen how load balancer distribute the request to tomcat instance in round robin fashion. [Check the video below for better understanding] In this post we discuss about what is the problem is occur in simple load balancer when we introduce sessions in our web application. and we will see how to resolve this issue. Its continuation of my first part of this series. so kindly go read my first part then continue here. How Session works in Servlet/Tomcat? Once response reached to client (Web Browser) its update the cookie value.

HTTP is stateless protocol. so server can't find the client session usual way. so server reads the header of the request and extract the cookie value and server got the Random session-id. then it search through group of session maintained by the tomcat. 5 Minute Guide to Clustering - Java Web Apps in Tomcat. Published Sat, 21 Aug 2010 • 62 comments I've been taking a break from posting for the last couple of weeks.

5 Minute Guide to Clustering - Java Web Apps in Tomcat

I was starting to get a bit run down, and feel like burn out was about to set in. The kind of blog posts I do take quite a bit of time, both in terms of the technical background work and the time to write and proof read the posts. Balancing that with work, plus personal projects, family life something had to take a break, and it's not going to be work or family life :) Anyhow, I'm back with another 5 minute guide. For the purposes of the rest of this article, when I say "Apache" I mean the web server, and when I say "Tomcat" I mean Tomcat.

There are pretty much two ways to set up basic clustering, which use two different Apache modules. Traffic is passed between Apache and Tomcat(s) using the binary AJP 1.3 protocol. Mod_jk stands for "jakarta" the original project under which Tomcat was developed. Modproxy is a newer and more generic way of setting this up.