How we ended up with microservices. Microservices are a thing these days.
When I was at SoundCloud, I was responsible for the migration from a monolithic Ruby on Rails application to a constellation of microservices. I’ve told the technical side of this story multiple times, both in presentations, and as a multi-part series for SoundCloud’s engineering blog. These engineering bits are what people are most interested in hearing about, but recently I realised I never explained to a wider audience how we ended up using microservices to begin with. I am sorry to disappoint my fellow techies, but the reason we migrated to microservices had to do much more with productivity than pure technical matters. I’ll explain. DevOps at Amazon: A Look at Our Tools and Processes.
Delivering eBay’s CI Solution with Apache Mesos – Part I. Problem statement In eBay’s existing CI model, each developer gets a personal CI/Jenkins Master instance.
This Jenkins instance runs within a dedicated VM, and over time the result has been VM sprawl and poor resource utilization. We started looking at solutions to maximize our resource utilization and reduce the VM footprint while still preserving the individual CI instance model. After much deliberation, we chose Apache Mesos for a POC. This post shares the journey of how we approached this challenge and accomplished our goal. Jenkins framework’s Mesos plugin The Mesos plugin is Jenkins’ gateway into the world of Mesos, so it made perfect sense to bring the plugin code in sync with our requirements. Untitled. /\ndy: The Failure of Agile. I am proud to be one of the 17 founders/authors of the The Agile Manifesto back in 2001.
I think it provided a jolt of energy, hope of a better way of doing things, of creating software and making the world work better. It was a pivotal turning point. But in the 14 years since then, we‘ve lost our way. Sebastien Goasguen: 1 Command to Mesos with Docker Compose. If you have not tried Docker, you should.
The sheer power it puts in your hands and the simplicity of the user experience will just wow you. In this post, I will show you how to start a one node Mesos setup with Docker compose. Docker announced compose on February 26th. Deploy a Mesos Cluster with 7 Commands Using Docker. Deploy a Mesos Cluster with 7 Commands Using Docker This tutorial will show you how to bring up a single node Mesos cluster all provisioned out using Docker containers (a future post will show how to easily scale this out to multi nodes or see the update on the bottom).
This means that you can startup an entire cluster with 7 commands! Nothing to install except for starting out with a working Docker server. DocOps: Interview with Jim Turcotte. Jim Turcotte The following is an interview with Jim Turcotte, a senior vice president for CA Technologies and business unit executive for the Information Services team.
Les DSI face au changement technologique : l'informatique à deux vitesses, et au-delà. Comment l'innovation technologique évolue-t-elle au mieux dans les grandes entreprises d'aujourd'hui ?
C'est la question que se posent les responsables informatiques en 2015, tandis que la transformation numérique arrive dans le peloton de tête des priorités des entreprises, d'après les principales études auprès des dirigeants réalisées par le MIT et d'autres institutions. La difficulté, bien entendu, est que le changement quel qu'il soit est notoirement difficile, notamment quand des individus sont impliqués, comme c'est invariablement le cas dans toute initiative d'entreprise. Ajoutez à cela la technologie, notamment les suites de produits, les piles, les plates-formes et les appareils technologiques très complexes et inextricablement liés d'aujourd'hui, et vous avez là tous les ingrédients d'une adoption lente et d'une situation de blocage. Il est donc impératif de trouver et d'adopter de meilleurs modèles informatiques. OpenTSDB - A Distributed, Scalable Monitoring System.
Monitoring & Management. FhGFS - The Fraunhofer Parallel Cluster File System. Releases · boot2docker/windows-installer. Removing Old Records for Logstash / Elasticsearch / Kibana » Raging Computer. Part 4 of 4 – Part 1 – Part 2 – Part 3 Now that you’ve got all your logs flying through logstash into elasticsearch, how to remove old records that are no longer doing anything but consuming space and ram for the index?
These are all functions of elasticsearch. Deleting is pretty easy, as is closing an index. The awesome people working on elasticsearch already have the solution! Securing Elasticsearch / Kibana with nginx » Raging Computer. OS X à marche forcée, Apple doit-elle ralentir la cadence ? Exclusive: a behind-the-scenes look at Facebook release engineering. Facebook is headquartered in Menlo Park, California at a site that used belong to Sun Microsystems.
A large sign with Facebook's distinctive "like" symbol—a hand making the thumbs-up gesture—marks the entrance. When I arrived at the campus recently, a small knot of teenagers had congregated, snapping cell phone photos of one another in front of the sign. Thanks to the film The Social Network, millions of people know the crazy story of Facebook's rise from dorm room project to second largest website in the world. But few know the equally intriguing story about the engine humming beneath the social network's hood: the sophisticated technical infrastructure that delivers an interactive Web experience to hundreds of millions of users every day.
Logstash, ElasticSearch & Kibana. Logstash config example. Logstash - open source log management. Milestone: 2 Stream events from files.
By default, each event is assumed to be one line. If you would like to join multiple log lines into one event, you’ll want to use the multiline codec. Experiment with the Kibana 4 beta : Luminis Amsterdam. A few weeks a go, the guys from elasticsearch made the public beta of Kibana 4 available. In this blog post I am going to generate some data and than play around with this data and the Kibana beta.
If you have never heard about these tools, read the introduction, if you do know it, you can safely skip the introduction. Introduction Kibana rocket the world in graphical ui world of logs. Together with Logstash and elasticsearch it has never been so easy to visualise the logs of applications. Log: Real Time Log Processing. I joined LinkedIn about six years ago at a particularly interesting time.
We were just beginning to run up against the limits of our monolithic, centralized database and needed to start the transition to a portfolio of specialized distributed systems. This has been an interesting experience: we built, deployed, and run to this day a distributed graph database, a distributed search backend, a Hadoop installation, and a first and second generation key-value store. One of the most useful things I learned in all this was that many of the things we were building had a very simple concept at their heart: the log.
Moving Security To the Left In a DevOps World - DevOps.comDevOps.com. Moving security to the left has become a coined phrase meant to describe the process of getting the security team involved earlier in a process. Most typically, the phrase is used in conjunction with IT or software development projects. One of the top suggestions for ensuring security in a DevOps world is to move security to the left in the process tool chain.