background preloader

Etsy/statsd

Etsy/statsd
Related:  Java

Ticketfly/pillage Ganglia Monitoring System Start page – collectd – The system statistics collection daemon Getting Started | Metrics Getting Started will guide you through the process of adding Metrics to an existing application. We’ll go through the various measuring instruments that Metrics provides, how to use them, and when they’ll come in handy. Setting Up Maven You need the metrics-core library as a dependency: <dependencies><dependency><groupId>io.dropwizard.metrics</groupId><artifactId>metrics-core</artifactId><version>${metrics.version}</version></dependency></dependencies> Note Make sure you have a metrics.version property declared in your POM with the current version, which is 3.1.0. Now it’s time to add some metrics to your application! Meters A meter measures the rate of events over time (e.g., “requests per second”). private final Meter requests = metrics.meter("requests"); public void handleRequest(Request request, Response response) { requests.mark(); // etc} This meter will measure the rate of requests in requests per second. Console Reporter Complete getting started So the complete Getting Started is <? To run

Analyzing the Analyzers Monitoring at Spotify: The Story So Far | Labs This is the first in a two-part series about Monitoring at Spotify. In this, I’ll be discussing our history, the challenges we faced, and how they were approached. Operational monitoring at Spotify started its life as a combination of two systems. In late 2013, we were starting to put more emphasis on self service and distributed operational responsibility. We tried to bandage up what we could: our Chief Architect hacked together an in-memory sitemon replacement that could hold roughly one month worth of metrics under the current load. Alerting as a service Alerting was the first problem we took a stab at. We considered developing Zabbix further. We found inspiration from attending Monitorama EU where we stumbled upon Riemann. We built a library on top of Riemann called Lyceum. Graphing We went a few rounds here. It became desirable to switch to a push-based approach to lower the barriers of entry for our engineers. The difficulties in sharding and rebalancing Graphite became prohibitive.

Composite Design Pattern Intent Compose objects into tree structures to represent whole-part hierarchies. Composite lets clients treat individual objects and compositions of objects uniformly.Recursive composition"Directories contain entries, each of which could be a directory."1-to-many "has a" up the "is a" hierarchy Problem Application needs to manipulate a hierarchical collection of "primitive" and "composite" objects. Discussion Define an abstract base class (Component) that specifies the behavior that needs to be exercised uniformly across all primitive and composite objects. Use this pattern whenever you have "composites that contain components, each of which could be a composite". Child management methods [e.g. addChild(), removeChild()] should normally be defined in the Composite class. Structure Composites that contain Components, each of which could be a Composite. Menus that contain menu items, each of which could be a menu. Directories that contain files, each of which could be a directory. Example Opinions

Mining Time-series with Trillions of Points: Dynamic Time Warping at scale Take a similarity measure that's already well-known to researchers who work with time-series, and devise an algorithm to compute it efficiently at scale. Suddenly intractable problems become tractable, and Big Data mining applications that use the metric are within reach. The classification, clustering, and searching through time series have important applications in many domains. In medicine EEG and ECG readings translate to time-series data collections with billions (even trillions) of points. The problem is that existing algorithms don't scale1 to sequences with hundreds of billions or trillions of points. Recently a team of researchers led by Eamonn Keogh of UC Riverside introduced a set of tools for mining time-series with trillions of points. What is Dynamic Time Warping? SQRT[ Σ (xi - yi)2 ] While ED is easy to define, it performs poorly as a similarity score. There are an exponential number of paths (from one time series to the other) through the warping matrix. 1. 1. 1.

Monitoring at Spotify: Introducing Heroic | Labs This is the second part in a series about Monitoring at Spotify. In the previous post I discussed our history of operational monitoring. In this part I’ll be presenting Heroic, our scalable time series database which is now free software. Heroic is our in-house time series database. We are aware Elasticsearch has a bad reputation for data safety, so we guard against total failures by having the ability to completely rebuild the index rapidly from our data pipeline or Cassandra. A key feature of Heroic is global federation. Every host in our infrastructure is running ffwd, which is an agent responsible for receiving and forwarding metrics. This setup allows us to rapidly experiment with our service topology. In the backend everything is stored exactly as it was provided to the agent. In using Heroic, we’ve been able to build custom dashboards and alerting systems that make use of the same interface. All parts of Heroic is now free software, feel free to grab the code on Github.

Starbucks Does Not Use Two-Phase Commit - Enterprise Integration Patterns Hotto Cocoa o Kudasai I just returned from a 2 week trip to Japan. One of the more familiar sights was the ridiculous number of Starbucks (スターバックス) coffee shops, especially around Shinjuku and Roppongi. Correlation By taking advantage of an asynchronous approach Starbucks also has to deal with the same challenges that asynchrony inherently brings. Exception Handling Exception handling in asynchronous messaging scenarios can be difficult. Write-off - This error handling strategy is the simplest of all: do nothing. All of these strategies are different than a two-phase commit that relies on separate prepare and execute steps. Conversations The coffee shop interaction is also a good example of a simple, but common Conversation pattern. In summary we can see that the real world is often asynchronous.

etsy/oculus Linux Performance Analysis in 60,000 Milliseconds You login to a Linux server with a performance issue: what do you check in the first minute? At Netflix we have a massive EC2 Linux cloud, and numerous performance analysis tools to monitor and investigate its performance. These include Atlas for cloud-wide monitoring, and Vector for on-demand instance analysis. While those tools help us solve most issues, we sometimes need to login to an instance and run some standard Linux performance tools. In this post, the Netflix Performance Engineering team will show you the first 60 seconds of an optimized performance investigation at the command line, using standard Linux tools you should have available. In 60 seconds you can get a high level idea of system resource usage and running processes by running the following ten commands. uptime dmesg | tail vmstat 1 mpstat -P ALL 1 pidstat 1 iostat -xz 1 free -m sar -n DEV 1 sar -n TCP,ETCP 1 top Some of these commands require the sysstat package installed. 1. uptime 2. dmesg | tail 3. vmstat 1 7. free -m

airlift/airline

Related: