Ganglia Monitoring System Analyzing the Analyzers Mining Time-series with Trillions of Points: Dynamic Time Warping at scale Take a similarity measure that's already well-known to researchers who work with time-series, and devise an algorithm to compute it efficiently at scale. Suddenly intractable problems become tractable, and Big Data mining applications that use the metric are within reach. The classification, clustering, and searching through time series have important applications in many domains. In medicine EEG and ECG readings translate to time-series data collections with billions (even trillions) of points. The problem is that existing algorithms don't scale1 to sequences with hundreds of billions or trillions of points. Recently a team of researchers led by Eamonn Keogh of UC Riverside introduced a set of tools for mining time-series with trillions of points. What is Dynamic Time Warping? SQRT[ Σ (xi - yi)2 ] While ED is easy to define, it performs poorly as a similarity score. There are an exponential number of paths (from one time series to the other) through the warping matrix. 1. 1. 1.
etsy/oculus etsy/skyline Introducing Kale Posted by Abe Stanway | Filed under data, monitoring, operations In the world of Ops, monitoring is a tough problem. It gets harder when you have lots and lots of critical moving parts, each requiring constant monitoring. At Etsy, we’ve got a bunch of tools that we use to help us monitor our systems. This tool is designed to solve the problem of metrics overload. Of course, if a graph isn’t being watched, it might misbehave and no one would know about it. We’d like to introduce you to the Kale stack, which is our attempt to fix both of these problems. Skyline Skyline is an anomaly detection system. You can hover over all the metric names and view the graphs directly. Once you’ve found a metric that looks suspect, you can click through to Oculus and analyze it for correlations with other metrics! Oculus Oculus is the anomaly correlation component of the Kale system. It lets you search for metrics, using your choice of two comparison algorithms… monitoring <3, Abe and Jon