background preloader

Dev Ops

Facebook Twitter

20 Best Real Time Analytic Tools. Web analytic tools helps you to collect, measure and analyze the internet data and the site visitor records for understanding and optimizing web usage. It not only helps you measure and analyses website traffic but it is also a wonderful tool for market & business research. Every website owner wants to know complete stats and information about his/her website and wants see how users are interacting with it. Many tools are present out there which will provide you with website analytics, but not all of the tools are good. So today we have a list of Best Real Time Analytic Tools that will provide visitor data in real time. Google Analytics Advertisement Some weeks ago Google Analytics introduced a new feature that provides you real-time reporting.

Clicky Clicky provides awesome real-time analytics. Woopra Woopra takes real time analytics to another level. Chartbeat This one provides real-time analytics for news publishing and other types of websites. GoSquared Mixpanel Reinvigorate Piwik ShinyStat. RequestReduce brings immediate Yslow and Google Page Speed score improvements not to mention a faster site! Since March I have been working in my “free” time on a framework to reduce the number and size of HTTP requests incurred from loading a web page. In short it merges and minifies css and javascript on your page and automatically sprites and optimizes css background images. All this is done on the fly (with caching) with no code changes or configuration required. All processed and reduced resources are served with far future caching headers and custom ETags.

In August I had something solid enough to push to production on Microsoft’s Gallery platform (what brings home my bacon) which hosts the Visual Studio Gallery, the MSDN Code Samples Gallery, the Technet Gallery and Script Center and many more galleries. In November, RequestReduce was adopted by MSDN and Technet Forums and Search which serve millions of page views a day.

Results on Microsoft Gallery Platform I’m very pleased with the results. Key RequestReduce WebPage Optimizations Look for background images that it can sprite. The Maven Cookbook / Books / Support Sonatype.com. Introducing Mantrid. Back when we were first developing Epio, we started off using HAProxy as our load balancer. HAProxy is a fine piece of software, and performed quite well for a while, but it quickly started having issues as we got more sites. The main issue was that Epio has at least two (and often more) hostname-matching rules for each app we host, and we have rather a large number of apps - numbering in the thousands at the time we're talking about, and more these days. HAProxy has two issues at this scale: One, it's not terribly efficient at matching domains (it looks to do a linear scan through several thousand regular expressions), and two, the state of our sites changes constantly; we were having to reload it every ten to fifteen seconds; this worked surprisingly well (HAProxy has a graceful restart mode), but still meant things were a little delicate.

Thus, one day in May, when the HaProxy setup was having a particularly bad day, I sat down and wrote a simple Eventlet-based load balancer. HAProxy - The Reliable, High Performance TCP/HTTP Load Balancer. Jobscheduler.com. Today's Ops - Dispelling Myths and Giving Facts. 1,000,000 daily users and no cache (Splash 2011) Semantic Versioning. How to get the file path of a Log4J log file? | Coder's Block. Logstash - open source log management. Syslog4j: Complete Syslog Implementation for Java. Graylog2 - Free open source self-hosted log management and exception tracking. .htaccess snippets to optimize your website. All of the snippets below have to be pasted into your .htaccess file, which is located on the root of your Apache server. Waring: Always make sure you have a working backup before editing your .htaccess file!

Force trailing slash Many clients of mine asked me for always having a trailing slash at the end of their urls. Looks like it’s great for SEO. The following snippet will alwyas add a trailing slash to your site urls. <IfModule mod_rewrite.c> RewriteCond %{REQUEST_URI} /+[^\.]+$ RewriteRule ^(.+[^/])$ %{REQUEST_URI}/ [R=301,L] </IfModule> Source: Prevent hotlinking Hotlinking (the act of using images from another site than yours) is unfortunely a common practice which can waste lots of your precious bandwidth. Source: Redirect mobile devices If your site is not using responsive web design yet, it could be very useful to be able to redirect mobile device to a mobile-specific version of your website.

Video: Operations is a Strategic Weapon. Ops Needs their own TDD: Test -Driven Infrastructure. The recently published book "Test-Driven Infrastructure with Chef" is not a proscriptive book as far as tooling, despite having "Chef" in the title. John Arundel a sysadmin and architect just gave an awesome, to-the-point review of the book, which is that even though there's a debate raging over which is better, Puppet or Chef, there's one thing that most people agree on: Operations needs their own TDD. If services are provisioned by programs, then we can - and should - test those programs. Further, we can write them in a test-driven style: start with the description of how the service should behave, and write a test that expresses this. Then write code that makes that test pass.Nelson-Smith argues that this and other techniques from the school of Agile software development are of great benefit to systems administrators building modern automated infrastructures.

--John Arundel Source: GroundWork - The Open Platform for IT Monitoring. Tired of Nagios and Cacti? Try Zabbix. Ganglia Monitoring System. Homepage of Zabbix :: An Enterprise-Class Open Source Distributed Monitoring Solution. Measure Anything, Measure Everything. Measure Anything, Measure Everything Posted by Ian Malpass | Filed under data, engineering, infrastructure If Engineering at Etsy has a religion, it’s the Church of Graphs. If it moves, we track it. Sometimes we’ll draw a graph of something that isn’t moving yet, just in case it decides to make a run for it. In general, we tend to measure at three levels: network, machine, and application. (You can read more about our graphs in Mike’s Tracking Every Release post.) Application metrics are usually the hardest, yet most important, of the three.

Meet StatsD StatsD is a simple NodeJS daemon (and by “simple” I really mean simple — NodeJS makes event-based systems like this ridiculously easy to write) that listens for messages on a UDP port. We like graphite for a number of reasons: it’s very easy to use, and has very powerful graphing and data manipulation capabilities. Why UDP? So, why do we use UDP to send data to StatsD? Measure Anything Here’s how we do it using our PHP StatsD library: List of statsd server implementations « joemiller.me. Statsd is a simple client/server mechanism from the folks at Etsy that allows operations and development teams to easily feed a variety of metrics into a Graphite system.

For more info on statsd read the seminal blog article on Statsd “Measure Anything, Measure Everything”. As would be expected there are statsd clients in many languages. But, there are also many implementations of the statsd server. This is nice because each organization can pick the one that best fits them. For example, a python shop might prefer to deploy a python based statsd instead of Etsy’s original node.js implementation. Also, there are some statsd implementations that diverge from the original design and provide additional features. I could not find a single resource that listed all of the different implementations, so I figured I would try to start one here.

Flickr’s StatsD: Perl. Estatsd (opscode): Erlang. Deprecated: statsite: python. Microcaching: Speed your app up 250x with no new code - Fenn's Thoughts. I recently had the opportunity to help some friends out preparing a content site (wordpress) for a fairly hefty traffic hit. It was potentially going to be a big spike (national radio campaign, time sensitive content, etc) and they particularly didn't want it to go down at the critical time. I put together a fairly typical "fast" PHP architecture: nginx, PHP-FPM, APC, front-end app cluster, load balancer, replicated DB, along with all the mess that comes with it - machine images, replicated filesystem, etc, etc.

Additionally, installed/tested the various appropriate Wordpress Super-Hyper Cache Pro Blitzen 2000+ plugins. After much mucking around, I had an awesome complicated, linearly scalable difficult to manage, app cluster that could scale to the stars very easily develop non-obvious bottlenecks. It turns out that you can throw all of this out and replace it with a 23 line nginx config. How? Concept Changing Data The Config Nginx has a way of dealing with this too: Benchmarks.

Jedi4ever/veewee - GitHub. Vagrant - Welcome. Gearman. Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work. It allows you to do work in parallel, to load balance processing, and to call functions between languages. It can be used in a variety of applications, from high-availability web sites to the transport of database replication events.

In other words, it is the nervous system for how distributed processing communicates. A few strong points about Gearman: Open Source It’s free! Content is being updated regularly, so please check back often. A Gearman powered application consists of three parts: a client, a worker, and a job server. We start off by writing a client application that is responsible for sending off the job and waiting for the result so it can print it out. <? <? This code defines a function my_reverse_function that takes a string and returns the reverse of that string. Now you’re probably asking what if the job server dies?

DevOps for Developers. Puppet Labs: The Leading IT Automation Software Solution. Foreman. Documentation | Puppet Labs - Learning Puppet. The Puppet Learning VM is an interactive tutorial and learning environment to get you started with Puppet or level up the skills you already have. Explore the technology in depth with a series of detailed multi-step quests. Working through the Puppet Enterprise console GUI and Linux command-line, you'll learn the building block concepts of Puppet such as resources, manifests, classes and modules, then move on to more advanced topics including defined resource types and application orchestration. Ready to get started? Complete the registration form to download the Learning VM. Download the zip file containing the Learning VM open virtualization archive (OVA) and setup instructions.

Follow the steps in the instructions to import the Learning VM OVA into your virtualization software and get started with the Quest Guide. Minimum requirements.