Shipping nginx access logs to LogStash — DevOps & Programming Note: this blog post is a bit outdated. Kibana is now part of Elasticsearch, and a pure JavaScript project, no longer dependent on Ruby. Additionally we now run Elasticsearch version 0.9.11 and LogStash 1.3.3. At Commando.io, we’ve always wanted a web interface to allow us to grep and filter through our nginx access logs in a friendly manner. LogsStash is a free and open source tool for managing events and logs. First, let’s setup our centralized log server. The following guide assumes that you are running CentOS 6.4 x64. Centralized Log Server cd $HOME # Add the required prerequisite remi yum repositorysudo rpm —import rpm -Uvh -i ‘0,/enabled=0/s//enabled=1/’ /etc/yum.repos.d/remi.repo # Install Redis and autostartsudo yum -y install redissudo service redis startsudo chkconfig redis on # Create LogStash configuration filecd /etcsudo touch logstash.conf Nginx Servers
Comprendre les APIs Pomme d’api, apiculture, Aisne Plastique Industrie, api birthday ? Vous n’y êtes pas… La notion que je souhaite aborder aujourd’hui est celle d’Application Programming Interface. Qu’est-ce donc qu’une API ? A quoi cela peut-il servir ? Définition La plupart des définitions disponibles aujourd’hui sur le web abordent la notion d’API par sa dimension technique : on apprend ainsi sur wikipedia qu’une API est « un ensemble de fonctions, procédures ou classes mises à disposition par une bibliothèque logicielle, un système d’exploitation ou un service ». Je vous propose de dépasser ces aspects techniques et de vous livrer les définitions suivantes : Une API, c’est ce qui permet à deux systèmes informatiques totalement indépendants de se parler de façon automatique. Quelques exemples simples : Les APIs SMS , comme celle d’Orange, sont proposées par tous les opérateurs : elles vous permettent de savoir comment envoyer de façon automatisée des campagnes de SMS. A quoi cela sert-il ? Conclusion :
InfoSphere Streams Quick Start Edition Real -time analytic processing at your fingertips Stream processing technology gives you the ability to analyze massive data volumes quickly, often in real-time, and turn data into actionable insight. InfoSphere Streams is an advanced computing platform that can quickly ingest, analyze and correlate information as it arrives from thousands of real-time sources. Now you can get your hands on this technology with InfoSphere Streams Quick Start Edition, a free, downloadable, non-production version. With InfoSphere Streams Quick Start Edition, there is no data capacity and no time limitation, so you can experiment with large data sets and work with different use cases, on your own timeframe. NOTE: InfoSphere Streams Quick Start Edition does not come with a support option. InfoSphere Streams Quick Start Edition includes: Comprehensive Development Tools: A variety of tools for developing and debugging analytic applications that have a high volume and variety of data types, including: Resources
Fathom Mobile Command Center | CrittercismCrittercism Set alert thresholds for proactive notifications Integrated and comprehensive app performance alerting Why it matters: Be proactively identified about issues. Identify issues before they affect end-user experience. Set alerts based on any metric—latencies, errors, crashes, exceptions, data bandwidth used to correlate issues
★ Pour ne plus être en REST, comprendre cette architecture Depuis quelques mois, j'apprends j'essaye d'apprendre REST et j'ai lu de nombreuses pages sans pour autant trouver d'explication simple, à la portée de tous. Du coup quand j'essaye d'expliquer les mécanismes et les avantages professionnellement, c'est pas toujours convaincant... et je voulais justement écrire un petit billet pour mettre tout ça au clair. Et puis je suis récemment tombé sur une série d'explications par Softies on Rails se basant sur des exemples concrets. En voici la traduction chronologique. Comprendre les ressources D'une certaine manière, apprendre REST fut très difficile pour moi. Si vous essayez de comprendre REST, cette série est faite pour vous. Les experts en HTTP, HTML et REST pourraient chercher la petite bête avec ma terminologie simplifiée me permettant d'aller droit au but. Premier point Commençons par le commencement. Et c'est à peu près tout. Deuxième point Remarquez comme j'ai glissé le mot « ressource » alors que vous ne l'attendiez pas. Troisième point Ugh.
horovits/rt-analytics Application Monitoring in Realtime - Instrumental AxonFramework DirectCompute’s unpopularity - Blog - StreamComputing In the world of GPGPU we have currently 4 players: Khronos OpenCL, NVIDIA CUDA, Microsoft DirectCompute and PathScal ENZO. You probably know CUDA and OpenCL already (or start reading more articles from this blog). ENZO is a 64bit-compiler which serves a small niche-market, and DirectCompute is built on top of CUDA/OpenCL or at least uses the same drivers. Edit 2011-01-03: I was contacted by Pathscale about my conclusions about ENZO. A while ago there was an article introducing OpenCL by David Kanter who claimed on page 4 that DirectCompute will win from CUDA. Judging by history though, OpenCL and DirectCompute will eventually come to dominate the landscape, just as OpenGL and DirectX became the standards for graphics. I twittered that I totally disagreed with him and in this article I will explain why I think that. If we ask Google Trends then we get the below graph. Microsoft-specific products are discussed on MSDN, so I checked there too. OS-support is also more and more important.
The error app. Bash Reference Manual Table of Contents This text is a brief description of the features that are present in the Bash shell (version 4.4, 7 September 2016). The Bash home page is This is Edition 4.4, last updated 7 September 2016, of The GNU Bash Reference Manual, for Bash, Version 4.4. Bash contains features that appear in other popular shells, and some features that only appear in Bash. This manual is meant as a brief introduction to features found in Bash. 1 Introduction 1.1 What is Bash? Bash is the shell, or command language interpreter, for the GNU operating system. Bash is largely compatible with sh and incorporates useful features from the Korn shell ksh and the C shell csh. While the GNU operating system provides other shells, including a version of csh, Bash is the default shell. 1.2 What is a shell? At its base, a shell is simply a macro processor that executes commands. A Unix shell is both a command interpreter and a programming language. 2 Definitions blank builtin \a
Build your own data-stream mining NSA in the cloud with “FunnelCake” There's more to "big data" than just lots of bits on disks. Some things you can't just store in the raw; others you need to analyze and process before they ever hit a disk. That way they can be acted upon in near-real time—like trying to pick specific communications sessions from the data stream of a network tap into an Internet backbone, for example. Stream processing, also known as Complex Event Processing, is the real-time querying, analysis, and conversion of information within torrents of live data. But the need for stream processing isn't unique to intelligence organizations like the National Security Agency (NSA). Riders on the Storm BrightContext CEO John Funge and CTO Leo Scott started working on the ideas behind the service in 2010. Part of the answer to that question arrived in 2011, when Twitter acquired data analytics startup BackType and then published a big part of BackType's technology as free and open source software. A series of funnels Listing image by Lorax