background preloader

Celery: Distributed Task Queue

Celery: Distributed Task Queue
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready). Celery is used in production systems to process millions of tasks a day. Getting started

http://www.celeryproject.org/

Related:  FranLorenzo.kred

Nice collection of icon sets for your Linux Desktop with installation instructions This is a nice collection of iconsets for your Linux Desktop, so if you are a fun of changing default themes of your Linux Distribution, then this post is for you. I tried to collect the nicest and most updated icons themes existing in this moment, i included also a detailed howto install the themes for Ubuntu/ LinuxMint and Fedora based distributions. I tested the icon themes in Ubuntu 12.04 Precise Pangolin Beta1, icons did work just fine. Several Solutions for Queue & Worker Systems Almost every dating site we’ve built has a job queue of some kind. We enqueue email sends, statistic updates, logging, fraud detection and more. Anything that doesn’t immediately impact the response we’re preparing for the user can be run in the background. Over the years we’ve tried several different queue/worker systems and we’d like to share our findings to help others decide on a technology. The solutions distinguish themselves in a few key areas: Scalability

Twisted Twisted is an event-driven networking engine written in Python and licensed under the open source ​MIT license. Twisted runs on Python 2 and an ever growing subset also works with Python 3. Twisted makes it easy to implement custom network applications. Here's a TCP server that echoes back everything that's written to it: Replacing Django's View function by re-usable Objects Abstract Never been frustrated doing functional programming in your django views ? This little module is intended to help you use objects for your views : Replace urls parameters and template context by class attributes.Use inheritance to group common task, do specializations, common decorators...Define template tags in your objects, call instance methods.Define your re-usable library. This code is now quite complete and we use it in production environment.

Gone Fishin': Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud All in all this is still my favorite post and I still think it's an accurate vision of a future. Not everyone agrees, but I guess we'll see... "But it is not complicated. [There's] just a lot of it." -- Richard Feynman on how the immense variety of the world arises from simple rules. Contents: alyssa frazee Thu 02 January 2014 | -- (permalink) My sister is a senior undergraduate majoring in sociology. She just landed an awesome analyst job for next semester and was told she'll be using some R in the course of her work. She asked me to show her the ropes during winter vacation, and of course I said yes! What better way to while away the days of a Minnesota winter?!1

Cache Machine: Automatic caching for your Django models Cache Machine hooks into Django’s ORM to provide easy object caching and invalidation. One of our primary goals with the rewrite of addons.mozilla.org was to improve our cache management. Large sites like AMO rely on layers of caching to stay afloat, and caching database queries in memcached is one of our favorite tools. Galaxy Galaxy is a high-performance in-memory data-grid (IMDG) that can serve as a basis for building distributed applications that require fine-tuned control over data placement and/or custom distributed data-structures. What makes Galaxy different from other IMDGs is the way it assigns data items to cluster node. Instead of sharding the data on one of the keys using a consistent hashing scheme, Galaxy dynamically moves objects from one node to another as needed using a cache-coherence protocol similar to that found in CPUs. This makes Galaxy suitable for applications with predictable data access patterns, i.e. applications where the data items behave according to some proximity metric, and items that are “closer” together are more likely to be accessed together than items that are “far” apart.

Open Source Finance 1. QuantLib - An Interview with Luigi Ballabio This is the first in a new series of interviews with leading developers involved in open source financial software. In the first interview I talk to Luigi Ballabio about QuantLib, a free software framework for Quantitative Finance which he has been involved with since it's original launch in the year 2000. Luigi currently participates in the design, maintenance and development of the core C++ library for QuantLib, including the review and management of the contributions from users. In his day job, Luigi is Senior Quantitative Developer at StatPro Italia where he designs and implements the C++ financial libraries used in the StatPro Risk Management (SRM) and Complex Asset Pricing (CAP) applications. In this interview, Luigi also talks about his 3-day training course, Introduction to QuantLib Development, which is next due to take place in London, 22-24th September 2014 and costs £1900 per person + VAT.

Related: