The Disruptor – Lock-free publishing « codeaholics.org In case you’ve been living on another planet, we recently open-sourced our high performance message passing framework. I’m going to give a quick run-down on how we put messages into the ring buffer (the core data structure within the Disruptor) without using any locks. Before going any further, it’s worth a quick read of Trish’s post, which gives a high-level overview of the ring buffer and how it works. The salient points from this post are: The ring buffer is nothing but a big array.All “pointers” into the ring buffer (otherwise known as sequences or cursors) are Java longs (64 bit signed numbers) and count upward forever. (Don’t panic – even at 1,000,000 messages per second, it would take the best part of 300,000 years to wrap around the sequence numbers).These pointers are then “mod’ed” by the ring buffer size to figure out which array index holds the given entry. Basic ring buffer structure The ring buffer maintains two pointers, “next” and “cursor”: Claiming a slot Summary
AdaptNCycleGDFT.pdf Dissecting the Disruptor: Wiring up the dependencies So now I've covered the ring buffer itself, reading from it and writing to it. Logically the next thing to do is to wire everything up together. I talked about multiple producers - they have the producer barrier to keep them in order and under control. I've talked about consumers in a simple situation. Multiple consumers can get a little more involved. The Disruptor paper and the performance tests cover some basic configurations that you might want. Diamond configurationDiamondPath1P3CPerfTest illustrates a configuration which is not too uncommon - a single producer with three consumers. Consumer three might be your business logic, consumer one could be backing up the data received, and consumer two may be preparing the data or something. Diamond configuration using queues In a SEDA-style architecture, each stage will be separated by a queue: (Why does queue have to have so many "e"s? It does look more complicated. How consumer dependencies work in the Disruptor Hmm.
The Secret to 10 Million Concurrent Connections -The Kernel is the Problem, Not the Solution Now that we have the C10K concurrent connection problem licked, how do we level up and support 10 million concurrent connections? Impossible you say. Nope, systems right now are delivering 10 million concurrent connections using techniques that are as radical as they may be unfamiliar. To learn how it’s done we turn to Robert Graham, CEO of Errata Security, and his absolutely fantastic talk at Shmoocon 2013 called C10M Defending The Internet At Scale. Robert has a brilliant way of framing the problem that I’ve never heard of before. He starts with a little bit of history, relating how Unix wasn’t originally designed to be a general server OS, it was designed to be a control system for a telephone network. Which is why he says the key is to understand: The kernel isn’t the solution. Which means: Don’t let the kernel do all the heavy lifting. With a data plane oriented system you can process 10 million packets per second. C10K Problem - So Last Decade The Apache Problem Multi-Core Scalability
Seeing in the Dark: The Rise of Dark Pools, and the Danger Below the Surface :: TabbFORUM - Where Capital Markets Speak How Dark Pools Work A trading pool is a venue where buyers and sellers can be brought together to trade a position in equities, foreign exchange, futures, etcetera. This was traditionally done on the floor of an exchange like the New York Stock Exchange and facilitated by market makers (Patterson, 2012: 7). With the developments in computer technology, this system began to be seen as arcane and has been largely replaced by electronic trading. Electronic trading can be seen as disruptive technology that has had an incredible impact on markets (Neil Crammond, 2013). The move to off-exchange trading has facilitated dark trading. The extent of the lack of knowledge is revealed in a NY Sun article that reported Erik Sirri, Head of the SEC’s Division of Market Regulation as stating that the “SEC believes the [dark pools] are available to all market participants.” [Related: “Can the FBI and SEC Stop Market Manipulation Together?”] “They have spent hundreds of millions of dollars on systems.
Axial Finance - Indicateurs - TD Sequential indicator Tom DeMark L'Indicateur séquentiel (TD Sequential Indicator ®) a été développé par Tom DeMark pour le trading sur le marché du FOREX. Il convient également pour le marché des matières premières et des options. Cet indicateur recherche la contre tendance afin d'anticiper et d'identifier les points de retournement du marché. Il fournit des signaux en intraday et aussi sur les cours journaliers. Généralement, les fréquences utilisées en intraday vont de 1 à 4 heures. Méthode de calcul Le TD Sequentiel Indicator ® est composé de deux figures dites TD Setup et TD Countdown. La figure TD Setup a la durée la plus courte (représentée en couleur verte dans le graphique ci-dessous). La figure TD Countdown commence après la formation d'un TD Setup (représentée en couleur rouge dans le graphique ci-dessous). Comme le TD Setup, le TD Countdown peut être "Perfect" et dans ce cas représenté par une flèche verticale pleine dans le graphique ci-dessous, sinon le coutour de la flèche est simplement affiché. Exemple
Disruptor 2.0 - All Change Please Martin recently announced version 2.0 of the Disruptor - basically there have been so many changes since we first open-sourced it that it's time to mark that officially. His post goes over all the changes, the aim of this article is to attempt to translate my previous blog posts into new-world-speak, since it's going to take a long time to re-write each of them all over again. Now I see the disadvantage of hand-drawing everything. In the old world This is an example of a configuration of the Disruptor (specifically a diamond configuration). The most obvious changes over the last few weeks have been: Updated naming conventionIntegrating the producer barrier into the ring bufferAdding the Disruptor wizard into the main code base. The New World Order You'll see the fundamentals are pretty much the same. Not shown on the diagram is the name change of the items in the RingBuffer - in the old world, we called this Entry, now they're an Event, hence EventProcessor at the other end.
Axel Fontaine: Optional Dependency Strategies for Java Libraries In software, contrary to common belief, . As you gradually accumulate code, little by little the pain sets in. The complexity increases. It gets harder to understand. And eventually, quality suffers. With each dependency you add, you pull in more code, more APIs to learn and more potential bugs. Transitive dependencies take this problem to 11. And yet, dependencies are a necessary evil. Special case: Libraries and Frameworks A library or framework in broad general use (like Flyway ) usually supports a wide range of scenarios for a very diverse set of users. 10+ Jdbc Drivers Spring Jdbc OSGi (Equinox) JBoss VFS v2 JBoss VFS v3 Ant Maven Nobody in their right mind would be keen on pulling all these dependencies in just to use the subset they need. How do we solve this? The answer lies in . There are for dealing with optional dependencies: Create a separate module which depends on the base module and the optional dependency Create a module per optional dependency which the base module depends on.
Exchange Quality Execution Benefits for FX Trading | LMAX Exchange The unique LMAX Exchange business model addresses the fundamental changes happening within the FX market, and solves two key industry problems: Lack of transparency of the true cost of OTC traded FX Lack of precise, consistent & reliable FX trade execution LMAX Exchange delivers conflict free, neutral execution and transparent cost of trade to both, the buy-side and sell-side. LMAX Exchange is not a market-maker, and unlike some ECNs, the open order book is driven by streaming, non ‘last look’ limit orders supplied by top tier banks and institutional liquidity providers. LMAX Exchange - precise, consistent and reliable Unparalleled execution quality - average trade latency is 4 ms more With an internal latency of 500 micro seconds, we deliver consistent and precise quality of execution which radically improves participants’ best execution objectives. close Award-winning FX trading venue - recognised for excellence and innovation more LMAX Exchange - connectivity and access Learn more
The Most Advanced Tool for Analyzing Price Action and Discovering Trading Systems Disruptor by LMAX-Exchange Download here... Discussion, Blogs & Other Useful Links Presentations Introduction to the Disruptor Read This First To understand the problem the Disruptor is trying to solve, and to get a feel for why this concurrency framework is so fast, read the Technical Paper. And now for some words from our sponsors... What is the Disruptor? LMAX aims to be the fastest trading platform in the world. The Disruptor is the result of our research and testing. This is not a specialist solution, it's not designed to work only for a financial application. It works in a different way to more conventional approaches, so you use it a little differently than you might be used to. If you prefer real, live people explaining things instead of a dry paper or content-heavy website, there's always the presentation Mike and Martin gave at QCon San Francisco. What's the big deal? It's fast. Note that this is a log-log scale, not linear. Great What do I do next?