24 March 2012

Hyperdecanting: Better Wine in a Minute, You Impatient Philistine

Hyperdecanting: Better Wine in a Minute, You Impatient Philistine
Former Microsoft CTO and master chef Nathan Myhrvold suggests a method he calls "hyperdecanting". Sounds fancy and high-tech, right? It's basically shorthand for "put your wine in a blender for a minute and it'll taste better".

When Disruptor is not a good fit

As with most new and useful technologies and techniques, it is often easy to overuse them in scenarios where they are not appropriate.  If you have a shiny new hammer, the screws start to look like nails :).  In this blog I want to discuss the requirement for high-frequency, low-latency event dispatch between different threads in a process and use of Disruptor.

When

The Disruptor framework is certainly a good fit where consumers of events need to receive all events that are published.  However I think it is a poor fit if:

1) Subscribers of events do not need to receive all events that are published.  For example, if they are receiving prices for a particular stock, they might only need the last value.  Interestingly, in my domain, the sort of events that occur in high-frequency and that need to be dispatched with low-latency are often events that can be conflated in some form.  Of course, that's not true for everybody and if all subscribers need to receive each and every event, then using Disruptor makes sense.

2) Subscribers can become slow and performance should remain good when some consumers are fast and others are slow.

Why

Lets take the scenario where a single publisher is generating stock prices for a single stock at a rate of 1000 prices per second.  There are three subsribers in the process.  If one of the subscribers blocks for 1 second, then 1000 events will backup in the RIngBuffer.  Subscribers that are not keeping up with the publisher will cause the Java VM to work harder on minor garbage collections as newly created events live longer in the ringbuffer.  In a more terminal scenario, say you size your ringbuffer at 1,000,000 events, if one subscriber blocks for more than 1000 seconds, then the publisher will have to block as well, as the ringbuffer becomes full, affecting the two fast subscribers that were keeping up.  The impact on garbage collection will be even heavier, causing latency jitter.

In this scenario, where events can be conflated, I would rather not use Disruptor, instead I would have the publisher dispatch to each subscriber, via some sort of conflating queue, here is a snippet that makes it clearer:

 public class SingleValueConflatingDispatcher<Subscriber, Event> {  
   private final Subscriber subscriber;                   
   private volatile Event lastEvent;                    
   private SingleValueConflatingDispatcher(Subscriber subscriber) {     
     this.subscriber = subscriber;                    
   }                                    
   public void add(Event event) {                      
     this.lastEvent = event;                       
   }                                    
   public void dispatch() {                         
     subscriber.dispatch(lastEvent);                   
   }                                    
 }  

The publisher thread invokes add() and the subscriber thread invokes dispatch().  For more complex conflation strategies it's likely you'll need to use locks rather than memory barriers to prevent race conditions in such a ComplicatedConflatingDispatcher, however for me that's a small price to pay.

The net effect is that there is no impact from any slow consumer, on other consumers.  The conflation happens on the publisher's thread, rather than queuing up a large number of events.

Mutability

One suggestion to reduce the impact of garbage collection when using Disruptor with slower consumers is to use mutable events for the entries in the ringbuffer.  Its true that mutable events are probably the only way to get to zero-GC, which helps give you the lowest levels of latency jitter possible.  In the scenario, where conflation is possible, I would much rather stick with immutable events.  Where you have lots of complex business logic operating on events downstream, with different developers/teams writing code operating on those events, its just too easy to introduce concurrency bugs with mutable events.  I sleep much easier at night, knowing all events are immutable.  Also while mutable events reduce the effects garbage collection, it does not prevent the publisher from blocking if one subscriber blocks for too long.

However, if your usecase complexity is constrained and/or you have a small, skilled team building it all, then you do have the option of using mutable events.

SpinLocks

If Disruptor is right for your usecase, do watch out overuse of the BusySpinWaitStrategy.  Stating the obvious, only use this when you can afford to burn away a CPU core when the application is doing nothing.  Otherwise you have a range of WaitStrategies available..

20 March 2012

Inspirational Quotes About System Design | Javalobby

Some nice quotes HERE. Particularly like
Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. Antoine de Saint-Exupery, French writer (1900 - 1944)

No more primitives that would be good if done efficiently

Java won't curl up and die like Cobol, insists Oracle • The Register
For the Java Development Kit (JDK) 10 or after, a fundamental change is being discussed: making the Java language Object Oriented. This might see the introduction of a unified type system that turns everything into objects and means no more primitives.

Seven Wastes

Lean Tools: Seeing Waste | Web Builder Zone
Partially Done Work

Also called WIP (Work In Progress), this waste is identified with all partially developed features or stories. Examples of WIP are features which are:

fully specified in a big document, but not developed yet.
Checked in, but not tested yet.
Unit tested, but not yet integrated with the rest of the system.
Integrated, but not yet released or deployed where a customer can use it.

These features are in a limbo, and won't provide any value to the customer until they reach him. Meanwhile, we are paying the burden of a bigger codebase, and accepting the risk that in the time that it takes to complete the feature the goals of the user will have changed, making part of our efforts useless. You won't deliver a Black Friday shopping application in December; the utility curve of other features is not usually so sharp, but the more is shifted into the future, the less its present value.

13 March 2012

JDK 8 backported ConcurrentHashMaps in Infinispan | Javalobby

Feature Toggles

If You Aren’t Using Feature Toggles, Start… Now | Javalobby
Feature toggles create a distinction between deploying your feature & making that feature available for use. They also remove the requirement that to disable a feature, or to go back to ‘old behavior’ you have to rollback your deployment to an older version of code. There are lots of other benefits too, as well as some challenges.

Bottom line though, if you aren’t using these you need to really seriously consider whether they would be a benefit. If you control your software release & you operate a multi-tenant system, and you want to increase the amount of control you have around the features you release, you need to be using these.