Wednesday 13 May 2009

What's so cool about an asynchronous API?

Inspired by some thoughts from a recent conversation with JBoss Messaging's Tim Fox, I've decided to go ahead and implement a new, asynchronous API for Infinispan.

To sum things up, this new API - additional methods on Cache - allow for asynchronous versions of put(), putIfAbsent(), putAll(), remove(), replace(), clear() and their various overloaded forms. Unimaginatively called putAsync(), putIfAbsentAsync(), etc., these new methods return a Future rather than the expected return type. E.g.,


V put(K key, V value);
Future<V> putAsync(K key, V value);

boolean remove(K key, V value);
Future<Boolean> removeAsync(K key, V value);

void clear();
Future<Void> clearAsync();

// ... etc ...

You guessed it, these methods do not block. They return immediately, and how cool is that! If you care about return values - or indeed simply want to wait until the operation completes - you do a Future.get(), which will block until the call completes. Why is this useful? Mainly because, in the case of clustered caches, it allows you to get the best of both worlds when it comes to synchronous and asynchronous mode transports.

Synchronous transports are normally recommended because of the guarantees they offer - the caller always knows that a call has properly propagated across the network, and is aware of any potential exceptions. However, asynchronous transports give you greater parallelism. You can start on the next operation even before the first one has made it across the network. But this is at a cost: losing out on the knowledge that a call has safely completed.

With this powerful new API though, you can have your cake and eat it too. Consider:



Cache<String, String> cache = getCache();
Future<String> f1 = cache.putAsync(k1, v1);
Future<String> f2 = cache.putAsync(k2, v2);
Future<String> f3 = cache.putAsync(k3, v3);

f1.get();
f2.get();
f3.get();


The network calls - possibly the most expensive part of a clustered write - involved for the 3 put calls can now happen in parallel. This is even more useful if the cache is distributed, and k1, k2 and k3 map to different nodes in the cluster - the processing required to handle the put operation on the remote nodes can happen simultaneously, on different nodes. And all the same, when calling Future.get(), we block until the calls have completed successfully. And we are aware of any exceptions thrown. With this approach, elapsed time taken to process all 3 puts should - theoretically, anyway - only be as slow as the single, slowest put().

This new API is now in Infinispan's trunk and yours to enjoy. It will be a main feature of my next release, which should be out in a few days. Please do give it a spin - I'd love to hear your thoughts and experiences.

10 comments:

  1. This makes perfect sense and is a breath of fresh air, Manik. Finally, we recognise by design that we can do the dishes while waiting for laundry to dry :)

    ReplyDelete
  2. Can you make the Future typed, e.g.
    Future[T] putAsync(K key, V value);

    So I can fetch the result without a downcast to my type

    ReplyDelete
  3. Bela, this is already the case in source code. ;-)
    I think it's just the www/html issue, same as yours [T] (which should be [V] ;-))

    ReplyDelete
  4. @Bela Ban

    Yes, the futures are typed. Damn blog authoring software! :-) Let me try and fix the post.

    ReplyDelete
  5. Why not also add support for a listener on Future that can be informed when the put is complete?

    ReplyDelete
  6. A reasonable request (current notifications are only for local puts). Feel like creating a JIRA with your suggestion?

    ReplyDelete
  7. @Gary if you wish to follow the discussion/design around async notifications, join infinispan-dev. I just posted:

    http://lists.jboss.org/pipermail/infinispan-dev/2009-May/000163.html

    ReplyDelete
  8. I understand, Oracle as SpecLead of JSR-107 may not let it die now that they took control over Sun, but as EC member of the JCP I know, a JSR that never even released an EDR since 2001 (!!) is a very bad sign for the community. Hence it is inactive.

    Any plans to help changing that from your side?

    ReplyDelete
  9. @keilw_id I agree that the JSR has been very slow in releasing stuff. Greg from EHCache and myself are in the process of trying to move things forward again - no promises as to what will happen, but we are trying.

    ReplyDelete