Distributed systems aggregate large numbers of heterogeneous components that are subject to failures and asynchrony. To tame such a capricious nature, systems designers resort to non-blocking techniques such as
state machine replication. This approach provides consistent
non-blocking operations to a shared object replicated at a quorum of machines. State machine replication is a classical paradigm to consistently orchestrate concurrency between remote processes in a distributed system, and as such a weapon of choice to manage metadata operations. This approach is at work in many services such as Apache
ZooKeeper, Google
Chubby, or
Open Replica.
The (experimental)
atomic object factory module is an implementation of the state machine replication paradigm over Infinispan. Using the factory is as simple as employing the synchronized keyword in Java: it suffices to call it with a Serializable class, and it wraps for you the dependability, consistency and liveness guarantees of the instantiated object over multiple Infinispan servers. The factory is universal in the sense that it can instantiate an object of any (serializable) class atop an Infinispan cache, making transparently the object replicated and durable, while ensuring strong consistency despite concurrent access.
Basic Usage
Using the AtomicObjectFactory is fairly simple. We illustrate below a base use case. Additional examples are provided in the maven test directories.
AtomicObjectFactory factory =
new AtomicObjectFactory(c1);
// c1 is a cache
Set set = (Set)
factory.getInstanceOf(HashSet.class, "k");
// k is the storage key
set.add("something");
// some call examples
System.out.println(
set.toString())
set.addAll(
set);
factory.disposeInstanceOf(HashSet.class, "set", true);
// to persistently store the object
Limitations & Guarantees
The implementation requires that all the arguments of the methods of the object are Serializable, as well as the object itself. An object created by the factory is atomic provided that the cache which supports it is both synchronous and transactional.
Going Further
White Paper.
The factory is described in Section 4 of the paper titled "On the Support of Versioning in Distributed Key-Value Stores" published at the 33rd IEEE Symposium on Reliable Distributed Systems (SRDS'14). A preprint version of this paper is available at the following
location.
High-level Implementation Details.
We built the factory on top of the transactional facility of Infinispan. In more details, when the object is created, we store both a local copy and a proxy registered as a cache listener. We serialize every call in a transaction consisting of a single put operation. When the call is de-serialized, it is applied to the local copy and, in case the calling process was local, the response value is returned.
