An Asylum podcast on Infinispan 8

If you are of the kind that get you source of info from podcasts, check out the latest JBoss Asylum podcast. Tristan and William give an overview of Infinispan 8 main features and in particular around distributed streams and all the new query features.

Go get the podcast on JBoss Asylum website or in your favorite podcatcher by looking for JBoss Asylum.

Why would you listen to podcasts?

First to learn new things but also to fill gaps where you can't look at a screen and would be borderline bored: walking the dog, running, standing up in the subway, dish washing, vacuum cleaning, you name it.
Don't listen to Infinispan podcasts while operating heavy machinery!

Expiration Enhancements

Infinispan has supported expiration now for quite some time.  However there have always been some nuances with how it operated, and with this latest wave of enhancements, we hope that they are mostly covered.

Existing Behaviours

The following describe how expiration works in certain circumstances with Infinispan 7 or older.

Max Idle

Max Idle, I would say is the black sheep of clustered expiration.  It works great with a single node, but if you have a cluster where a key is accessed on different nodes the recent access time is not in sync.  When a clustered cache is used with max idle this can make some nodes contain data and some others to not, it can be a bit confusing.

Single node expiration

Expiration has only ever took place on a per node basis.  That is that the entry is only removed from a node when it has that key accessed or the reaper thread finds it.  This means that the different nodes can have different amount of entries (although expired entries - don't show up).

Expiration Event?

When an entry expires it should raise an expiration event, correct?  Infinispan didn't have such an event, instead an invalidation event was raised.  This could be for obvious reasons a bit confusing.  The worst part is that the event is raised on each node at possibly different times since the entries aren't removed at approximately the same time.

New Enhancements

The following are new enhancements added with Infinispan 8 to allow for better handling of entries expiring.

Cluster wide expiration 

When an entry in a replicated or distributed cache expires it will now expire that entry across the entire cluster at once.

When an entry expires on one node (either by access or reaper thread) that node will asynchronously send a remove expired command.  This command runs just like a remove except it has some conditional values such as checking for the lifespan and value to make sure they match before actually removing the entry.  This is to prevent a concurrent write from being overwritten.  This then ensures that the entry is removed from all nodes at approximately the same time.

Unfortunately cluster wide expiration is not as safe when an entry expires from a store and it wasn't in memory.  In very rare circumstances, it can overwrite a concurrent update.  This case is very rare because entries only expire from a store when the reaper thread runs, you would then have to have a concurrent put at the precise moment the reaper thread is expiring that entry.  This is because the API for cache store expiration only exposes the key and doesn't include the value or metadata.  This is planned to be enhanced hopefully somewhat soon though!

Yes, Expiration Events!

Infinispan also has added a new event, CacheEntryExpired.  This is fired whenever an entry expires or is removed due to a cluster wide expiration.  Note in the latter case, the event is raised across the entire cluster at the same time, which also includes only receiving a cluster expiration event.

Also there was a bug here where the event was only raised for store expirations, in memory expirations never even raised an event, doh!

Unchanged

Max Idle

Unfortunately, max idle is unchanged in the latest enhancements.  Implementing a correct max idle is a very network costly operation as you would have to send updates to other nodes.  Instead it is highly recommended to not use max idle in a clustered cache as it can cause unexpected behaviour (where an entry may expire early even though it was accessed recently).

Other Remarks

Concurrent expiration access

There is an interesting case when using a clustered cache and you have an expired entry.  If that entry is read from multiple nodes at the same time it can cause more than 1 expiration event to occur.  If this does occur the first event will contain the value (if applicable) and any others will show a null value.

What do I need to change?

The new expiration changes come along for free, no configuration etc. required.  However if you were listening to cache entry invalidation events for entries expiring you should change your Listener to use the new annotation.  Other than that everything should just work!

I have a question or concern

If you have any questions or concerns please get in contact with us!

Infinispan 8.1.0.Alpha2

Dear all,

The second Alpha release of Infinispan 8.1 is now available for use.  Our new admin console has gone through quite a lot of changes recently.  As mentioned before the new console allows for administration of Infinispan cluster nodes and now we would like to show some screenshots of a running server in the below gallery.

We would greatly appreciate your feedback regarding the admin console web application, let's shape it together!

For all other improvements as well as bug fixes you can see the release notes. Infinispan 8.1 Final is still on track for the end of this month. If you are new to Infinispan you can learn how to use it, and help us continually improve it.

• 
• 
• 
• 
• 
•  

Cheers!
Will

Stored Script Execution

One of the questions we get asked a lot is: when will I be able to run Map/Reduce and DistExec jobs over HotRod.

I'm happy to say: now !

Infinispan Server comes with Stored Script Execution which means that remote clients can invoke named scripts on the server. If you're familiar with the concept of Stored Procedures of the SQL world, then you already have an idea of what this feature is about. The types of scripts you can run are those handled by Java's scripting API. Out of the box this means Javascript (which uses either the Nashorn engine on JDK 8+), but you can add many more (Groovy, Scala, JRuby, Jython, Lua, etc). Scripts are stored in a dedicated script cache ("___scriptcache") so that they can be easily created/modified using the standard cache operations (put/get/etc.).

Here's an example of a very simple script:

The script above just obtains the default cache, retrieves the value with key 'a' and returns it (the Javascript script engine uses the last evaluated expression of a script as its return value).
The first line of the script is special: it looks like a comment, but, like the first line in Unix shell scripts, it actually provides instructions on how the script should be run in the form of properties.

The mode property instructs the execution engine where we want to run the script: local for running the script on the node that is handling the request and distributed for running the script wrapped by a distributed executor. Bear in mind that you can certainly use clustered operations in local mode.

Scripts can also take named parameters which will "appear" as bindings in the execution scope.

Invoking it from a Java HotRod client would look like this:

Server-side scripts will be evolving quite a bit in Infinispan 8.1 where we will add support for the broader concept of server-side tasks which will include both scripts and deployable code which can be invoked in the same way, all managed and configured by the upcoming changes in the Infinispan Server console.

Functional Map API: Listeners

We continue with the blog series on the experimental Functional Map API which was released as part of Infinispan 8.0.0.Final. In this blog post we'll be focusing on how to listen for Functional Map events. For reference, here are the previous entries in the series:

1. Functional Map Introduction
2. Working with single entries
3. Working with multiple entries

The first thing to notice about Functional Map listeners is that they only send events post-event, so that means the events are received after the event has happened. In contrast with Infinispan Cache listeners, there are no pre-event listener invocations. The reason pre-events are not available is because listeners are meant to be an opportunity to find out what has happened, and having pre-events can sometimes hint as if the listener was able to alter the execution of the operation, for which the listener is not really suited. If interested in pre-events or potentially altering the execution, plugging custom interceptors is the recommended solution.

Functional Map offers two type of event listeners: write-only operation listeners and read-write operation listeners.

Write-Only Listeners

Write listeners enable users to register listeners for any cache entry write events that happen in either a read-write or write-only functional map.

Listeners for write events cannot distinguish between cache entry created and cache entry modify/update events because they don’t have access to the previous value. All they know is that a new non-null entry has been written. However, write event listeners can distinguish between entry removals and cache entry create/modify-update events because they can query what the new entry’s value via ReadEntryView.find() method.

Adding a write listener is done via the WriteListeners interface which is accessible via both ReadWriteMap.listeners() and WriteOnlyMap.listeners() method. A write listener implementation can be defined either passing a function to onWrite(Consumer<ReadEntryView<K, V>>) method, or passing a WriteListener implementation to add(WriteListener<K, V>) method. Either way, all these methods return an AutoCloseable instance that can be used to de-register the function listener. Example and expected output:

Read-Write Listeners

Read-write listeners enable users to register listeners for cache entry created, modified and removed events, and also register listeners for any cache entry write events. Entry created, modified and removed events can only be fired when these originate on a read-write functional map, since this is the only one that guarantees that the previous value has been read, and hence the differentiation between create, modified and removed can be fully guaranteed.

Adding a read-write listener is done via the ReadWriteListeners interface which is accessible via ReadWriteMap.listeners() method. If interested in only one of the event types, the simplest way to add a listener is to pass a function to either onCreate, onModify or onRemove methods. Otherwise, if interested in multiple type of events, passing a ReadWriteListener implementation via add(ReadWriteListener<K, V>) is the easiest. As with write-listeners, all these methods return an AutoCloseable instance that can be used to de-register the listener.

Here's an example of adding a ReadWriteListener that handles multiple type of events:

Closing Notes

More listener event types are yet to be implemented for Functional API, such as expiration events or passivation/activation events. We are capturing this future work and other improvements under the ISPN-5704 issue.

We'd love to hear from you on how you are finding this new API. To provide feedback or report any problems with it, head to our user forums and create a post there.

In next blog post in the series, we'll be looking into how to pass per-invocation parameters to tweak operations.

Cheers,

Galder

Hibernate Second Level Cache improvements

Infinispan has been implementing Hibernate Second Level Cache for a long time, replacing the previous JBoss Cache implementation with very similar logic. The main aim of the implementation has always been to have very fast reads, keeping the overhead of cache during reads on minimum. This was achieved using local reads in invalidation-mode cache and Infinispan's putForExternalRead operation, where the request to cache never blocks.

Recently we've looked on the implementation again to see whether we can speed it up even more. For a long time you could use only transactional caches to keep the cache in sync with database. However transactions come at some cost so we thought about a way to get around it. And we have found it, through custom interceptors we have managed to do two-phase updates to the cache and now the non-transactional caches are the default configuration. So, if you're using Hibernate with your own configuration, don't forget to update that when migrating to Hibernate ORM 5!

With transactions gone, our task was not over. So far entity/collection caching has been implemented for invalidation mode caches, but it's tempting to consider replication mode, too. For replicated caches, we got rid of a special cache for pending puts (this local cache detects out-of-date reads, keeping the entity cache consistent). Instead, we used different technique where a logical removal from the cache is substituted by replace with a token called tombstone, and updates pre-invalidate the cache in a similar way. This change opened the possibility for non-transactional replicated and distributed caches (transactional mode is not supported). We were pleased to see the results of some benchmark where the high hit ratio in replicated caches has dramatically speeded up all operations.

There is one downside of the current implementation - in replication mode, you should not use eviction, as eviction cannot tell regular entity (which can be evicted) from the tombstone. If tombstone was evicted, there's a risk of inconsistent reads. So when using replicated caches, you should rely on expiration to keep your cache slender. We hope that eventually we'll remove this limitation.

All modes described above give us cache without any stale reads. That comes at a cost - each modification (insert, update or removal) requires 2 accesses to the cache (though, sometimes the second access can be asynchronous). Some applications do not require such strict consistency - and that's where nonstrict-read-write comes to the scene. Here we guarantee that the cache will provide the same result as DB after the modifying transaction commits - between DB commit and transaction commit a stale value can be provided. If you use asynchronous cache, this may be delayed even more but unless the operation fails (e.g. due to locking timeout) the cache will eventually get into a state consistent with DB. This allows us to limit modifications to single cache access per modification.

Note that nonstrict-read-write mode is supported only for versioned entities/collections (that way we can find out which entity is actually newer). Also, you cannot use eviction in nonstrict-read-write mode, for the same reason as in tombstone-based modes. Invalidation cache mode is not supported neither.

If you'll try out the most recent Hibernate ORM, you'll find out that Infinispan 7.2.x is used there. This is because ORM 5.0.0.Final was released before Infinispan 8.0.0.Final went out and we can't change the major version of dependency in micro-release. However, we try to keep Infinispan 8.0.x binary compatible (in parts used by Hibernate), and therefore you can just replace the dependencies on classpath and use the most recent Infinispan, if you prefer to do so.

To sum things up, here is the table of supported configurations:

Concurrency strategy	Cache transactions	Cache mode	Implementation	Eviction
transactional	transactional	invalidation	pending puts	yes
read-write	non-transactional
	non-transactional	replicated/distributed	tombstones	no
nonstrict-read-write			versioned entries

There's also the read-only mode - this can be used instead of both transactional or read-write modes, but at this point it does not offer any further performance gains, since we have to make sure that you don't see a removed value. Actually, it also does not matter whether you specify transactional or read-write mode; the proper strategy will be picked according to your cache configuration (transactional vs. non-transactional).

We hope that you'll try these new modes and many consistency fixes included along (you should use Hibernate ORM 5.0.2.Final or later), and tell us about your experience.

Happy caching!