2017: A busy year for Infinispan evangelisation!

It's been a very busy year from an Infinispan evangelisation perspective, so before the year ends, we wanted to round up the conferences and user groups that featured Infinispan related talks. This is a good opportunity to catch up on what we've been showing in our Infinispan outings:

Patterns d’utilisation de systèmes in memory 

by Emmanuel Bernard and Galder Zamarreño
@ Devoxx France

This was our first outing of the year with Emmanuel and I showing the audience at Devoxx France the different use cases for data grids. This was also the first time we showed swiss train related demos which we'd improve throughout the year. Slides can be found here and video below:

Big Data In Action with Infinispan

by Galder Zamarreño
@ Great Indian Developer Summit
@ Berlin Buzzwords
@ Red Hat DevNation Live

In this talk, I focused on the big data related use cases for Infinispan. I showed how Infinispan could be used for near real-time data processing using Continuous Querys, and how to analise data using distributed Java Streams. Slides can be found here and the demo here. The video from the DevNation talk can be found here:

HACEP: Highly available and horizontally scalable complex event processing

by Ugo Landini, Sanne Grinovero, Andrea Leoncini, Andrea Tarocchi
@ Red Hat Summit

In this hands on lab, Ugo et al showed how to build a highly available and scalable complex event processing engine with Red Hat JBoss BRMS and and Red Hat JBoss Data Grid (the commercial supported version of Infinispan). This workshop resulted in unmissable architecture reference guide.

Learn how to build Functional Reactive Applications with Elm, Node.js and Infinispan

by Galder Zamarreño
@ Great Indian Developer Summit
@ J On The Beach

The aim of this talk is to demonstrate how Infinispan's Node.js client could be combined with Elm and Infinispan server instances to build a functional reactive application. The talk focused on how build a CRUD application using Elm and Node.js technologies and using Infinispan in-memory data grids for storing and querying information. Slides can be found here:

Streaming Data Analysis with Kubernetes

by Galder Zamarreño

@ Basel One

This talk is a spin off of the big data in action talk which focuses on exploring in greater detail how streaming data can be analysed within a Kubernetes orchestrated cluster. The demo can be found here and the slides here:

Data grids : descubre qué esconden los datos

by Galder Zamarreño

@ Madrid Java User Group

This talk is a Spanish spin off the analytics part of the big data in action talk. Not only did it showed how to use distributed Java Streams to analyse data with Infinispan, but also demonstrated the capabilities of Spark/Hadoop integrations. The demos can be found here and here, and the slides here:

Streaming Data : ni pierdas el tren, ni esperes en balde

by Galder Zamarreño

@ Codemotion Madrid

A Spanish version of the streaming data analysis talk delivered in Basel One.

This talk was recorded, so hoping for the video to be available soon. 
Update (2018/02/02): The video is now available here:

In the mean time, the demo can be found here and the slides here:

Streaming Data Workshop

by Katia Aresti, Thomas Segismont and Galder Zamarreño

@ Duchess France Meetup

@ Devoxx Belgium

@ Codemotion Madrid

A hands-on workshop showing how to work with streaming data with Infinispan and Vert.x, on top of OpenShift container platform. No videos were recorded but you can follow this workshop step by step following instructions here. The Github repository can be found here and the latest slides here:

There's plenty the to keep you entertained between now and the new year! :)

If you have any doubts about the talks above, or you encounter any issues running the demos, do not hesitate contacting us! In the mean time, the Infinispan team is working hard on Infinispan 9.2 as well as improving our OpenShift integration.

Cheers,

Galder

Infinispan 9.1.4.Final Released

Dear Infinispan Community,

We're excited to announce the release of Infinispan 9.1.4.Final, which can be found on our download page.

Full details of the various fixes included in this release can be found here.

Infinispan 9.2.0.Beta2 Released

Dear Infinispan Community,

We're excited to announce the release of Infinispan 9.2.0.Beta2, which can be found on our download page.

The highlights of 9.2.0.Beta2 are:

• MultiMaps are now available over hotrod [ISPN-7887].
• Clustered Counters are now available in the server and can be managed via the management api [SPN-8376] or the web console [ISPN-7927].
• Biased reads in scattered cache [ISPN-8458].
• Conflict resolution on partition merges are now implemented for DENY_READ_WRITES and ALLOW_READS strategies [ISPN-8440].

Full details of the new features and enhancements included in this release can be found here.
Thank you for following us and stay tuned!

The Infinispan Team

First steps with Vert.x and Infinispan - PUSH API (Part 2)

Welcome to the second in a multi-part series of blog posts about creating Eclipse Vert.x applications with Infinispan. In the previous blog post we have seen how to create a REST API. The purpose of this tutorial is to showcase how to create a PUSH API implemented with Vert.x and using Infinispan as a server.

All the code of this tutorial is available in this GitHub repository. The backend is a Java project using Maven, so all the needed dependencies can be found in the pom.xml. The front is a super simple react application.

PUSH API

Creating a REST API is very straightforward. But today, even if we are heavily using REST, we don't always want to use request/response or polling, but instead we want to push directly from the server to the client. In this example, we are going to create an API that pushes every new value inserted in the default cache of Infinispan. These values are cute names, as we did in the REST API example.

We are using two features here :

• Infinispan client listeners
• Vert.x bridge between the Event Bus and the browser

Infinispan Listeners provide a way to the client get notified when something happens in a cache.

The Event Bus Bridge that connects to the browser, uses SockJS. SockJS is a JavaScript library that provides a WebSocket API, but it can be used with browsers that don't support web-sockets (see the website of the project for more detailed information). Vert.x supports this library and creates a bridge between your browser and your back-end easily through the Event Bus.

Creating an Event Bus bridge

Vert.x is a reactive framework, which means that uses RxJava too, and provides a fancy API on top of it.

First, we are going to create a new verticle called SendCuteNamesAPI. This verticle extends the CacheAccessVerticle we created in the previous blog post. CacheAccessVerticle initialises the connection with Infinispan using the Hot Rod protocol.

Now we need to create a SocketJSHandler. This handler has a method called bridge, where we configure some BridgeOptions. Obviously we don't want the client to be able to read everything traveling on the event bus, and this won't happen. We configure an address, 'cute-names', and we add the permission to read and write to this address.

This handler is passed to the event bus route, where the path is /eventbus/*.

Finally, we create a http server as we did in the REST API example. The difference is that instead of calling listen method, we call rxListen and subscribe.

Getting notified and publishing

Using Infinispan listeners is very easy.

First, we are going to create a class that has the @ClientListener annotation. The client listener has to be added to the cache client configuration. We add a protected method called addConfigToCache that will be called just after the initialisation of the defaultCache in the abstract CacheAccessVerticle. Verticles extending the abstract class can now add custom configuration to the client.

We want to be notified when a new entry is created. In this case, our listener has to contain a method with the @ClientCacheEntryCreated annotation on it. The signature of the method has to include a ClientCacheEntryCreatedEvent<String> parameter. This parameter will hold the 'key' of the entry that has been created.

Finally, we use the key to retrieve the name using the getAsync method and then publish the value in the Vert.x event bus to the address where the socket listener is permitted to read : cute-names.


Now we can run the main method and whenever we post a new name, we will see in the logs that the client listener is notified!

Client code

We are going to create a super simple react application that will just display hello. React community is *huge*, so there are lot's of tutorials out there to create a hello world client application. This application has a single component that displays "Hello".

The react application runs calling npm install and npm start  in http://localhost:9000/.

Now we need to connect the client to the backend with SockJS. Vert.x provides a JavaScript library for that: vertx3-eventbus-client, built on top of SockJS. We create an EventBus object that will connect to http://localhost:8082/eventbus as we configured in the SendCuteNamesAPI. We register a handler on the 'cute-names' address. The body of the message will contain the new cute name published in the event bus. Every time the handler is called, we update the component's state, and it will be rendered.

Wrap up

We have learned how to create PUSH APIs with Vert.x, powered by Infinispan. The repository has some unit tests. Feedback is more than welcome to improve the code and the provided examples. I hope you enjoyed this tutorial ! On the next tutorials we will talk about Infinispan as the cluster manager for Vert.x. Stay tuned !

First steps with Vert.x and Infinispan - REST API (Part 1)

Welcome to the first in a multi-part series of blog posts about creating Eclipse Vert.x applications with Infinispan. The purpose of this first tutorial is to showcase how to create a REST API.

All the code of this tutorial is available in this GitHub repository. The backend is a Java project using Maven, so all the needed dependencies can be found in the pom.xml.

What is Vert.x ?

Vert.x is a tool-kit for building reactive applications on the JVM. It’s an event driven and non blocking tool-kit. It is based on the Reactor Pattern, like Node.js, but unlike Node it can easily use all the cores of your machine so you can create highly concurrent and performant applications. Code examples can be found in this repository.

REST API

Let’s start creating a simple endpoint that will display a welcome message on '/'. In Vert.x this is done by creating a Verticle. A verticle is a unit of deployment and processes incoming events over an event-loop. Event-loops are used in asynchronous programming models. I won't spend more time here explaining these concepts as this is very well done in this Devoxx Vert.x talk or in the documentation available here.

We need to override the start method, create a 'router' so '/' requests can be handled, and finally create a http server.

The most important thing to remember about vert.x, is that we can NEVER EVER call blocking code (we will see how to deal with blocking API's just after). If we do so, we will block the event loop and we won't be able to serve incoming requests properly.



Run the main method, go to your browser to http://localhost:8081/ and we see the welcome message !

Connecting with Infinispan

Now we are going to create a REST API that uses Infinispan. The purpose here is to post and get names by id. We are going to use the default cache in Infinispan for this example, and we will connect to it remotely. To do that, we are going to use the Infinispan hotrod protocol, which is the recommended way to do it (but we could use REST or Memcached protocol too)

Start Infinispan locally

The first thing we are going to do is to run an Infinispan Server locally. We download the Infinispan Server from here, unzip the downloaded file and run ./bin/standalone.sh. 

If you are using Docker on Linux, you can use the Infinispan Docker Image Available easily. If you are using Docker for Mac, at the time of this writing there is an issue with internal IP addresses and they can't be called externally. Different workarounds exist to solve the problem, but the easiest thing for this example is simply downloading and running the standalone server locally. We will see how to use the docker image in Openshift just after.

The hotrod server is listening in localhost:11222.

Connect the client to the server

The code we need to connect with Infinispan from our java application is the following :



This code is blocking. As I said before, we can't block the event loop and this will happen if we directly call these API's from a verticle. The code must be called using vertx.executeBlocking method, and passing a Handler. The code in the handler will be executed from a worker thread pool and will pass the result back asynchronously.
Instead of overriding the start method, we are going to override start(Future<Void> startFuture). This way, we are going to be able to handle errors.

To stop the client, the API supplies a non blocking method that can be called when the verticle is stopped, so we are safe on that.

We are going to create an abstract CacheAccessVerticle where we will initialise the manager and get default cache. When everything is correct and the defautCache variable is initialised, we will log a message and execute the initSuccess abstract method.

REST API to create names

We are going to add 3 new endpoints.

• GET /api displays the API name
• POST /api/cutenames creates a new name
• GET /api/cutenames/id displays a name by id

CuteNamesRestAPI verticle can now extend this class and override the initSuccess method instead of the start method.

POST

Our goal is to use a curl to create a name like this :

curl -X POST \ 
-H "Content-Type: application/json" \
-d '{"name":"Oihana"}' "http://localhost:8081/api/cutenames"

For those that are not familiar with basques names, Oihana means 'rainforest' and is a super cute name. Those who know me will confirm that I'm absolutely not biased making this statement.

To read the body content, we need to add a body handler to the route, otherwise the body won't be parsed. This is done by calling router.route().handler(BodyHandler.create()).

The handler that will handle the post method in '/api/cutenames' is a RoutingContext handler. We want to create a new name in the default cache. For that, we will call putAsync method from the defaultCache.

The server responds 201 when the name is correctly created, and 400 when the request is not correct.

GET by id

To create a get endpoint by id, we need to declare a route that will take a parameter :id. In the route handler, we are going to call getAsync method.



If we run the main, we can POST and GET names using curl !

 curl -X POST -H "Content-Type: application/json" \
 -d '{"id":"42", "name":"Oihana"}' \ 
"http://localhost:8081/api/cutenames" 

Cute name added 

 curl -X GET -H "Content-Type: application/json" \ 
"http://localhost:8081/api/cutenames/42"

{"name":"Oihana"}

Wrap up

We have learned how to create a REST API with Vert.x, powered by Infinispan. The repository has some unit tests using the web client. Feedback is more than welcome to improve the code and the provided examples. I hope you enjoyed this tutorial ! On the next tutorial you will learn how to create a PUSH API.

Back from Madrid JUG and Codemotion Madrid!!

We've just come back from our trip to Spain and first of all, we'd like to thank everyone who attended our talks and workshops at Madrid Java User Group and Codemotion Madrid as well organisers and sponsors who made it possible!

We had a very hectic schedule, which started with a Red Hat double bill for Madrid JUG. Thomas Segismont started the evening with Vert.x talk and Galder followed up with a talk on how to do data analytics using Infinispan-based data grids.

In the data analytics talk, Galder focused on how to use distributed Java Streams to do analytics and also showed how to use Infinispan Spark connector when Java Streams are not enough. The distributed Java Streams demo he ran can be found here. The most relevant files of that demo are:

• Live coding instructions, follow them to do the same steps he did in the talk.
• Server task solution class using distributed Java Streams.

Galder also demonstrated how to use Infinispan Spark connector by showing the Twitter example. The slides from this talk (in Spanish) can be found here:


Next day on Friday, Galder gave a talk at Codemotion Madrid on working with streaming data with Infinispan, Vert.x and OpenShift. For the first time he was running it all on top of Google Cloud, so he could finally free up my laptop from running the demos and take advantage of the power of a cloud provider!

The demo can be found here where you can also find instructions on how to run it on top of Google Cloud. If you want to follow the same steps he followed during the talk, live coding instructions are here. The slides from this talk (in Spanish) can be found here:


Finally on Saturday we delivered the Streaming Data workshop at Codemotion Madrid. Once again, basing our workshop on top of Virtual Box still caused us some issues, but people managed to get through it. We have some plans for next year to avoid the need of Virtual Box, stay tuned!

We have added more detailed instructions on how to run the workshop in your office or at home, so if you're interested in going through it, make sure you check these steps and let us know how they work for you:

This trip to Madrid wraps up a very intense year in terms of promoting Infinispan! Next month we'll be doing a recap of the talks, videos...etc so that you can catch up with them in case you missed any of them :)

Katia & Galder
Un saludo!

Infinispan and Vert.x coming to Codemotion Spain and Madrid JUG

I'm happy to announce that Infinispan and Eclipse Vert.x team members are in Madrid (Spain) this week!

The 23th of November, Galder (Infinispan) and Thomas (Vert.x) will be presenting at Madrid Java User Group two talks about Infinispan and Ver.x. From my side, I've been invited to give a talk about programming career path to Computer Science and Engineering students in the Universidad San Pablo CEU. 

The following days, 24th and 25th of November, Codemotion Spain will take place in Madrid. Codemotion is the biggest developer centric conference in Spain. More than 1.500 people will be attending this year. The conference does not focus on any particular technical stack. You can learn about mobile, web, front-end, back-end, devops and there are some inspirational tracks too during the two days of the conference.

On Friday, Galder will be presenting a talk about Streaming and BigData architecture. Thomas will be presenting "Better performance on server side with HTTP/2" and I will join a debate around Enterprise Culture, where I will give my insights about RedHat's Culture.
On Saturday we will run our Streaming Data Workshop: a hands on lab that showcases a simplified streaming architecture built on top of Infinispan and Vert.x and running on OpenShift. If you cannot attend any of these events, you can always do the workshop at your self peace.

We hope to see you in Madrid, or anywhere else in 2018!!

Cheers,
Katia 

Hotrod clients C++/C# 8.2.0.Alpha2 are out!

Dear Infinispanners,

we're pleased to announce the C++ and C# 8.2.0.Alpha2 releases.

Code for continuous queries (CQ) and SASL is almost complete. Worth mentioning improvements for Alpha2 are:

• added xunit framework for build verification testing (googletest)
• CQ for C#
• EXTERNAL mech for SASL on windows (C++ and C#)
• more bugs resolved than added :)

Check the release notes, browse the source code (C++, C#) or just download and try it!
Cheers,
The Infinispan Team

Merci Duchess et Devoxx!

We've had two very hectic weeks delivering the Infinispan/Vert.x/OpenShift Streaming Data workshop in Duchess France and Devoxx Belgium. First of all, thanks to all attendees for taking the time to attend this workshop and working through it, thanks Duchess and Devoxx for letting us present the workshop, and thanks the sponsors for making it happen!

In case you want to walk through the workshop at your own pace, the version of the workshop delivered at Duchess and Devoxx can be found here. The slides from the workshop can be found here.

The aim of the workshop was to show how to work with multiple data streams and combine them in interested ways. These two streams of data came from Transport API at OpenData.ch and SBB.ch respectively. The end-game of the workshop was to see to combine both sets of information to produce a map that tracked positions of delayed trains around Switzerland.

On 2nd November we did a dry run of the workshop in the Duchess France group. This was a very compact group of people, around 10-12, which gave us thve first taste of what it was like to work through it. In particular, we realised that getting Virtual Box to run on each attendees laptop took some time to get working. From the workshop perspective, we found that the exercises themselves were a bit too long for the people to complete.

With all the invaluable input, Katia, Thomas and myself got ready for delivering the workshop at Devoxx Belgium, one of the top developer conferences in Europe. The pressure was on! We had ~60 attendees, so we had to make sure everyone could progress.



At Devoxx, we were more prepared to help out with set up problems, so everyone got to the starting point much faster. After working through the initial exercise to get used to Infinispan, Vert.x and OpenShift, attendees got on with the workshop itself.

During the workshop exercises, the learnt about Infinispan's in-memory data grid capabilities, and in particular about continuous query which is very well suited for feeding data to live updating maps or dashboards. They also learnt about how easy it is to build reactive applications with Vert.x and RxJava. Finally, they learnt how to deploy applications to OpenShift, how to monitor their progress...etc.

A majority of Devoxx attendees managed to get to the end of the workshop which was a success for us, but we listen to their feedback and we will continue to improve the content and delivery as we prepare for Codemotion Madrid where we are delivering the workshop once again.

Thanks attendees, Devoxx, Duchess and sponsors!!

Cheers,
Galder

Infinispan 9.2.0.Beta1 and 9.1.3.Final have been released

Dear Infinispan Community,

Two new releases are ready for you today, our newest and greatest 9.2.0.Beta1 and an updated Final of our stable branch, 9.1.3. Both can be found as usual on our download page.

The highlights of 9.2.0.Beta1 are:

• New API for querying using Ickle via the REST server, taking advantage of the JSON/Protobuf interoperability [ISPN-8113].
• Clustered Locks, first experimental release. A ClusteredLock is a data structure used for concurrent programming between Infinispan instances in cluster mode.
• Clustered Counters have received serveral improvements. It's now possible to manage your counters via JMX [ISPN-7926] and other management methods are on the way. The strong counter has a new method: compare-and-swap [ISPN-8489]. If you used compare-and-set, the compare-and-swap may improve your code performance. And last but not least, a new sync() method was added to the strong and weak counters, useful if you don't want to handle the CompletableFuture!
  
• Clustered Cache Configuration: Both EmbeddedCacheManager and RemoteCacheManager now have an API for creating/removing caches across the cluster [ISPN-7776]. Caches created with this API will also be created on any new members that will join the cluster. Additionally, the configuration for caches created in this way can be made persistent so that they are automatically recreated when a cluster restarts.
• Server deployment of Lucene analyzers for remote query [ISPN-7714].
  

9.1.3.Final and 9.2.0.Beta1 both add several Off-heap improvements and fixes.

Full details of the new features and enhancements included in these two releases can be found here and here.

We have even more exciting stuff coming up for Infinispan 9.2. Thank you for following us and stay tuned!

The Infinispan Team

Infinispan coming to Duchess France and Devoxx Belgium !

This week, 2nd November, Galder and I will be presenting at Duchess France Meetup in Paris our "Streaming Data Workshop" : a 3h hours hands on lab that showcases a slightly simplified streaming architecture built on top of Infinispan and Vert.x and running on OpenShift. 

If you are in Paris, the Duchess France workshop is actually full; if you are planning to assist Devoxx Belgium this year, the workshop will take place the 6th of November there.


Cheers,
Katia 

Infinispan 9.2.0.Alpha2 & 9.1.2.Final released

Dear Infinispan Community,

We have 2 new releases ready for everyone. We have our newest and greatest, 9.2.0.Alpha2 and an updated Final for our stable branch 9.1.2. As usual, both can be found on our download page.

9.2.0.Alpha2 contain the following new features

• New Counters can be removed ISPN-8093
• Interoperability between JSON and Protobuf, allowing to retrieve Protobuf content (written via Hot Rod) as JSON via Rest; also JSON documents written via Rest can be automatically converted to Protobuf and indexed. ISPN-7422
• Wildcard cache configurations allowing for multiple caches from 1 config ISPN-8379
• Reactive streams based iterator implementation providing better throughput and less resources usage ISPN-7865
• An experimental evalAll like method on the LockedStream interface allowing for exclusive operation on entries and a return value ISPN-8310

9.1.2.Final and  9.2.0.Alpha both added

• Updates to third party JGroups, Hibernate and Hibernate Search
• Off heap improvements and fixes for Query & Encoding

Besides these there were also quite a few bug fixes and code clean ups. Full details of the new features and enhancements included in this release can be found here and here.

We have even more exciting features still coming up for Infinispan 9.2. Thank you for following us and stay tuned!

The Infinispan Team

Thanks Basel One 2017!

Thanks a lot to all attendees at Basel One 2017, it was great fun presenting about Infinispan, OpenShift and Vert.x topics just around the corner from my house :)

The slides for my talk can be found here. The code and scripts demo that I showed can be found here. I've also uploaded step-by-step instructions for the live-coding done during the talk.

Cheers,
Galder

Cache Store Batch Operations

Infinispan 9.1.x introduces batch write and delete operations for cache stores. The introduction of batching should greatly improve performance when utilising write-behind cache stores, using putAll operations and committing transactions in non-transactional stores.

CacheWriter Interface Additions

The CacheWriter interface has been extended so that it exposes two additional methods: deleteBatch and writeBatch.  For the sake of backwards compatibility a default implementation of these methods is provided, however if your cache store is able to utilise batching we strongly recommend you create your  own implementations. The additional methods and docs are show below: 

Updated Stores

Currently the JDBC, JPA, RocksDB and Remote stores have all been modified to take advantage of these latest changes.

Configuration Changes

As each store implementations has different batching capabilities, it was necessary to introduce a max-batch-size attribute to the AbstractStoreConfiguration. This attribute defines the maximum number of entries that should be included in a single batch operation to the store. If a value less than one is provided, then the underlying store implementation should not place a upper limit on the number of entries in a batch. 

Deprecated Attributes

Both TableManipulationConfiguration#batchSize and JpaStoreConfiguration#batchSize have been deprecated, as they serve the same purpose as AbstractStoreConfiguration#maxBatchSize.

Store Benchmark

To measure the impact of batch writes on Cache.putAll, we created a simple benchmark to compare the performance of Infinispan 9.1.1.Final (with batching) and 9.0.3.Final (without).  The benchmark consisted of 20 threads inserting 100000 cache entries as fast as possible into a cache via putAll; with each putAll operation containing 20 cache entries and the max-batch-size of each store being set to 20. The table below shows the average time taken for each store type after the benchmark was executed three times.

Store Type	9.0.3.Final	9.1.1-Final	Latency Decrease
JdbcStringBasedStore	29368ms	2597ms	91.12%
JPAStore	30798ms	16640ms	45.97%
RocksDBStore	1164ms	209ms	82.04%

The benchmark results above clearly show that performance is increased dramatically when utilising batch updates at the store level.

Conclusions

Infinispan 9.1.x introduces batching capabilities to the CacheWriter interface in order to improve performance. If you currently utilise a custom cache store, we strongly recommend that you provide your own implementation of the delete and write batch methods. 

If you have any feedback on the CacheWriter changes, or would like to request some new features/optimisations, let us know via the forum, issue tracker or the #infinispan channel on Freenode.

Infinispan coming to Basel One 2017!

Tomorrow, 19th October, I'll be presenting at Basel One 2017 a talk on "Streaming Data Analysis with Kubernetes", where I'll be combining OpenShift, Infinispan and Vert.x to demonstrate how to process streaming data.

If you're in Basel and want to find out more about these technologies, make sure you come! The talk is at 16:00 :D

Cheers,
Galder

Infinispan 9.2.0.Alpha1 released

Dear Infinispan Community,

Today we continue with our time-boxed releases, this time on 9.2 release branch! We prepared The Infinispan 9.2.0.Alpha1 for you. As usual, it can be found on our download page.

9.2.0.Alpha1 brings Infinispan MultimapCache - a new distributed and clustered collection type that maps keys to values in which each key can contain multiple values. We rolled out support for embedded cache mode, but upcoming releases will have support for other Infinispan modes, including client invocation through hotrod.

Aside from MultimapCache we also include the usual slew of bug fixes, clean ups, and general improvements. Full details of the new features and enhancements included in this release can be found here.

We have a lot more exciting features coming up on Infinispan 9.2 branch. Thank you for following us and stay tuned!

The Infinispan Team

Better Late than Never: Remote Cache collections

One of the main benefits of Infinispan extending the java.util.Map interface when we introduced our Cache interface  was that users would immediately be able to use a well established and familiar API.

The unfortunate thing about this relationship is that now the Cache interface also has to implement all of the other methods such as keySet, values and entrySet. Originally Infinispan didn't implement these collections or returned an immutable copy (requiring all elements to be in memory). Neither choice is obviously desirable.

This all changed with ISPN-4836 which provided backing implementations of keySet, values and entrySet collections. This means that all methods were now provided and would keep up to date with changes to the underlying Cache and updates to these collections would be persisted down to the Cache. The implementation also didn't keep a copy of all contents and instead allowed for memory efficient iteration. And if the user still wanted to use a copy they could still do that, by iterating over the collection and copying themselves. This later spring boarded our implementation of Distributed Stream as well.

The problem was that the RemoteCache was left in the old state, where some things weren't implemented and others were copies just like how embedded caches used to be.
Well I can now gladly say with the release of Infinispan 9.1 that RemoteCache now has backing implementations of keySet, values and entrySet implemented via ISPN-7900. Thus these collections support all methods on these collections and are backed by the underlying RemoteCache.

Unfortunately the Stream methods on these collections are not distributed like embedded, but we hope to someday improve that as well. Instead these streams must iterate over the cache to perform the operations locally. By default these will pull 10,000 entries at a time to try to make sure that memory is not overburdened on the client. If you want to decrease this number (less memory - lower performance) or increase (more memory - higher performance) you can tweak this by changing the batchSize parameter via ConfigurationBuilder or infinispan.client.hotrod.batch_Size if you use a property based file.

You can read more about this and the remote iterator which drives these collections on our user guide.

We hope you find that this improves your usage of RemoteCaches in the future by allowing you to have backed collections that also allow you to use the improvements of Java 8 with Streams.

If you have yet you can acquire Infinispan 9.1.1 or the latest stable version at http://infinispan.org/download/

JGroups workshops in Rome and Berlin in November

I'm happy to announce that I've revamped the JGroups workshop and I'll be teaching 2 workshops in November: Rome Nov 7-10 and Berlin Nov 21-24.

The new workshop is 4 days (TUE-FRI). I've updated the workshop to use the latest version of JGroups (4.0.x), removed a few sections and added sections on troubleshooting/diagnosis and handling of network partitions (split brain).

Should be a fun and hands-on workshop!

To register and for the table of contents, navigate to http://www.jgroups.org/workshops.html

To get the early-bird discount, use code EARLYBIRD.

Cheers,

Bela

Infinispan 9.1.1.Final is out

Dear Infinispan users,

after 9.1.0.Final we decided to go on a bug-squashing spree before embarking on the feature work that will be part of 9.2.

So, fresh out of the build baking machine is 9.1.1.Final. You can peruse the list of fixed bugs and you can get the bits from our download page.

Enjoy !

DevNation Live video and slides available now!

Last week I gave a talk for DevNation Live on how to handle big data with Infinispan. The recording of the talk is available now from here, and the slides are here too.

In case you missed the talk, don't miss the opportunity to see what Infinispan can do with big data!

Cheers,
Galder

Multi-tenancy - Infinispan as a Service (also on OpenShift)

In recent years Software as a Service concept has gained a lot of traction. I'm pretty sure you've used it many times before. Let's take a look at a practical example and explain what's going on behind the scenes.

Practical example - photo album application

Imagine a very simple photo album application hosted within the cloud. Upon the first usage you are asked to create an account. Once you sign up, a new tenant is created for you in the application with all necessary details and some dedicated storage just for you. Starting from this point you can start using the album - download and upload photos. 

The software provider that created the photo album application can also celebrate. They have a new client! But with a new client the system needs to increase its capacity to ensure it can store all those lovely photos. There are also other concerns - how to prevent leaking photos and other data from one account into another? And finally, since all the content will be transferred through the Internet, how to secure transmission?

As you can see, multi-tenancy is not that easy as it would seem. The good news is that if it's properly configured and secured, it might be beneficial both for the client and for the software provider. 

Multi-tenancy in Infinispan

Let's think again about our photo album application for a moment. Whenever a new client signs up we need to create a new account for him and dedicate some storage. Translating that into Infinispan concepts this would mean creating a new CacheContainer. Within a CacheContainer we can create multiple Caches for user details, metadata and photos. You might be wondering why creating a new Cache is not sufficient? It turns out that when a Hot Rod client connects to a cluster, it connects to a CacheContainer exposed via a Hot Rod Endpoint. Such a client has access to all Caches. Considering our example, your friends could possibly see your photos. That's definitely not good! So we need to create a CacheContainer per tenant. Before we introduced Multi-tenancy, you could expose each CacheContainer using a separate port (using separate Hot Rod Endpoint for each of them). In many scenarios this is impractical because of proliferation of ports. For this reason we introduced the Router concept. It allows multiple clients to access their own CacheContainers through a single endpoint and also prevents them from accessing data which doesn't belong to them. The final piece of the puzzle is transmitting sensitive data through an unsecured channel such as the Internet. The use of TLS encryption solves this problem. The final outcome should look like the following:

The Router component on the diagram above is responsible for recognizing data from each client and redirecting it to the appropriate Hot Rod endpoint.
As the name implies, the router inspects incoming traffic and reroutes it to the appropriate underlying CacheContainer. To do this it can use two different strategies depending on the protocol: TLS/SNI for the Hot Rod protocol, matching each server certificate to a specific cache container  and path prefixes for REST.
The SNI strategy detects the SNI Host Name (which is used as tenant) and also requires TLS certificates to match. By creating proper trust stores we can match which tenant can access which CacheContainers.
URL path prefix is very easy to understand, but it is also less secure unless you enable authentication. For this reason it should not be used in production unless you know what you are doing (the SNI strategy for the REST endpoint will be implemented in the near future). Each client has its own unique REST path prefix that needs to be used for accessing the data (e.g. http://127.0.0.1:8080/rest/client1/fotos/2).

Confused? Let's clarify this with an example.

Foto application sample configuration

The first step is to generate proper key/trust stores for the server and client:

The next step is to configure the server. The snippet below shows only the most important parts:

Let's analyze the most critical lines:

• 7, 15 - We need to add generated key stores to the server identities
• 25, 30 - It is highly recommended to use separate CacheContainers
• 38, 39 - A Hot Rod connector (but without socket binding) is required to provide proper mapping to CacheContainer. You can also use many useful settings on this level (like ignored caches or authentication).
• 42 - Router definition which binds into default Hot Rod and REST ports.
• 44 - 46 - The most important bit which states that only a client using SSLRealm1 (which uses trust store corresponding to client_1_server_keystore.jks) and TLS/SNI Host name client-1 can access Hot Rod endpoint named multi-tenant-hotrod-1 (which points to CacheContainer multi-tenancy-1).

Improving the application by using OpenShift

Hint: You might be interested in looking at our previous blog posts about hosting Infinispan on OpenShift. You may find them at the bottom of the page.

So far we've learned how to create and configure a new CacheContainer per tenant. But we also need to remember that system capacity needs to be increased with each new tenant. OpenShift is a perfect tool for scaling the system up and down. The configuration we created in the previous step almost matches our needs but needs some tuning.

As we mentioned earlier, we need to encrypt transport between the client and the server. The main disadvantage is that OpenShift Router will not be able to inspect it and take routing decisions. A passthrough Route fits perfectly in this scenario but requires creating TLS/SNI Host Names as Fully Qualified Application Names. So if you start OpenShift locally (using oc cluster up) the tenant names will look like the following: client-1-fotoalbum.192.168.0.17.nip.io. 

We also need to think how to store generated key stores. The easiest way is to use Secrets:

Finally, a full DeploymentConfiguration:

If you're interested in playing with the demo by yourself, you might find a working example here. It mainly targets OpenShift but the concept and configuration are also applicable for local deployment.

Links

• Running Infinispan cluster on OpenShift
• Configuration management on OpenShift, Kubernetes and Docker
• Checking Infinispan cluster health and Kubernetes/OpenShift
• Source code for the demo

Join us online on 7th September for DevNation talk on Infinispan!

Tomorrow, 7th of September at 12:00pm EDT, I will be doing an online live tech talk for Red Hat DevNation on showing how to handle big data with Infinispan.

If you didn't manage to attend my talk at Great Indian Developer Summit or Berlin Buzzwords, this is your chance to see it live and direct, with live coding demos included!

You can register to see the talk live visiting the Red Hat DevNation Live site.

Cheers,
Galder

Cluster Counter

In Infinispan 9.1 we introduce the clustered counters. It is a counter distributed and shared among all nodes in the cluster and today we are going to know more about it.

Installation

To use a counter in your Infinispan cluster, first you have to add the maven dependency to your project. As you can see, it is simple as doing:

After adding the module, you can retrieve the CounterManager and start creating and using counters.

CounterManager

Each CounterManager is associated to a CacheManager. But, before showing how to use it, first we have some configuration to be done.

There are two attributes that you can configure: The num-owner - that represents the number of copies of the counter's value in a cluster; and the reliability - that represents the behavior of the counters in case of partitions.

Below, is the configuration example with their default values.
XML:
Programatically:
Then, you can retrieve the CounterManager from the CacheManager, as shown below, and start using the counters!

Counter

A counter is identified by its name. Also, it is initialized with an initial value (by default 0) and it can be persisted, if the value needs to survive a cluster restart.

There are 2 types of counters: strong and weak counters.

Strong Counters

The strong counter provides higher consistency. Its value is known during the update and its updates are applied atomically. This allows to set boundaries and provides conditional operation (as compare-and-set).

Configuration

A strong counter can be configured in the configuration file or programatically. They can be also created dynamically at runtime. Below shows us how it can be done:

XML:
Programatically:
Runtime:

Use Case

The strong counter fits the following uses cases:

• Global Id Generator

Due to its strong consistency, it can be used as a global identifier generator, as in the example below:

• Rate Limiter

If bounded, it can be used as a simple rate limiter. Just don't forget to invoke reset()...

• Simply count "stuff"

Well, it is a counter after all...

Weak Counters

The weak counter provides eventual consistency and its value is not known during updates. It provides faster writes when comparing with the strong counter.

Configuration

As in strong counter, the weak counter can be configure its name and its initial value. In addition, a concurrency-level can be configure to set the number of concurrent updates that can be handled in parallel. Below shows us how to configure it:

XML:
Programatically:
Runtime:

Use Case

The main use case for the weak counter includes all scenarios where its value isn't needed while updating the counter. For example, it can be used to count the number of visits to some resource:

For more information, take a look at the documentation. If you have any feedback, or would like to request some new features, or found some issue, let us know via the forum, issue tracker or the #infinispan channel on Freenode.