Spring Boot Starters

Ho, ho, hooo! It looks like all members of Infinispan Community have been nice and Santa brought you Spring Boot Starters!

This will make you even more productive and your code less verbose!

Why do I need starters?

Spring Boot Starters make the bootstrapping process much easier and faster. The starter brings you required Maven dependencies as well as some predefined configuration bits.

What do I need to get started?

The starter can operate in two modes: client/server (when you connect to a remote Infinispan Server cluster) and embedded (packaged along with your app). The former is the default. It's also possible to use both those modes at the same time (store some data along with your app and connect to a remote Infinispan Server cluster to perform some other type of operations).

Assuming you have an Infinispan Server running on IP address 192.168.0.17, all you need to do is to use the following dependencies:

	<?xml version="1.0" encoding="UTF-8"?>
	<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example</groupId>
	<artifactId>myproject</artifactId>
	<version>1.0.0-SNAPSHOT</version>
	<properties>
	<infinispan.starters.version>1.0.0.Alpha1</infinispan.starters.version>
	</properties>
	<dependencies>
	<dependency>
	<groupId>org.infinispan</groupId>
	<artifactId>inifinispan-spring-boot-starter</artifactId>
	<version>${infinispan.starters.version}</version>
	</dependency>
	</dependencies>
	<build>
	<plugins>
	<plugin>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-maven-plugin</artifactId>
	<version>${spring-boot.version}</version>
	</plugin>
	</plugins>
	</build>
	</project>

view raw pom.xml hosted with ❤ by GitHub

By default, the starter will try to locate hotrod-client.properties file. The file should contain at least the server list:

infinispan.client.hotrod.server_list=192.168.0.17:11222

view raw hotrod-client.properties hosted with ❤ by GitHub

It is also possible to create RemoteCacheManager's configuration manually:

	@Configuration
	public class InfinispanCacheConfiguration {
	public static final String IP = "192.168.0.17";
	@Bean
	public InfinispanRemoteConfigurer infinispanRemoteConfigurer() {
	return () -> new ConfigurationBuilder().addServer().host(IP).build();
	}
	}

view raw InfinispanCacheConfiguration.java hosted with ❤ by GitHub

That's it! Your app should successfully connect to a remote cluster and you should be able to inject RemoteCacheManager.

Using Infinispan embedded is even simpler than that. All you need to do is to add additional dependency to the classpath:

	<?xml version="1.0" encoding="UTF-8"?>
	<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example</groupId>
	<artifactId>myproject</artifactId>
	<version>1.0.0-SNAPSHOT</version>
	<properties>
	<infinispan.starters.version>1.0.0.Alpha1</infinispan.starters.version>
	<version.infinispan>8.2.4.Final</version.infinispan>
	</properties>
	<dependencyManagement>
	<dependencies>
	<dependency>
	<groupId>org.infinispan</groupId>
	<artifactId>infinispan-bom</artifactId>
	<version>${version.infinispan}</version>
	<type>pom</type>
	<scope>import</scope>
	</dependency>
	<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-parent</artifactId>
	<version>${spring-boot.version}</version>
	<type>pom</type>
	<scope>import</scope>
	</dependency>
	</dependencies>
	</dependencyManagement>
	<dependencies>
	<dependency>
	<groupId>org.infinispan</groupId>
	<artifactId>inifinispan-spring-boot-starter</artifactId>
	<version>${infinispan.starters.version}</version>
	</dependency>
	<dependency>
	<groupId>org.infinispan</groupId>
	<artifactId>infinispan-core</artifactId>
	</dependency>
	</dependencies>
	<build>
	<plugins>
	<plugin>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-maven-plugin</artifactId>
	<version>${spring-boot.version}</version>
	</plugin>
	</plugins>
	</build>
	</project>

view raw pom.xml hosted with ❤ by GitHub

The starter will provide you a preconfigured EmbeddedCacheManager. In order to customize the configuration, use the following code snippet:

	@Configuration
	public class InfinispanCacheConfiguration {
	public static final String TEST_CLUSTER = "TEST_CLUSTER";
	public static final String TEST_CACHE_NAME = "test-simple-cache";
	public static final String TEST_GLOBAL_JMX_DOMAIN = "test.infinispan";
	@Bean
	public InfinispanCacheConfigurer cacheConfigurer() {
	return cacheManager -> {
	final org.infinispan.configuration.cache.Configuration testCache =
	new ConfigurationBuilder().simpleCache(true)
	.eviction().size(1000L).strategy(EvictionStrategy.LRU)
	.jmxStatistics().enable()
	.build();
	cacheManager.defineConfiguration(TEST_CACHE_NAME, testCache);
	};
	}
	@Bean
	public InfinispanGlobalConfigurer globalConfigurer() {
	return () -> {
	final GlobalConfiguration globalConfiguration = new GlobalConfigurationBuilder()
	.transport().clusterName(TEST_CLUSTER)
	.globalJmxStatistics().jmxDomain(TEST_GLOBAL_JMX_DOMAIN).enable()
	.build();
	return globalConfiguration;
	};
	}

view raw InfinispanCacheConfiguration.java hosted with ❤ by GitHub

Further reading

There are two link I highly recommend you to read. The first is the Spring Boot tutorial and the second is the Github page of the Starters project. 

Kudos

Special thanks go to Marco Yuen, who donated us with Spring Boot Starters code and Tomasz Zabłocki, who updated them to current version and Stéphane Nicoll who spent tremendous amount of time reviewing the Starters.

Data Container Changes Part 1

Infinispan 9.0 Beta 1 introduces some big changes to the Infinispan data container.  This is the first of two blog posts detailing those changes.

This post will cover the changes to eviction which utilizes a new provider, Caffeine.  As you may already know Infinispan has supported our own implementations of LRU (Least Recently Used) and LIRS (Low Inter-reference Receny Set) algorithms for our bounded caches.

Our implementations of eviction were even rewritten for Infinispan 8, but we found we still had some issues or limitations with them, especially LIRS.  Our old implementation had some problems with keeping the correct number of entries.  The new implementation while not having that issue had others, such as being considerably more complex.  And while it implemented the entire LIRS specification, it could have memory usage issues.  This led us to looking at alternatives and Caffeine seemed like a logical fit as well as being well maintained and the author, Ben Manes, is quite responsive.

Enter Caffeine

Caffeine doesn't utilize LRU or LIRS for its eviction algorithm and instead implements TinyLFU with an admission window.  This has the benefit of the high hit ratio like LIRS, while also requiring low memory overhead like LRU.  Caffeine also provides custom weighting for objects, which allow us to reuse the code that was developed for MEMORY based eviction as well.

The only thing that Caffeine doesn't support is our idea of a custom Equivalence.  Thus Infinispan now wraps byte[] instances to ensure equals and hashCode methods work properly.  This also gives us a good opportunity to reevaluate the dataContainer configuration element.

Deprecations

The data container configuration has thus been deprecated and is now replaced by a new configuration element named memory.   Also since we are adding a new element the eviction configuration could also be consolidated into memory, and thus eviction is also deprecated.  And last but not least the storeAsBinary configuration element has also been integrated into the new memory configuration element.  Now we have 1 configuration element instead of 3, can't beat that!

New Configuration

The new memory configuration will start out pretty simple and new elements can be added as there is a need.  The memory element will be composed of a single sub element that can be of three different choices.  For this post we will go over two of the sub elements: OBJECT and BINARY.

OBJECT

Object storage stores the actual objects as provided from the user in the Java Heap.  This is the default storage method when no memory configuration is provided.  This method will provide the best performance when using operations that operate upon the entire data set, such as distributed streams, indexing and local reads etc.

Unfortunately OBJECT storage only allows for COUNT based eviction as we cannot properly estimate user object types properly.  This could be improved in a feature version if there is enough interest. Note that you can technically configured MEMORY eviction type with the OBJECT storage type with declarative configuration, but it will throw an exception when you build the configuration.  Therefore OBJECT only has a single element named size to determine the amount of entries that can be stored in the cache.

An example of how Object storage can be configured:

XML

	<local-cache ...>
	<memory>
	<object size="100000"/>
	</memory>
	</local-cache>

view raw object.xml hosted with ❤ by GitHub

DECLARATIVE

	configuration.memory()
	.storageType(StorageType.OBJECT)
	.size(100_000);

view raw object.java hosted with ❤ by GitHub

BINARY

Binary storage stores the object in its serialized form in a byte array.  This has an interesting side effect of objects are always stored as a deep copy.  This can be useful if you want to modify an object after retrieving it without affecting the underlying cache stored object.  Since objects have to be deserialized when performing operations on them some things such as distributed streams and local gets will be a little bit slower.

A nice benefit of storing entries as BINARY is that we can estimate the total on heap size of the object.  Thus BINARY supports both COUNT and MEMORY based eviction types.

An example of how Binary storage can be configured:

XML

	<local-cache ...>
	<memory>
	<binary eviction="MEMORY" size="1000000000"/>
	</memory>
	</local-cache>

view raw binary.xml hosted with ❤ by GitHub

DECLARATIVE

	configuration.memory()
	.storageType(StorageType.BINARY)
	.evictionType(EvictionType.MEMORY)
	.size(1_00_000_000);

view raw binary.java hosted with ❤ by GitHub

OFF-HEAP

This option will be described in more detail in the next blog post.  Stay tuned!

Conclusion

Caffeine should bring us a great solution, while also reducing a lot of maintenance ourselves.  The new memory configuration also provides a simpler solution by removing two other configuration elements.

We hope you enjoy the new changes to the data container and look out for another blog post coming soon to detail the other new changes to the data container!  In the meantime please check out our latest Infinispan 9.0 before it goes final and give us any feedback on IRC or JIRA

Thanks Soft-Shake and Devoxx Morocco!

Last month I presented about building functional reactive applications with Infinispan, Node.js and Elm at both Soft-Shake in Geneva (slides) and Devoxx Morocco (slides).

Thanks a lot to all the participants who attended the talks and thanks also to the organisers for accepting my talk. Both conferences were really enjoyable!

At Soft-Shake I managed to attend a few presentations, and the one that really stuck with me was the one from Alexandre Masselot on "Données CFF en temps réel: tribulations techniques dans la stack Big Data" (slides). It was a very interesting use case on doing big data with the information from the Swiss Rail system. Although there was no live demo, Alexandre gave the link to a repo where you can run stuff yourself. Very cool!

On top of that, I also attended a talk by Tom Bujok on Scaling Your Application Out. Tom happens to be an old friend who since I last met him has joined Hazelcast ;)

Shortly after Shoft-Shake I headed to Casablanca to speak at Devoxx Morocco. This was a fantastic conference with a lot of young attendees. The room was almost packed up for my talk and I got good reaction from the audience on both the talk and the live demo.

During the conference I also attended other talks, including a couple of Kubernetes talks by Ray Tsang, who is an Infinispan committer himself. In his presentations he uses a Kubernetes visualizer which is very cool and I'm hoping to use it in future presentations :)

No more conferences for this year, thanks to all who've attended Infinispan presentations throughout the year!

Cheers,

Galder

Meet Ickle!

As you’ve already learned from an earlier post this week, Infinispan 9 is on its final approach to landing and is bringing a new query language. Hurray! But wait, was there something wrong with the old one(s)? Not wrong really ...  I’ll explain.

Infinispan is a data grid of several query languages. Historically, it has offered search support early in its existence by integrating with Hibernate Search which provides a powerful Java-based DSL enabling you to build Lucene queries and run them on top of your Java domain model living in the data grid. Usage of this integration is confined to embedded mode, but that still succeeds in making Java users happy.

While the Hibernate Search combination is neat and very appealing to Java users it completely leaves non-JVM languages accessing Infinispan via remote protocols out in the cold.

Enter Remote Query. Infinispan 6.0 starts to address the need of searching the grid remotely via Hot Rod. The internals are still built on top of Lucene and Hibernate Search bedrock but these technologies are now hidden behind a new query API, the QueryBuilder, an internal DSL resembling JPA criteria query. The QueryBuilder has implementations for both embedded mode and Hot Rod. This new API provides all relational operators you can think of, but no full-text search initially, we planned to add that later.

Creating a new internal DSL was fun. However, having a long term strategy for evolving it while keeping complete backward compatibility and also doing so uniformly across implementations in multiple languages proved to be a difficult challenge. So while we were contemplating adding new full-text operators to this DSL we decided on making a long leap forward and adopt a more flexible alternative by having our own string based query language instead, another DSL really, albeit an external one this time.

So after the long ado, let me introduce Ickle, Infinispan’s new query language, conspicuously resembling JP-QL.

Ickle:

• is a light and small subset of JP-QL, hence the lovely name
• queries Java classes and supports Protocol Buffers too
• queries can target a single entity type
• queries can filter on properties of embedded objects too, including collections
• supports projections, aggregations, sorting, named parameters
• supports indexed and non-indexed execution
• supports complex boolean expressions
• does not support computations in expressions (eg. user.age > sqrt(user.shoeSize + 3) is not allowed but user.age >= 18 is fine)
• does not support joins
• but, navigations along embedded entities are implicit joins and are allowed
• joining on embedded collections is allowed
• other join types not supported
• subqueries are not supported
• besides the normal relational operators it offers full-text operators, similar to Lucene’s  query parser
• is now supported across various Infinispan APIs, wherever a Query produced by the QueryBuilder is accepted (even for continuous queries or in event filters for listeners!)

That is to say we squeezed JP-QL to the bare minimum and added full-text predicates that closely follow the syntax of Lucene’s query parser.

If you are familiar with JPA/JP-QL then the following example will speak for itself:

select accountId, sum(amount) from com.acme.Transaction
    where amount < 20.0
    group by accountId
    having sum(amount) > 1000.0
    order by accountId

The same query can be written using the QueryBuilder:

Query query = queryFactory.from(Transaction.class)
.select(Expression.property("accountId"), Expression.sum("amount"))
.having("amount").lt(20.0)
.groupBy("accountId")
.having(Expression.sum("amount")).gt(1000.0)
.orderBy("accountId").build();

Both examples look nice but I hope you will agree the first one is better.

Ickle supports several new predicates for full-text matching that the QueryBuilder is missing. These predicates use the : operator that you are probably familiar from Lucene’s own query language.  This example demonstrates a simple full-text term query:

select transactionId, amount, description from com.acme.Transaction
where amount > 10 and description : "coffee"

As you can see, relational predicates and full-text predicates can be combined with boolean operators at will.

The only important thing to remark here is relational predicates are applicable to non-analyzed fields while full-text predicates can be applied to analyzed field only. How does indexing work, what is analysis and how do I turn it on/off for my fields? That’s the topic of a future post, so please be patient or start reading here.

Besides term queries we support several more:

• Term                     description : "coffee"
• Fuzzy                    description : "cofee"~2
• Range                   amount : [40 to 90}
• Phrase                  description : "hello world"
• Proximity               description : "canceling fee"~3
• Wildcard                description : "te?t"
• Regexp                 description : /[mb]oat/
• Boosting                description : "beer"^3 and description : "books"

You can read all about them starting from here.

But is Ickle really new? Not really. The name is new, the full-text features are new, but a JP-QL-ish query string was always internally present in the Query objects produced by the QueryBuilder since the beginning of Remote Query. That language was never exposed and specified until now. It evolved significantly over time and now it is ready for you to use it. The QueryBuilder / criteria-like API is still there as a convenience but it might go out of favor over time. It will be limited to non-full-text functionality only. As Ickle grows we’ll probably not be able to include some of the additions in the QueryBuilder in a backward compatible manner. If growing will cause too much pain we might consider deprecating it in favor of Ickle or if there is serious demand for it we might continue to evolve the QueryBuilder in a non compatible manner.

Being a string based query language, Ickle is very convenient for our REST endpoint, the CLI, and the administration console allowing you to quickly inspect the contents of the grid. You’ll be able to use it there pretty soon. We’ll also continue to expand Ickle with more advanced full-text features like spatial queries and faceting, but that’s a subject for another major version. Until then, why not grab the current 9.0 Beta1 and test drive the new query language yourself? We’d love to hear your feedback on the forum, on our issue tracker or on IRC on the #infinispan channel on Freenode.

Happy coding!

Distributed Stream Quality of Life Improvements

As I hope most people reading this already know, since Infinispan 8 you can utilize the entire Java 8 Stream API and have it be distributed across your cluster.  This performs the various intermediate and terminal operations on the data local to the node it lives on, providing for extreme performance.  There are some limitations and things to know as was explained at distributed-streams.

The problem with the API up to now was that, if you wanted to use lambdas, it was quite an ugly scene.  Take for example the following code snippet:

8.0 Distributed Streams Example

	Map<String, Integer> wordCountMap = c1.entrySet().parallelStream()
	.map((Serializable & Function<Map.Entry<String, String>, String[]>) e -> e.getValue().split("\\s"))
	.flatMap((Serializable & Function<String[], Stream<String>>) Arrays::stream)
	.collect(CacheCollectors.serializableCollector(() -> Collectors.groupingBy(Function.identity(), Collectors.counting())));

view raw old.java hosted with ❤ by GitHub

However, for Infinispan 9 we utilize a little syntax feature added with Java 8 [1] to add some much needed quality of life improvements.  This allows the most specific interface to be chosen when a method is overloaded.  This allows for a neat interaction when we add some new interfaces that implement Serializable and the various function interfaces (SerializableFunction, SerializablePredicate, SerializableSupplier, etc).  All of the Stream methods have been overridden on the CacheStream interface to take these arguments.

This allows for the code to be much cleaner as we can see here:

9.0 Distributed Streams Example

	Map<String, Integer> wordCountMap = c1.entrySet().parallelStream()
	.map(e -> e.getValue().split("\\s"))
	.flatMap(Arrays::stream)
	.collect(CacheCollectors.serializableCollector(() -> Collectors.groupingBy(Function.identity(), Collectors.counting())));

view raw new.java hosted with ❤ by GitHub

Extra Methods

This is not the only benefit of providing the CacheStream interface: we can also provide new methods that aren't available on the standard Stream interface.  One example is the forEach method which allows the user to more easily provide a Cache that is injected on each node as required.  This way you don't have to use the clumsy CacheAware interface and can directly use lambdas as desired.

Here is an example of the new forEach method in action:

	cache.entrySet().stream()
	.filter(e -> e.getKey().startsWith("RedHat"))
	.forEach((c, e) -> {
	c.getCacheManager().getCache("RedHat").put(e.getKey(), e.getValue());
	});

view raw forEach.java hosted with ❤ by GitHub

In this example we take a cache and, based on the keys in it, write those values into another cache. Since forEach doesn't have to be side effect free, you can do whatever you want inside here.

All in all these improvements should make using Distributed Streams with Infinispan much easier.  The extra methods could be extended further if users have use cases they would love to suggest.  Just let us know, and I hope you enjoy using Infinispan!

Infinispan 9.0.0.Beta1 "Ruppaner"

It took us quite a bit to get here, but we're finally ready to announce Infinispan 9.0.0.Beta1, which comes loaded with a ton of goodies.

• Performance improvements
• JGroups 4
• A new algorithm for non-transactional writes (aka the Triangle) which reduces the number of RPCs required when performing writes 
• A new faster internal marshaller which produced smaller payloads. 
• A new asynchronous interceptor core
• Off-Heap support
• Avoid the size of the data in the caches affecting your GC times
• CaffeineMap-based bounded data container
• Superior performance
• More reliable eviction
• Ickle, Infinispan's new query language
• A limited yet powerful subset of JPQL
• Supports full-text predicates
• The Server Admin console now supports both Standalone and Domain modes
• Pluggable marshallers for Kryo and ProtoStuff
• The LevelDB cache store has been replaced with the better-maintained and faster RocksDB 
• Spring Session support
• Upgraded Spring to 4.3.4.RELEASE

We will be blogging about the above in detail over the coming weeks, including benchmarks and tutorials.

The following improvements were also present in our previous Alpha releases:

• Graceful clustered shutdown / restart with persistent state
• Support for streaming values over Hot Rod, useful when you are dealing with very large entries
• Cloud and Containers
• Out-of-the box support for Kubernetes discovery
• Cache store improvements
• The JDBC cache store now use transactions and upserts. Also the internal connection pool is now based on HikariCP

Also, our documentation has received a big overhaul and we believe it is vastly superior than before.

There will be one more Beta including further performance improvements as well as additional features, so stay tuned.

Infinispan 9 is codenamed "Ruppaner" in honor of the Konstanz brewery, since many of the improvements of this release have been brewed on the shores of the Bodensee !

Prost!

Spring as a Spec?

We've been maintaining Spring integrations bits in Infinispan for quite a while. Our development version, Infinispan 9, contains a set of changes in those modules which deserve some explanation...

Versions... versions everywhere...

When you use Infinispan, you rely on some, very specific version. The same applies to Spring. Before Infinispan 9 our integration modules had a compile-time dependency to a very specific version of Spring. In practice, we imposed this specific version to each project which used our integration bits. The question is - what to do to use some other version of Spring or Spring Boot? Till Infinispan 9, the simplest solution was to exclude Spring from infinispan-spring4-embedded (or infinispan-spring4-remote) artifact using Maven exclusions. Not a very nice and intuitive solution was it?

How about treating Spring as a Spec or API

You probably noticed, when you use JPA you don't care about underlying implementation. Is it Hibernate, OpenJPA... it doesn't matter?

If you think carefully about Spring for a while, it's a bit similar. All core classes might be delivered using standard dependency mechanism (adding spring-beans, spring-context manually), using Spring Boot (adding spring-boot-starter) or even using 3rd party integration tools like DropWizard. In case of a bigger project, a decision whether or not use any of those solutions needs to be taken long before Infinispan is chosen as a Distributed Store or Cache.

At this point, we can do a mental experiment - treat Spring classes as a Spec or API and those delivery mechanisms treat as implementations.

Scope? Provided, of course!

From Infinispan integration perspective we need the Spring API (the classes) and we don't care about the implementation (delivery mechanism). Having this in mind we decided to change Spring's scope in Infinispan modules to provided.

I use Infinispan and Spring, what shall I do?

Starting from Infinispan 9, you can stop using exclusions. You probably already use Spring and your favorite delivery mechanism (Spring Boot for example). Then add infinispan-spring4-embedded or infinispan-spring4-remote artifact. Finally, you need to decide how would you like to use Infinispan - via Uber Jars (infinispan-embedded and infinispan-remote) or using Small Jars (infinispan-core for example).

That's it! Have fun!

Infinispan 8.2.5.Final

Dear Infinispan users,

we have just released Infinispan 8.2.5.Final which fixes a number of community-reported issues. Get the bits from our download page. As usual feedback is always welcome.

Hotrod clients C++/C# 8.1.0.Alpha2 released!

Dear Infinispan community,

I'm pleased to announce that the C++/C# clients version 8.1.0.Alpha2 are out!

Some of the good news coming with this release:

• more bugs fixed than added
• SNI support
• C++ Client listener for remote events

Download it from the usual link http://infinispan.org/hotrod-clients/


We're trying to keep track of the 8.1 trip at this Jira url:
Features list for 8.1
Feedbacks, proposals, hints are welcome!

Cheers,
The Infinispan Team

Infinispan coming to Devoxx Morocco!

Tomorrow Thursday, 3rd November, I'll be speaking at the Devoxx Morocco conference (Casablanca, Morocco) about writing apps in functional reactive programming style with Infinispan, Elm and Node.js. If you're interested in the topic and live in the area, make sure you come to my talk!

To find out more, head to the Devoxx Morocco site, where you can find exact details about the rest of the programme, location...etc.

Cheers,

Galder

Learn Functional Reactive Programming with Infinispan, Elm and Node.js at Soft-Shake conference

Tomorrow Friday, 28th October, I'll be speaking at the Soft-Shake conference (Geneva, Switzerland) about writing apps in functional reactive programming style with Infinispan, Elm and Node.js. If you're interested in the topic and live in the area, make sure you come to my talk!

To find out more, head to the Soft-Shake site, where you can find exact details about the rest of the programme, location...etc.

Cheers,

Galder

OpenShift and Node Affinity

OpenShift (and Kubernetes) has a great feature called Affinity. In some scenarios, it might be beneficial to configure it along with Infinispan node affinity.

Before we start... this tutorial is heavily based on our previous blog post about deploying Infinispan on Openshift and OpenShift Scheduler functionality. It is highly recommended to read those articles before continuing this tutorial.

How does the Affinity feature work... in short?

In order to decide on which node given Pod should be running, OpenShift looks at so called Predicates and Priority Functions. A predicate must match the one configured in DeploymentConfiguration and Priority Function is responsible for choosing the best node for running Pods.

Let's assume that we have a sample policy (similar to one provided in OpenShift manual), that uses site as a Predicate along with rack and machine as Priority Functions. Now let's assume we have two nodes:

• Node 1 - site=EU, rack=R1, machine=VM1
• Node 2 - site=US, rack=R2, machine=VM2

And two DeploymentConfiguration with Node Selectors (this tells OpenShift on which nodes given DeploymentConfiguration wishes to run) defined as follows:

• DeploymentConfiguration 1 - site=EU, rack=R5, machine=VM5
• DeploymentConfiguration 2 - site=JAP, rack=R5, machine=VM5

With the above example only DeploymentConfiguration 1 will be scheduled (on Node 1), since site matches the predicate. In this case rack and machine parameters are not used (because we have only one node).

Note that the default OpenShift's configuration uses region (as a Predicate) and zone (as a Priority Function). However it can be reconfigured very easily. 

And I need it because....

Some OpenShift deployments might span multiple racks in a data center or even multiple sites. It is important to tell Infinispan where physical machines are located, which will allow to choose better nodes for backing up your data (in distribution mode). 

As the matter of fact, Infinispan uses site, rack and machine. The main goal is to avoid backing up data on the same host.

The implementation

The implementation is pretty straightforward but there are several gotchas. 

The first one is that OpenShift uses regions and zones by default and Infinispan uses sites, racks and machines. The good news is that all those three are optional, so you have two options - reuse existing region and zone (map them to rack and site for example), or adjust OpenShift scheduler settings. In my example I used the former approach.

The second one is the need of hardcoding those parameters into DeploymentConfiguration. Unfortunately Node Selectors are not exposed through Downward API, so there's no other way.

So let's have a look at our DeploymentConfiguration:

	apiVersion: v1
	kind: DeploymentConfig
	metadata:
	name: infinispan-server
	namespace: slaskawi
	labels:
	app: infinispan-server
	spec:
	replicas: 2
	selector:
	app: infinispan-server
	deploymentconfig: infinispan-server
	template:
	metadata:
	labels:
	app: infinispan-server
	deploymentconfig: infinispan-server
	spec:
	containers:
	-
	name: infinispan-server
	image: jboss/infinispan-server
	args:
	- cloud
	- '-Djboss.default.jgroups.stack=kubernetes'
	- '-Djboss.jgroups.topology.rack=default'
	- '-Djboss.jgroups.topology.site=primary'
	ports:
	-
	containerPort: 7600
	protocol: TCP
	-
	containerPort: 8080
	protocol: TCP
	-
	containerPort: 8181
	protocol: TCP
	-
	containerPort: 8888
	protocol: TCP
	-
	containerPort: 9990
	protocol: TCP
	-
	containerPort: 11211
	protocol: TCP
	-
	containerPort: 11222
	protocol: TCP
	-
	containerPort: 57600
	protocol: TCP
	env:
	-
	name: OPENSHIFT_KUBE_PING_NAMESPACE
	valueFrom: {fieldRef: {apiVersion: v1, fieldPath: metadata.namespace}}
	nodeSelector:
	region: primary
	zone: default

view raw Deployment Configuration with affinity hosted with ❤ by GitHub

• Line 26 - Zone default used as a rack
• Line 27 - Region primary used as a site
• Lines 57 - 59 - Node Selector for scheduling Pods

Conclusion

Combining OpenShift Affinity Service and Infinispan Server Hinting allows to optimize data distribution across the cluster. Keep in mind that your configuration might be totally different (OpenShift Scheduler is a highly configurable thing). But once you understand how it works, you can adjust the hinting strategy for your needs. 

Happy Scaling!