Configuration management on OpenShift, Kubernetes and Docker

When deploying Infinispan on Docker based Cloud environments, the most critical thing is how to manage configuration. In this blog post we'll explore some of the options.

Extending our Docker image

Creating your own Docker image based on jboss/infinispan-server is quite simple. At first you need to prepare a configuration XML file, which is shipped with Infinispan release. Go to Infinispan download section and grap a server release corresponding to a chosen Docker image tag.  After unpacking it, grab the configuration (I use cloud.xml as a template) and introduce all necessary changes. 

Finally, build your image:

	FROM jboss/infinispan-server
	COPY custom-cloud.xml /opt/jboss/infinispan-server/standalone/configuration/cloud.xml

view raw Extending Dockerfile with COPY hosted with ❤ by GitHub

Now, that was quick! Wasn't it?

Using ConfigMaps with OpenShift

If you're using OpenShift, there's a sophisticated tool called ConfigMap. A ConfigMap can store a configuration file (or a configuration directory) and mount it somewhere in the Pod.

Use the command below to create a ConfigMap based on a configuration file:

	$ oc create configmap cloud-xml --from-file=custom-cloud.xml
	configmap "cloud-xml" created

view raw OpenShift ConfigMap from file hosted with ❤ by GitHub

Now create Infinispan application based on the configuration below (you can use 'oc create -f <file.yaml>' for this):

	apiVersion: v1
	items:
	- apiVersion: v1
	kind: DeploymentConfig
	metadata:
	labels:
	app: infinispan-server
	name: infinispan-server
	spec:
	replicas: 1
	selector:
	app: infinispan-server
	deploymentconfig: infinispan-server
	template:
	metadata:
	labels:
	app: infinispan-server
	deploymentconfig: infinispan-server
	spec:
	containers:
	- args:
	- custom/custom-cloud
	- -Djboss.default.jgroups.stack=kubernetes
	image: jboss/infinispan-server
	imagePullPolicy: Always
	name: infinispan-server
	ports:
	- containerPort: 8181
	protocol: TCP
	- containerPort: 8888
	protocol: TCP
	- containerPort: 9990
	protocol: TCP
	- containerPort: 11211
	protocol: TCP
	- containerPort: 11222
	protocol: TCP
	- containerPort: 57600
	protocol: TCP
	- containerPort: 7600
	protocol: TCP
	- containerPort: 8080
	protocol: TCP
	terminationMessagePath: /dev/termination-log
	volumeMounts:
	- mountPath: /opt/jboss/infinispan-server/standalone/configuration/custom
	name: config-volume
	restartPolicy: Always
	volumes:
	- configMap:
	name: cloud-xml
	name: config-volume
	triggers:
	- type: ConfigChange
	kind: List
	metadata: {}

view raw OpenShift Infinispan App hosted with ❤ by GitHub

• (lines 50 - 52) - ConfigMap volume declaration
• (lines 45 - 47) - Mounting configuration into /opt/jboss/infinispan-server/standalone/configuration/custom
• (line 22) - bootstrapping the Infinispan with custom configuration (note there is no xml extension there)

Using ConfigMaps with Kubernetes

Kubernetes ConfigMaps work exactly the same way as in OpenShift.

The command below creates a ConfigMap:

	$ kubectl create configmap cloud-xml --from-file=cloud.xml
	configmap "cloud-xml" created

view raw Kubernetes ConfigMap from file hosted with ❤ by GitHub

The second step is to create a Deployment with ConfigMap:

	apiVersion: v1
	items:
	- apiVersion: extensions/v1beta1
	kind: Deployment
	metadata:
	annotations:
	labels:
	run: infinispan-server
	name: infinispan-server
	namespace: default
	spec:
	replicas: 3
	selector:
	matchLabels:
	run: infinispan-server
	strategy:
	rollingUpdate:
	maxSurge: 1
	maxUnavailable: 1
	type: RollingUpdate
	template:
	metadata:
	creationTimestamp: null
	labels:
	run: infinispan-server
	spec:
	containers:
	- args:
	- custom/cloud
	- -Djboss.default.jgroups.stack=kubernetes
	env:
	- name: OPENSHIFT_KUBE_PING_NAMESPACE
	valueFrom:
	fieldRef:
	apiVersion: v1
	fieldPath: metadata.namespace
	image: jboss/infinispan-server
	imagePullPolicy: Always
	name: infinispan-server
	ports:
	- containerPort: 8080
	protocol: TCP
	- containerPort: 8181
	protocol: TCP
	- containerPort: 8888
	protocol: TCP
	- containerPort: 9990
	protocol: TCP
	- containerPort: 11211
	protocol: TCP
	- containerPort: 11222
	protocol: TCP
	resources: {}
	terminationMessagePath: /dev/termination-log
	volumeMounts:
	- mountPath: /opt/jboss/infinispan-server/standalone/configuration/custom
	name: config-volume
	dnsPolicy: ClusterFirst
	restartPolicy: Always
	securityContext: {}
	volumes:
	- configMap:
	name: cloud-xml
	name: config-volume
	terminationGracePeriodSeconds: 30
	kind: List
	metadata: {}

view raw Kubernetes Infinispan Deployment hosted with ❤ by GitHub

Conclusion

If you're using any Docker orchestration tool - have a look at provided tools. OpenShift and Kubernetes ConfigMaps are really great for this.

However if you need a fine grained control - either extend our Docker image (this is the preferred way) or simply fork and modify it.

Happy configuring and scaling!

Hotrod clients C/C# 8.0.0.Final released!

Dear Infinispan community,
I'm glad to announce the Final release of the C++ and C# clients version 8.0.0.

You can find the download on the Infinispan web site:

http://infinispan.org/hotrod-clients/

Major new features for this release are:

• queries
• remote script execution
• asynchronous operation (C++ only)

plus several minor and internal updates that partially fill the gap between C++/C# and the Java client.

Some posts about the 8 serie of the C++/C# clients have been already published on this blog, you can recall them clicking through the list below.

• HotRod C++ Native Client 8 Series
• Remote Execution with C++ client 
• Infinispan queries with C++ client

The equivalent C# examples are collected here:

https://github.com/rigazilla/dotnet-client-examples

Enjoy!