APM for your Spring Boot microservices on OpenShift

Have no doubt, OpenShift is a great platform for running microservices architectured applications! However, when it comes to Application Performance Management (APM) some may think there’s a lack because no components are coming off-the-shelf in the platform. Indeed this makes sense cause APM is a tricky problem and may be addressed in many mays depending on application technology – and after all microservices architecture is also about picking the right techno for the right problem! 😉 However, this article introduces a quite non-intrusive solution for Spring Boot (and more generally Servlet based) microservices by using Hawkular APM.

Hawkular APM ?

Hawkular logo

OpenShift provides a built-in monitoring tool called Hawkular. That tool is in charge of collecting metrics from Docker containers through Kubernetes interface and storing, aggregating, visualizing them. The metrics collected are CPU, Memory, Disk and Network usage. Hawkular offers a “black-box” view of container performances but does not deal with application metrics like service performance or distribution of response time through application layers. For that specific cases, Hawkular community is working on another module called Hawkular APM that provides insight into the way an application executes across multiple (micro) services in a distributed (e.g. cloud) environment. Hawkular APM supports agent-based instrumention for Java or API based metrics collection for other languages. It also supports distributed tracing frameworks like ZipKin or OpenTracing. It is not provided as a built-in module in OpenShift but it’s really easy to setup on OpenShift and having Java application being monitored without touching the source code.

OpenShift part setup

Assuming you already have an OpenShift cluster running (through a complete installation, a local oc cluster up, the Red Hat Container Development Kit or something like Minishift), the first step is to install the Hawkular APM server that will be responsible of storing, aggregating and visualizing the application metrics.

The following commands can be issued in any OpenShift project/namespace of your choice.

[root@rhel72-openshift Development]# oc create -f https://raw.githubusercontent.com/jboss-dockerfiles/hawkular-apm/master/openshift-templates/hawkular-apm-server-deployment.yml
secret "hawkular-apm-admin-account" created
route "hawkular-apm" created
deploymentconfig "hawkular-apm" created
deploymentconfig "hawkular-apm-es" created
serviceaccount "hawkular-apm" created
serviceaccount "hawkular-apm-es" created
service "hawkular-apm" created
service "hawkular-apm-es" created
[root@rhel72-openshift Development]#

It’s just as easy as this! OpenShift template for Hawkular APM creates deployments for server and Elasticsearch backend, services and route for accessing server. Just use the created route within a browser and access the UI after having authenticated with default admin/password created. Here is the result on below screenshot:

hawkular-apm

Container part setup

The second part of the process is now to prepare a base Docker file for OpenShift that will be used for containerizing our Spring Boot applications from sources. In OpenShift, this process of directly using sources and handling the Dockerization is called Source-to-image. For our Spring Boot / APM purpose, we will start from a base image handling basic Java Dockerization.

The goal of customization is just to retrieve Hawkular APM Java agent and to make it available as a lib within the base image that will later contain our Spring Boot fat jar application. A little trick here is to temporary switch to root user to chmod correctly the Jar file. Otherwise, JVM that will execute with other user will not be able to read Java agent Jar during the boot. Here’s the DockerFile:

FROM docker.io/fabric8/s2i-java:1.3.6

ENV APM_VERSION=0.13.0.Final
ENV APM_AGENT=/libs/hawkular-apm-agent.jar

ADD https://repository.jboss.org/nexus/service/local/artifact/maven/redirect?r=releases&g=org.hawkular.apm&a=hawkular-apm-agent&v=$APM_VERSION&e=jar $APM_AGENT

# Temporary switch to root
USER root

RUN chmod 444 /libs/hawkular-apm-agent.jar

# S2I requires a numeric, non-0 UID. This is the UID for the jboss user in the base image
USER 185

Now that you’ve got this file, you can produce your own base Docker image and store it into a Docker registry for a later use from OpenShift. You can use for example the Docker Hub registry.

$> docker build -t lbroudoux/s2i-java-apm .
$>...
$> docker push lbroudoux/s2i-java-apm
$>...

Finally, in order to use this newly built image into OpenShift and to use it conveniently, we may want to create some ImageStream and Template. We’re are not going in details on how to create those files. The one I have used for testing and demonstrating are just available in my GitHub sample repository in the /openshift directory.

Just use these 2 files to declare resources into your OpenShift installation that way:

[root@rhel72-openshift Development]# oc create -f https://raw.githubusercontent.com/lbroudoux/spring-boot-hello/master/openshift/spring-boot-apm-is.json -n openshift
imagestream "spring-boot-apm" created
[root@rhel72-openshift Development]# oc create -f https://raw.githubusercontent.com/lbroudoux/spring-boot-hello/master/openshift/spring-boot-apm-template.json -n openshift
template "spring-boot-apm" created

Bringing it all together!

It’s now time to create a Spring Boot application to check everything is ok. For that, go to the project/namespace of your choice and use the spring-boot-apm template that is now available. The default of this template uses the sample Spring Boot microservices of my GitHub repository: https://github.com/lbroudoux/spring-boot-hello. You have nothing to change to have something running. After having pressed the “Create” button, build starts (checking out sources, invoking maven build and so on…) and deployment should end-up in a few minutes.

At that time, application is up and you should have a microservice running at http://spring-boot-app-project.domain.com/api/hello?name=World. Check your exact route configuration through OpenShift console. However if you check Hawkular APM console back again, you’ll did not notice anything new… That’s because application performance management has not been turned on, it is disabled by default. In order to turn it on, go to your application deployment configuration and manage to add the environment variables like below:

spring-boot-deployment

In order to get APM up, you have to add these 4 variables:

  • HAWKULAR_APM_URI: the HTTP URI for accessing your hawkular server (make sur to not use the HTTPS URL, I did not manage to got it working for now),
  • HAWKULAR_APM_USERNAME: the username for connecting APM server, use default admin user,
  • HAWKULAR_APM_PASSWORD: the password for connecting APM server, use default admin password,
  • JAVA_OPTIONS: the JVM options for enabling Java agent, use -javaagent:/libs/hawkular-apm-agent.jar=boot:/libs/hawkular-apm-agent.jar that refers to the Hawkular agent previously added to image libs.

Once you’ve hit the “Save” button, deployment is updated, a new container is deployed and the Spring Boot application is now instrumented by Hawkular Java agent. Make few calls to the previous “Hello World” microservice and go check the results in Hawkular APM…. tadam !

hawkular-apm-data

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s