Monitoring Spring Boot with Micrometer and Prometheus

by

This tutorial covers how to monitor Spring Boot metrics using Prometheus. We will also learn how to trigger Alerts using the Alert Manager component.

This tutorial covers several steps which we will summarize here:

  1. Enable Metrics in your Spring Boot application
  2. Install Prometheus
  3. Capturing Spring Boot Metrics from Prometheus
  4. Install Prometheus’ Alert Manager
  5. Configure Alerts for Spring Boot Metrics

Let’s get started!

Enable Spring Boot Metrics

Spring Boot provides a library called Actuator which helps you to monitor and manage your applications.

If you want a quick introduction to Spring Boot Actuator, we recommend checking this article: Configuring Spring Boot Actuator

Behind the hoods, Spring Boot Actuator uses Micrometer to instrument and capture different
metrics from the code, such as: JVM Memory usage, CPU usage, Connection Pool information, HTTP requests and so on.

To enable monitoring, as first step you have to include the following dependencies in your Spring Boot project:

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

<dependency>
	<groupId>io.micrometer</groupId>
	<artifactId>micrometer-registry-prometheus</artifactId>
	<scope>runtime</scope>
</dependency>

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-web</artifactId>
</dependency>

Out of the box Spring Boot 2 does not emit metrics (except for Health metrics). Therefore, you have to enable at minimum prometheus metrics in application.properties:

management.endpoints.web.exposure.include=prometheus

On the other hand, if you want to capture all available metrics, you can just include the following exposure in application.properties:

management.endpoints.web.exposure.include=*

Before moving to the next section, verify that your Spring Boot application emits Prometheus metrics. Firstly, start your Spring Boot application:

mvn install spring-boot:run

Then, open another shell and execute:

$ curl http://localhost:8080/actuator/prometheus

# HELP process_uptime_seconds The uptime of the Java virtual machine
# TYPE process_uptime_seconds gauge
process_uptime_seconds 141.871
# HELP jvm_gc_memory_promoted_bytes_total Count of positive increases in the size of the old generation memory pool before GC to after GC
# TYPE jvm_gc_memory_promoted_bytes_total counter
jvm_gc_memory_promoted_bytes_total 0.0
# HELP process_start_time_seconds Start time of the process since unix epoch.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.641307416934E9
# HELP executor_pool_core_threads The core number of threads for the pool
# TYPE executor_pool_core_threads gauge
. . . . .

Great. We will now move to Prometheus installation.

Installing Prometheus

Prometheus is a monitoring system and time-series database that allows us to store time-series data, which includes the metrics of an application over time, a simple way to visualize the metrics, or setting up alerts on different metrics.

To install Prometheus, head to the latest release of Prometheus: https://github.com/prometheus/prometheus/releases/latest

Next, download the release for your OS and unzip in a folder.

We will now configure Prometheus to scrape Spring Boot metrics. To do that, open the prometheus.yml file and add the following job_name in the scrape_configs section:

  - job_name: 'spring-actuator'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 5s
    static_configs:
    - targets: ['localhost:8080']

The above job basically says to collect every 5 seconds metrics which are available at localhost:8080 under the Path “actuator/prometheus”.

Move to the root folder of your Prometheus installation and run:

./prometheus

You should see the following log from the Console:

ts=2022-01-04T15:22:27.281Z caller=main.go:1129 level=info msg="Loading configuration file" filename=prometheus.yml
ts=2022-01-04T15:22:27.283Z caller=main.go:1166 level=info msg="Completed loading of configuration file" filename=prometheus.yml totalDuration=1.829109ms db_storage=860ns remote_storage=1.207µs web_handler=384ns query_engine=1.152µs scrape=1.043746ms scrape_sd=42.932µs notify=32.475µs notify_sd=9.708µs rules=407.236µs
ts=2022-01-04T15:22:27.283Z caller=main.go:897 level=info msg="Server is ready to receive web requests.

Next, head to the following URL to see all registered endpoints: http://localhost:9090/targets

spring boot monitor prometheus

The first metric is the built-in Prometheus endpoint. The second one, is our spring-actuator Endpoint we have just added. As a proof of concept, you can stop your Spring Boot application and verify that the State switch to DOWN.

Capturing Spring Boot Metrics

As next exercise, we will monitor some metrics from our Spring Boot application. Click on the Graph upper tab and enter an expression to monitor.

For example, process_cpu_usage:

prometheus spring boot

Within the Path “actuator/prometheus” there are many interesting metrics, for example the http_server_requests_seconds_sum:

As you can see, capturing Spring Boot metrics is quite straightforward. On the other hand, we would like to have an Alert when some key metrics reach or go below a threshold. You can easily accomplish this task by installing another Prometheus component: the Alert Manager.

Installing the Agent Manager

The Alertmanager manages alerts sent by Prometheus server. It takes care of grouping alerts, and routing them to the correct receiver. For example an email receiver.

Firstly, download the latest version of the Alert Manager from: https://prometheus.io/docs/alerting/latest/alertmanager/

Next, unzip it in a folder of your like. Before starting the Alert manager, we need to enable it on the Prometheus configuration file. Open again the prometheus.yml file and register the Alert manager at the default address in the alerting section:

alerting:
   alertmanagers:
     - scheme: http
       static_configs:
         - targets: ['localhost:9093']

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - "rules.yml"

Alerts are generally included in a separate file, in our case it’s rules.yml. Create this file in the same folder with this content:

groups:
- name: example
  rules:

  - alert: service_down
    expr: up{job="spring-actuator"} == 0
    labels:
      severity: major
    annotations:
      description: Service {{ $labels.instance }} is unavailable.
      value: DOWN ({{ $value }})

This is a very basic rules which merely check the value of the job “spring-actuator”.

Now start the Alert Manager:

./alertmanager

Next, start Prometheus:

./prometheus

We are now ready to capture Spring Boot alerts!

Using the Alert Manager to signal Spring Boot metrics

You can reach Prometheus Alert UI from the following URL: http://localhost:9090/alerts

If your Spring Boot application is available there should be no Alerts. Now, try to stop your Spring Boot application:

spring boto alert manager prometheus

As you can see, an Alert is firing since the application is not available.

Let’s write a more complex rule. For example, we want an alert if CPU usage is over a certain threshold. To let this alert fire easily, we will set up a minimal CPU usage (> 0.1).

Edit the rules.yml file to include this rule:

groups:
  - name: default
    rules:
      - alert: CPU_Rate
        expr:  system_cpu_usage > 0.1
        labels:
          severity: high
        annotations:
          summary: WARN ! CPU High

Next, let’s stress a bit our Spring Boot application with some request. You will see that in a matter of a few seconds the Alert will be firing:

spring boot metrics prometheus

Routing Alerts via email

Finally, we can configure the Alert Manager to send mails whenever an alert reaches the firing state. To do that, we need to add the below configuration in alertmanager.yml

global:
 resolve_timeout: 1m

route:
 receiver: 'gmail-notifications'

receivers:
- name: 'gmail-notifications'
 email_configs:
 - to: monitoringinstances@gmail.com
   from: monitoringinstances@gmail.com
   smarthost: smtp.gmail.com:587
   auth_username: monitoringinstances@gmail.com
   auth_identity: monitoringinstances@gmail.com
   auth_password: password
   send_resolved: true

Finally, adjust the above configuration with your email address and smtp server.

Conclusion

This tutorial covered how to install Prometheus and configure it to monitor Spring Boot’s actuator metrics. We also discussed how to create Alerts for your metrics via Alertmanager. As a quick refererece, we include the endpoints which we have used across the example:

Spring Boot’s Prometheus Metrics: http://localhost:8080/actuator/prometheus

Prometheus Target Endpoints: http://localhost:9090/targets

Prometheus Alerts: http://localhost:9090/alerts