4. Prometheus exporters on Prometheus Basics Training

4.1 Tasks: Blackbox exporter

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.1.1: Add a blackbox target

We will use the blackbox exporter to create a new probe which accepts a 2xx return code as a valid http return code. This will return the probe_success metric from the blackbox exporter with the value 1, if the http status code is 2xx.

Task description:

Create a probe in the monitoring directory which uses the HTTP prober and expects a 2xx return code as a valid status code
Define https://bitbucket.balgroupit.com/status as a single static target, which the blackbox should probe
Use the following documentation as reference 06 - HTTP and TCP endpoint monitoring

Hints

To configure the blackbox exporter you have to add the following file training_blackbox_target.yaml to your monitoring directory, commit and push the changes:

4.1 Tasks: Blackbox exporter

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.1.1: Install Blackbox exporter

Similar to the basic setup, we are just going to update our configuration of the ArgoCD application to install the Blackbox exporter. Update your monitoring application (charts/user-monitoring/values.yaml) and update the blackboxexporter.enabled flag to true:

charts/user-monitoring/values.yaml:

user: <user> # Replace me
# prometheus
prometheus:
 enabled: true
# thanos-query
query:
 enabled: true
# grafana
grafana:
 enabled: true
 datasources:
 - name: prometheus
 access: proxy
 editable: false
 type: prometheus
 url: http://prometheus-operated:9090
# blackboxexporter
blackboxexporter:
 enabled: true
# pushgateway
pushgateway:
 enabled: false
# alertmanager
alertmanager:
 enabled: false
# thanos-ruler
ruler:
 enabled: false

Commit and push the changes.

4.2 Tasks: Pushgateway

Mon, 01 Jan 0001 00:00:00 +0000

Note

As you will be executing some oc commands in the following labs, make sure you are logged in to your OpenShift Cluster.

You can copy the login Command from the OpenShift UI:

Browse to http://LOCALHOST_OPENSHIFT
Click on your name in the top right
Copy login command
Replace 6443 with 443

Task 4.2.2 - Push metrics to Pushgateway

In this task you’re going to push metrics to the Pushgateway. This is what you would normally do, after a cronjob has completed successfully.

4.3 Tasks: Exporter as a sidecar

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.3.1: Deploy a database and use a sidecar container to expose metric

Task description:

As we’ve learned in Lab 4 - Prometheus exporters when applications do not expose metrics in the Prometheus format, there are a lot of exporters available to convert metrics into the correct format. In Kubernetes this is often done by deploying so called sidecar containers along with the actual application.

Use the following command to deploy a MariaDB database your monitoring or application namespace on CAAST.

4.3 Tasks: Exporter as a sidecar

Mon, 01 Jan 0001 00:00:00 +0000

Task 4.3.1: Deploy a database and use a sidecar container to expose metric

This lab demonstrates that with a mariadb database. In the first step, we are going to install a plain mariadb with a Secret (username password to access the database), a Service and the Deployment .