4.3 Tasks: Exporter as a sidecar
Task 4.3.1: Deploy a database and use a sidecar container to expose metric
Task description:
As we’ve learned in Lab 4 - Prometheus exporters when applications do not expose metrics in the Prometheus format, there are a lot of exporters available to convert metrics into the correct format. In Kubernetes this is often done by deploying so called sidecar containers along with the actual application.
Use the following command to deploy a MariaDB database your monitoring or application namespace on CAAST.
Create the following deployment (training_baloise_mariadb-deployment.yaml)
apiVersion: apps/v1
kind: Deployment
metadata:
name: mariadb
labels:
app: mariadb
spec:
selector:
matchLabels:
app: mariadb
strategy:
type: Recreate
template:
metadata:
labels:
app: mariadb
spec:
containers:
- image: quay.balgroupit.com/acend/mariadb:10.5
name: mariadb
args:
- --ignore-db-dir=lost+found
env:
- name: MYSQL_USER
valueFrom:
secretKeyRef:
key: database-user
name: mariadb
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
key: database-password
name: mariadb
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
key: database-root-password
name: mariadb
- name: MYSQL_DATABASE
valueFrom:
secretKeyRef:
key: database-name
name: mariadb
livenessProbe:
tcpSocket:
port: 3306
ports:
- containerPort: 3306
name: mariadb
volumeMounts:
- mountPath: /var/lib/mysql
name: data
volumes:
- name: data
emptyDir: {}Create the following secret (training_baloise_mariadb-secret.yaml)
apiVersion: v1
kind: Secret
metadata:
name: mariadb
labels:
app: mariadb
data:
database-name: YWNlbmRleGFtcGxlZGI= # example
database-password: bXlzcWxwYXNzd29yZA== # example
database-root-password: bXlzcWxyb290cGFzc3dvcmQ= # example
database-user: YWNlbmQtdXNlcg== # exampleCreate the following service (training_baloise_mariadb-service.yaml)
apiVersion: v1
kind: Service
metadata:
name: mariadb
labels:
app: mariadb
prometheus-monitoring: 'true'
spec:
ports:
- name: mariadb
port: 3306
protocol: TCP
targetPort: 3306
selector:
app: mariadbThis will create a Secret (username password to access the database), a Service and the Deployment .
- Deploy the mariadb exporter
from https://registry.hub.docker.com/r/prom/mysqld-exporter/
as a sidecar container
- Alter the existing MariaDB deployment definition to contain the side car
- Create a ServiceMonitor to instruct Prometheus to scrape the sidecar container
Hints
First we need to alter the MariaDB deployment training_baloise_mariadb-deployment.yaml by adding the MariaDB exporter as a second container.
apiVersion: apps/v1
kind: Deployment
metadata:
name: mariadb
labels:
app: mariadb
spec:
selector:
matchLabels:
app: mariadb
strategy:
type: Recreate
template:
metadata:
labels:
app: mariadb
spec:
containers:
- image: quay.io/prometheus/mysqld-exporter:v0.14.0
name: mariadb-exporter
env:
- name: MYSQL_USER
valueFrom:
secretKeyRef:
key: database-user
name: mariadb
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
key: database-password
name: mariadb
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
key: database-root-password
name: mariadb
- name: MYSQL_DATABASE
valueFrom:
secretKeyRef:
key: database-name
name: mariadb
- name: DATA_SOURCE_NAME
value: $(MYSQL_USER):$(MYSQL_PASSWORD)@(127.0.0.1:3306)/$(MYSQL_DATABASE)
ports:
- containerPort: 9104
name: mariadb-exp
- image: quay.balgroupit.com/acend/mariadb:10.5
name: mariadb
args:
- --ignore-db-dir=lost+found
env:
- name: MYSQL_USER
valueFrom:
secretKeyRef:
key: database-user
name: mariadb
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
key: database-password
name: mariadb
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
key: database-root-password
name: mariadb
- name: MYSQL_DATABASE
valueFrom:
secretKeyRef:
key: database-name
name: mariadb
livenessProbe:
tcpSocket:
port: 3306
ports:
- containerPort: 3306
name: mariadb
volumeMounts:
- mountPath: /var/lib/mysql
name: data
volumes:
- name: data
emptyDir: {}Then extend the service training_baloise_mariadb-service.yaml by adding a second port for the MariaDB exporter.
apiVersion: v1
kind: Service
metadata:
name: mariadb
labels:
app: mariadb
prometheus-monitoring: 'true'
spec:
ports:
- name: mariadb
port: 3306
protocol: TCP
targetPort: 3306
- name: mariadb-exp
port: 9104
protocol: TCP
targetPort: 9104
selector:
app: mariadbThen we also need to create a new ServiceMonitor training_baloise_mariadb-servicemonitor.yaml.
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
app.kubernetes.io/name: mariadb
name: mariadb
spec:
endpoints:
- interval: 60s
port: mariadb-exp
scheme: http
path: /metrics
selector:
matchLabels:
prometheus-monitoring: 'true'
Verify that the target gets scraped in the Prometheus user interface
. Target name: serviceMonitor/<team>-monitoring/mariadb/0 (It may take up to a minute for Prometheus to load the new configuration and scrape the metrics).
Task 4.3.2: generic-chart MariaDB deployment (optional)
Task description:
- Deploy the mariadb exporter from quay.io/prometheus/mysqld-exporter as a sidecar container.
- Define all parameters using the generic-chart .
Hints
Create an application on CAAST and deploy the following configuration.
Chart.yaml:
apiVersion: v2
name: example-mariadb
version: 1.0.0
dependencies:
- name: generic-chart
version: 3.13.1
repository: https://charts.shapp.os1.balgroupit.com/shared/release/
alias: example-mariadbvalues.yaml:
example-mariadb:
replicaCount: 1
image:
repository: quay.balgroupit.com/acend/mariadb
tag: 10.5
additionalContainers:
- name: "mariadb-exporter"
image: quay.io/prometheus/mysqld-exporter:v0.14.0
env:
- name: MYSQL_USER
valueFrom:
secretKeyRef:
key: database-user
name: mariadb
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
key: database-password
name: mariadb
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
key: database-root-password
name: mariadb
- name: MYSQL_DATABASE
valueFrom:
secretKeyRef:
key: database-name
name: mariadb
- name: DATA_SOURCE_NAME
value: $(MYSQL_USER):$(MYSQL_PASSWORD)@(127.0.0.1:3306)/$(MYSQL_DATABASE)
network:
mariadb-exp:
servicePort: 9104
serviceMonitor: {}
mariadb:
servicePort: 3306
env:
- name: MYSQL_USER
valueFrom:
secretKeyRef:
key: database-user
name: mariadb
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
key: database-password
name: mariadb
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
key: database-root-password
name: mariadb
- name: MYSQL_DATABASE
valueFrom:
secretKeyRef:
key: database-name
name: mariadb
volumes:
- name: data
emptyDir: {}
volumeMounts:
- mountPath: /var/lib/mysql
name: datatemplates/secret.yaml:
apiVersion: v1
kind: Secret
metadata:
name: mariadb
labels:
app: mariadb
data:
database-name: YWNlbmRleGFtcGxlZGI= # example
database-password: bXlzcWxwYXNzd29yZA== # example
database-root-password: bXlzcWxyb290cGFzc3dvcmQ= # example
database-user: YWNlbmQtdXNlcg== # exampleVerify that the target gets scraped in the Prometheus user interface
. Target name: application-metrics/mariadb/0 (it may take up to a minute for Prometheus to load the new configuration and scrape the metrics).
Make sure to remove the files Chart.yaml, values.yaml and templates/secret.yaml once finished.