This is a sub-project of Observability, which exists to help create alert definition files for Prometheus and Grafana.
Probably not. All you need to do is modify the alert definition file with your
alerts and run docker-compose up
- everything should work from there. If it doesn't, please open an issue!
It's YAML. It's a custom structure, but it's very close to the format used by Prometheus. The similarity is intentional, but it differs so that we can also generate Grafana alerts relatively easily as well.
It reads a YAML alert definition file and writes
out alerts.yml for Grafana and alert-rules.yml It does that using a number of templates found in alert-generation/templates
Environment variables - see the command line example below. This helps when running the command in Docker / Kubernetes environments.
# This is the alert definitions input file for processing
ALERT_DEFINITIONS_YAML_FILE_LOCATION=path/to/alert-definitions.yml
# This is where the Grafana alerts file should be written
GRAFANA_ALERTS_YAML_FILE_LOCATION=path/to/grafana/alerts.yml
# This is where the Prometheus alerts file should be written
PROMETHEUS_ALERTS_YAML_FILE_LOCATION=path/to/prometheus/alert-rules.yml
# This runs the alert generation process
python3 generate.py
In order that the demo work with a simple docker-compose up
, the alert generation is already integrated into the
docker-compose.yml. When docker compose is started, a pre-built image
pmwrp/alert-generation:0.1 is run, which performs the alert
generation process. This writes out new alert definitions for both Prometheus and Grafana into the
../config
folder, which are used by their respective containers.
version: '3.7'
services:
# ... other services here
generate-alert-configs:
image: pmwrp/alert-generation:0.1
environment:
- "ALERT_DEFINITIONS_YAML_FILE_LOCATION=/config/alert-definitions.yml"
- "GRAFANA_ALERTS_YAML_FILE_LOCATION=/config/grafana/provisioning/alerting/alerts.yml"
- "PROMETHEUS_ALERTS_YAML_FILE_LOCATION=/config/prometheus/alert-rules.yml"
volumes:
- "./config:/config"
# ... other services here
(This docker-compose.yaml snippet runs the alert generation process over the current alert definition file and writes out alerts.yml for Grafana and alert-rules.yml for Prometheus.)