How to set up Alertmanager configs in Monitoring V2 in Rancher

This document (000020737) is provided subject to the disclaimer at the end of this document.

Environment

Validated with Rancher Monitoring version v2 (v102.0.1+up40.1.2).
It should work all all monitoring v2.

Situation

Before we get started:

Admin permissions are needed
Gather the required information for the alerting you'd like to set up
Monitoring will need to be installed

Please note that I will set up Slack alerts in this example

To set up Slack, add and configure the Incoming webhooks app within your Slack environment
Copy the Webhook URL and paste it into a blank notepad

Resolution

Create an Opaque Secret in the cattle-monitoring-system namespace
- Click on Projects/Namespaces
- Select cattle-monitoring-system
- Then click on Secrets
- Then select Create
Choose Opaque Secret
- Specify a key name (Insert name here for the secret)
- Then paste the Slack Webhook URL under Value
- Verify that the secret has been created successfully
Next, select the Monitoring tab on the left side
- Select Alerting
- Then create a new AlertManagerConfig in the same namespace
- Add a name for this configuration
- Then select Create
Once added, we will need to Edit Config
- From there, we can add a new Receiver (This is where we can specify which type of notifications to receive)
- For Slack, select Add Slack
- Under Secret with Slack Webhook URL, select the Secret name
- Then under the Key drop-down, we'll see the new, generated webhook secret key
- Next, specify the channel that the notifications will be sent to
  - (Optional) specify a proxy if applicable
- Then select Create
After creation, a Slack receiver will be shown, pointing to the webhook URL secret
- Next, edit the AlertManagerConfig again and point the base route of the config to the slack receiver
- Under the drop-down, our receiver will show up
- Select the Receiver
- Specify Groupings and Matchers and set different intervals here
  - For faster testing, change the default interval times to quicker times like 5 seconds, 10 seconds, 1 minute, etc.
- Then click Save
Under status, now we see the resulting config in the Alertmanager section in the Rancher UI
Then, if everything has been set up correctly, a notification should be sent to the desired Slack channel

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.