How to use the Alert Manager

Overview

The Alert Manager application is a comprehensive monitoring tool designed to manage and track the status of system services, Edge Node's telemetry, and external variables. It operates through three primary functions: service monitoring, telemetry, and integration with external systems via a REST API. Here's how each of these components works:

Service Monitoring: The application regularly checks the status of configured services, ensuring that they are operating correctly. It uses a set interval to perform checks, and if a service fails, it will attempt to recover it based on a predefined number of retries. If recovery attempts fail, an alert is triggered.
Telemetry: Every 60 seconds, the application monitors resources usage, comparing it against user-defined warning and critical thresholds. Alerts are generated when disk usage surpasses these thresholds, helping prevent system overloads or failures.
API for External Variables: The application includes a REST API that allows external systems to send status data in JSON format. This feature enables the integration of external variables, which can be monitored and trigger alerts through the Alert Manager.

With these capabilities, the Alert Manager ensures that services and critical system metrics are continuously monitored, and alerts are promptly generated when issues arise, helping to maintain system health and prevent downtime.

Adding the Alert Manager Application to your Library

You can find the Alert Manager app in the Marketplace.

Alert Manager in Barbara Marketplace

Go to Barbara Marketplace, search for Alert Manager and add it to your Panel's library.

note

You will find the Alert Manager app in this link of the Barbara Marketplace.

Installing the Alert Manager and setting the notifications for monitoring the disk usage

To begin, navigate to your node's Device Details and install the Alert Manager application on your Edge Node. Once in the panel, follow the installation wizard and configure the following settings:

App Secrets. These are the MQTT Broker-related credentials. If you wish to enable MQTT notifications, enter the required parameters for connecting to the MQTT Broker. If you do not need MQTT notifications, you can leave these values as default.

App Secrets in the wizard

App Config. This section allows you to configure key parameters for the service. Use the following JSON parameters to set up disk usage monitoring.

App Config in the wizard

{
  "alarmmanager": {
    "debugLevel": "info",
    "notification": {
      "email_enabled": true,
      "email_recipients": [
        "user@company.tech"
      ],
      "mqtt_enabled": false,
      "summary_enabled": false,
      "summary_time_utc": "09:00"
    },
    "telemetry": {
      "disk_alarm_warning_threshold": 75,
      "disk_alarm_critical_threshold": 90
    },
    "services": []
  }
}

Important parameters to configure here are:

Parameter	Subparameter	Description
`notification`	`email_enabled`	Enable the mail notification service.
`notification`	`email_recipients`	Set the list of recipients to receive the alert notification mails.
`mqtt_notification`		Enable the MQTT Notification service.
`telemetry`	`disk_alarm_warning_threshold`	Threshold for the disk warning notification.
`telemetry`	`disk_alarm_critical_threshold`	Threshold for the disk critical notification.

By configuring these parameters, you will receive an email notification whenever disk usage exceeds the warning or critical thresholds. If you set the mqtt_notification parameter to true, an MQTT notification will also be sent to the MQTT Broker (ensure that an MQTT broker is already installed for this feature to work).

Setting Up the Alert Manager for monitoring a service

Now, let's say you need to monitor the status of a service that is already installed on your Edge Node. For example, we'll configure notifications for an MQTT Broker application that is already deployed on your node.

In that case, we need to configure the services parameter config in the following way:

{
  "alarmmanager": {
    "debugLevel": "info",
    "notification": {
      "email_enabled": true,
      "email_recipients": [
        "eramirez@barbara.tech"
      ],
      "mqtt_enabled": true,
      "summary_enabled": true,
      "summary_time_utc": "16:50"
    },
    "telemetry": {
      "disk_alarm_warning_threshold": 40,
      "disk_alarm_critical_threshold": 90
    },
    "services": [
      {
        "enabled": true,
        "serviceName": "mqttbbr",
        "interval": 30,
        "retries": 1
      }
    ]
  }
}

Important parameters to configure here for every service included (check that the services parameter is an array of objects containing):

Parameter	Description
`enabled`	Enable the notification for the service.
`serviceName`	The name of the service being monitored
`interval`	The time interval (in seconds) between each health check for the service.
`retries`	The number of retry attempts to recover the service if a failure is detected before triggering an alarm or notification.

Did you know?

You can check the name of a service in the section Docker Compose Info of the application.

For example, in the case of an MQTT Broker, check the service name is: mqttbbr.

App Secrets in the wizard

Monitoring external parameters in Alert Manager

The Alert Manager application exposes a REST API endpoint in the URL: http://alarmmanager:5100/event.

External systems can send a JSON payload to that API, which includes the alarm configuration and the current status of the variable. The Alert Manager processes this data and, based on the provided configuration, checks if the variable’s value meets the conditions required to trigger an alarm. If the value falls outside the defined criteria (e.g., it exceeds the allowed range), the application generates and sends the appropriate alert. This API service allows for the extension of monitoring to additional system components or external variables, enabling them to be managed within the same alarm platform.

The JSON payload sent to the API must have the following format (check the Technical Notes of the Alert Manager application in marketplace to see the specification):

{
    "enabled": false,
    "name": "Low pressure",
    "deviceDisplayName": "machine_01",
    "displayName": "variable_01",
    "value": 20,
    "operation": "<",
    "threshold": 19,
    "retries": 3
}

For example, you can send events to that API in a Python application with the following code:

import requests
import json

url = "http://alarmmanager:5100/event"

payload = json.dumps([
  {
    "enabled": True,
    "name": "Low pressure",
    "deviceDisplayName": "machine_00",
    "displayName": "variable_01",
    "value": 18,
    "operation": "<",
    "threshold": 19,
    "retries": 3
  },
  {
    "enabled": False,
    "name": "Low pressure",
    "deviceDisplayName": "machine_01",
    "displayName": "variable_01",
    "value": 20,
    "operation": "<",
    "threshold": 19,
    "retries": 3
  }
])
headers = {
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Monitoring external parameters from Marketplace's Industrial Connectors

The industrial connectors available in the Marketplace are pre-integrated with the Alert Manager, allowing you to easily monitor the read variables by simply adjusting their configuration. Refer to the Technical Notes for each connector to learn how to configure it correctly and begin monitoring any value.

note

Check the connectors' Technical Notes to configure and monitorize its parameters here.

Setting Up the Alert Manager to receive Summary Notification

You can configure the Alert Manager application to send a daily notification summarizing the most important events from the day. Use the following parameters in the JSON configuration to enable this feature:

Parameter	Subparameter	Description
`notification`	`summary_enabled`	Enable sending a daily summary mail.
`notification`	`summary_time_utc`	Sets the time to send the daily summary mail.

Here is an example of a JSON configuration with the summary email enabled:

{
  "alarmmanager": {
    "debugLevel": "info",
    "notification": {
      "email_enabled": true,
      "email_recipients": [
        "user@company.tech"
      ],
      "mqtt_enabled": false,
      "summary_enabled": false,
      "summary_time_utc": "09:00"
    },
    "telemetry": {
      "disk_alarm_warning_threshold": 75,
      "disk_alarm_critical_threshold": 90
    },
    "services": []
  }
}

Overview​

Adding the Alert Manager Application to your Library​

Installing the Alert Manager and setting the notifications for monitoring the disk usage​

Setting Up the Alert Manager for monitoring a service​

Monitoring external parameters in Alert Manager​

Monitoring external parameters from Marketplace's Industrial Connectors​

Setting Up the Alert Manager to receive Summary Notification​