Deviation alert type

[ 1 Overview ] [ 2 What data do I need to create this alert? ] [ 3 Defining the alert ] [ 4 Using column values in the Summary and Description ] [ 5 Query example ]

Overview

The deviation method triggers an alert every time an aggregated value of a single grouped element is significantly higher or lower than the median value of all the elements within the same grouping period. This alert type is similar in concept and execution to the gradient type; they both use deviations from values to trigger alerts, however, they differ in the value they use to calculate the deviation. In the case of the deviation type, it is from the median of the values in the same period while in the gradient type, it is from the analogous value of the previous period with data. See the following picture for a more visual explanation.

This type of alert could be useful when monitoring periodic tasks and their data patterns to be informed whenever the aggregated values inside a period differ too much from the frequency distribution midpoint.

What data do I need to create this alert?

To create an alert using this triggering method, your query must group events by at least one grouping key using a time-based option and add an aggregation. However, to have meaningful data for the alert, it is necessary to group by at least two keys.

If you did not group, this alert type will not appear for you to select in the alert definition window.
If you grouped without the necessary key or used a no time-based option, the alert variables will not appear for you to define and a message will inform you about the requirements you still need to meet.

If you did not aggregate, you will not have the column you need for the Add a numeric column field (see the following section) and therefore you will not be able to create the alert.

Defining the alert

After selecting this type of alert, you have to define the following variables:

This setting specifies the proportions of the deviation from the median, in other words, the upper and lower bound that must be exceeded for an alert to be triggered. Write the desired number.

This setting specifies the method to analyze, according to the designated thresholds, the deviation from the median of the values in the same period; in other words, the way in which the threshold will be considered (either as an absolute value or as a percentage). Select the desired option.
The following formulas describe the calculations performed using absolute values or percentages to check if the alert has to be triggered. In both cases, i represents each of the values of the grouping period.

Absolute: abs(median of values in group) - value(i) > threshold.
Using an absolute value means that the threshold specified will be considered as the number above and below which the alert will be triggered. For example, if the median is 100 and the threshold specified is 50 means that an alert will be triggered above 150 and below 50.
When using absolute values it is important to use a threshold that is consistent with the range of values, otherwise, you might trigger alerts constantly or hardly ever.
Percentage: abs(median of values in group) - value(i) > threshold / 100 * median of values in group.
Using a percentage means that the threshold specified will be considered as the percentage of the median value above and below which an alert will be triggered. For example, if the median is 200 and the threshold specified is 25 means that an alert will be triggered above 250 and below 150.

This setting specifies an aggregation column whose values will be set against the designated threshold to trigger the alert. You can choose from any of the aggregation columns added to the query but you cannot add more than one. Drag the required column into the field below or select it on the table and click the Add selected column button.
You must add one column, otherwise, you will receive an error message prompting you to do so in order to successfully create the alert.

Using column values in the Summary and Description

You can use the $columnName command to display in the Summary and Description fields the column values of the events that triggered the alert. This command can be employed with the names of the columns and properties below. Using a different one will not activate the command and will be interpreted as plain text.

$eventdate
$columnName of those resulting from grouping operations.
$columnName of those resulting from aggregation operations.
$median: even though it is not the name of a column, it is a feature that can be used with the deviation alert type to make reference to the calculated median value from which the deviation is measured.

In the following query, you could use:

from demo.ecommerce.data 
where statusCode = 404 
group every 30m by method, timeTaken 
select avg(bytesTransferred) as avgBytesTransferred

$eventdate
$method
$timeTaken
$avgBytesTransferred
$median

Query example

In the demo.ecommerce.data table, imagine that you want to receive an alert whenever the number of events received for client IP addresses displaying the 404 status code is 25% higher or lower than the median in every 30 minute-period.

First of all, you need to filter your query data using the Equal (eq, =) operation, group your query data by two keys using a time-based option and then aggregate it. Then, you need to open the alert definition window, select the deviation type alert and fill in all the details (pay special attention to the specific settings of this alert type).

To save time, you can copy the following query to reproduce the aforementioned example from the demo.ecommerce.data sample table and create a deviation type alert.

from demo.ecommerce.data
where statusCode = 404
group every 30m by clientIpAddress, statusCode
select count() as count

Related Articles: