Steps for creating an alert (DOS detection)

Overview

We will explain in the following sections how new alerts can be added aside from existing standard Security Operations alerts.

The first thing to do is to apply all the filtering you can before you start to define your logic. We’re looking to reduce the amount of data to process.

Filter as soon as you can and every time you are able. Just try to avoid “null” data and remove all “noisy” data during the query creation. That makes your query more efficient and decreases false positives. 

Query creation

Each alert is based on a query that is run continuously over the data stream. When an alert is triggered, it generates a record in the Devo table.

This table is a read-only table. Devo SecOps then enriches the alerts, adding information based on the ‘ExtraData’ field at the end of the alert record.

There are some requirements that we have to follow to create compatible alerts. There are two types of requirements: Mandatory and optional.

SecOps will parse this data, based on the kinds of information values SecOps expects, and create all the new data which makes up the application.

Let's walk through the steps we must take to start creating alerts. An alert is just a query active forever, so we have to compound the query that fits with our search needs and put it to work.

The following picture shows the overall SecOps alert creation process. 

Alert creation

Let's say we want to analyze the requests made to our web server.

Pre-filter

Firstly, we apply filters to obtain the requests where the source IP is not a null value and also that the IP belongs to a public range.

from web.all.access 
where ispublic(srcIp) 
where isnotnull(srcIp)

Aggregation

Then, we want to group by srcIp with one hour as the grouping period. In this case, we're adding a count operation that we use to filter based on the amount of data grouped. 

group every 30m by srcIp 
every 1h 
select count() as count 
where count > 5000

Entity creation

In this step, we will start to deal with SecOps requirements. SecOps needs to fit with a pre-defined nomenclature to detect and process the entities. Here we create the entities and optionally we can apply the whitelisting procedure. Click here to learn more about entities.

In our example, we’re using just one entity, srcIp., so we create a new field following the nomenclature. 

select str(srcIp) as entity_sourceIP

SecOps uses a Common Information Model for entity naming. It’s needed to track entities over the SecOps app. If you want to include more entities please contact Devo Support.

Assign a role to the entity

Next, we need to add following detections using lookups:

• Assign it a role using lookup SecOpsAssetRole.

• Add a description using SecOpsAlertDescription.

• Add a location using SecOpsLocation.

To learn more about lookups, go to SecOps Lookups.

The following example shows how to add to an IP a DNS role with a category server and type system:

asset,class,category,type
8.8.8.8,DNS,server,system

SecOpsAssetRole follows the terminology used by the SecOps app to populate the Entity Graph with known definitions of class (role) and category (entity type).

In our example, we are using just one entity, srcIp, so we create a new field with the entity

select `lu/SecOpsAssetRole/role`(entity_sourceIP) as entity_sourceIP_AssetRole

Whitelisting

In order to avoid some events from some assets, customers can add whitelisting checks on alerts just adding an extra check based on data from a Lookup.

Here we will use the global whitelist lookup SecOpsGWL.

Example:

AssetHyphenRole,reason,description
8.8.8.8-GoogleDNS,Mainstream DNS, Avoid alerts from public services
9.9.9.9,Company DNS,Avoid

The way we can use these two lookups is just getting the role from an asset and then compounding a string to know if the asset + role is the GWL lookup.

Here is an example query that contains three cases. 

• First, we have an IP that it’s on SecOpsGWL lookup and has a role associated with SecOpsAssetRole.

from web.all.access
where requestLength = 1200
select "8.8.8.8" as testIP // create a fake asset
select `lu/SecOpsRole/role`(testIP) as AssetRole // Get asset role from SecOpsRole Lookup
select ifthenelse(isnull(AssetRole),testIP,testIP+"-"+AssetRole) as AssetToCheck
select `lu/SecOpsGWL`(AssetToCheck) as GWL // Check Asset+Role in SecOpsGWL Lookup

• Second, we have an IP that it’s on SecOpsGWL lookup, but has not a role associated with SecOpsAssetRole.

select "9.9.9.9" as testIPAnother // create another fake asset
select `lu/SecOpsRole/role`(testIPAnother) as AssetRoleAnother // Get asset role from SecOpsRole Lookup
select ifthenelse(isnull(AssetRoleAnother),testIPAnother,testIPAnother+"-"+AssetRoleAnother) as AssetToCheckAnother
select `lu/SecOpsGWL`(AssetToCheckAnother) as GWLAnother // Check Asset+Role in SecOpsGWL Lookup

• Third and last, we have an IP that is not on SecOpsGWL and has not a role associated with SecOpsAssetRole.

select "4.4.4.4" as testIPNone // create another fake asset
select `lu/SecOpsRole/role`(testIPNone) as AssetRoleNone // Get asset role from SecOpsRole Lookup
select ifthenelse(isnull(AssetRoleNone),testIPNone,testIPNone+"-"+AssetRoleNone) as AssetToCheckNone
select `lu/SecOpsGWL`(AssetToCheckNone) as GWLNone // Check Asset+Role in SecOpsGWL Lookup
group every 1h by GWLNone, GWL, GWLAnother
every 1h

Using these two lookups allows customers to associate a role or not to an asset.  They can use SecOpsGWL as a simple whitelist or combine it with SecOpsAssetRole to give more context.

Enrichment using lookups

Using Lookups after aggregation ensures that the new columns created are available in SecOps. 

For the full list of lookups for enrichment, see the article on Security Operations lookups.

In this example, we will use a lookup to get the position from Umbrella Top 1M domains:

select `lu/UmbrellaTop1M/position`(fqdn) as enrichStream_entity_destinationHostname_positionInUmbrellaTop1M

Filter

At this point, you can add all the logic you want to implement. You can filter by the new field creating from the enrichment part, filter by statistical operations you made after the grouping, etc.

Following the example, in the enrichment step we can filter based on the lookup result. For example, we want to filter and get only the domains that are in the first one hundred positions in Umbrella Top 1M.

where enrichStream_entity_destinationHostname_positionInUmbrellaTop1M < 100

Location

SecOps performs an entity and location mapping process based on the lookup information. All you need is get that info from SecOpsLocation lookup.

Following the lookup operation standards, we get all the fields from SecOpsLocation lookups and create new five columns that contain location info from the asset (the source IP in the example).

select `lu/SecOpsLocation/country` (str(entity_sourceIP)) as enrichStream_entity_sourceIP_locationCountry
select `lu/SecOpsLocation/city` (str(entity_sourceIP) as enrichStream_entity_sourceIP_locationCity`
select `lu/SecOpsLocation/state` (str(entity_sourceIP)) as enrichStream_entity_sourceIP_locationState
select `lu/SecOpsLocation/lat` (str(entity_sourceIP)) as enrichStream_ entity_sourceIP_locationLat
select `lu/SecOpsLocation/lon` (str(entity_sourceIP)) as enrichStream_entity_sourceIP_locationLon

Categorization

The query to use must have the following fields to define the type of alert, the priority of the alert, MITRE Tactics, and MITRE Techniques.

select `lu/SecOpsAlertDescription/alertType`("SecOpsDenialOfService") as alertType  
select `lu/SecOpsAlertDescription/alertMitreTactics`("SecOpsDenialOfService") as alertMitreTactics  
select `lu/SecOpsAlertDescription/alertMitreTechniques`("SecOpsDenialOfService") as alertMitreTechniques  
select `lu/SecOpsAlertDescription/alertPriority`("SecOpsDenialOfService") as alertPriority

Naming

Finally, as an alert creator, have to fill the last four fields answering the who, the what, the when, the where, and how much. Be as descriptive in the description and message as possible to provide the analyst with the most information in SecOps alert data.

• Summary: Could include column values using $ + column name.

• Description: Could include column values using $ + column name.

• Subcategory: Have to be SecOps

• Alert name: Have to start with “SecOps” prefix and be Upper Camel Case

The end query will look like this:

from web.all.access 
where ispublic(srcIP) 
where isnotnull(srcIP) 
group every 30m by srcIP 
every 1h select count() as count 
where count > 5000 
select str(srcIP) as entity_sourceIP 
select `lu/SecOpsAssetRole/role`(entity_sourceIP) as entity_sourceIP_AssetRole
select `lu/SecOpsLocation/country`(entity_sourceIP) as enrichStream_entity_sourceIP_locationCountry 
select `lu/SecOpsLocation/city`(entity_sourceIP) as enrichStream_entity_sourceIP_locationCity 
select `lu/SecOpsLocation/state`(entity_sourceIP) as enrichStream_entity_sourceIP_locationState 
select `lu/SecOpsLocation/lat`(entity_sourceIP) as enrichStream_entity_sourceIP_locationLat 
select `lu/SecOpsLocation/lon`(entity_sourceIP) as enrichStream_entity_sourceIP_locationLon 
select `lu/SecOpsAlertDescription/alertType`("SecOpsDenialOfService") as alertType   select `lu/SecOpsAlertDescription/alertMitreTactics`("SecOpsDenialOfService") as alertMitreTactics   
select `lu/SecOpsAlertDescription/alertMitreTechniques`("SecOpsDenialOfService") as alertMitreTechniques   
select `lu/SecOpsAlertDescription/alertPriority`("SecOpsDenialOfService") as alertPriority