Contains tokens - case insensitive (weaktoktains)

[ 1 Description ] [ 2 How does it work in the operations window? ] [ 3 How does it work in the search window? ] [ 4 How does it work in LINQ? ]

The results of this element aren’t visible when editing. Preview the page or publish it to see how it will look.

Description

You can apply this operation either as a Filter or Create field operation:

Filter	Creates a filter that returns only those strings including a specific token, ignoring case. Optionally, you can add one or two boolean values to extend the left and right length of the token.
Create field	Adds a new Boolean field that shows true when a specific token is present in a given string, ignoring case. Optionally, you can add one or two boolean values to extend the left and right length of the token.

Use the Contains tokens (toktains) operation if you need to discriminate between uppercase and lowercase letters.

What is a token?

A token is simply a unit of data composed of alphanumeric characters separated by ASCII symbols. Devo also recognizes as a token any value assigned the ip or ip6 data format. Therefore, not only will Devo identify 10.0.1.2 and aa:bb:cc:dd as tokens, but also their component parts, 10, 0, 1, 2, aa, bb, cc, and dd because these component parts are delimited by ASCII symbols (the periods and colons).

Learn more in How Devo indexes data.

How does it work in the operations window?

Select Filter / Create field in the search window toolbar, then select the Contains tokens - case insensitive operation. This operation requires at least two arguments (Value and contains). Optionally, you can add the right extended and left extended arguments to expand the length of the token.

Argument	Data type	More information

Argument	Data type	More information
Value ^mandatory	string	-
contains ^mandatory	string	-
left extended	boolean	Add this argument if you want to get strings that contain the indicated token with alphanumerical values to the left. For example, if you enter 10 as a token and set the left extended argument to true, you will get strings containing, for example, -10- or .10. but also abc10- or 12310-
right extended	boolean	Add this argument if you want to get strings that contain the indicated token with alphanumerical values to the right. For example, if you enter 10 as a token and set the right extended argument to true, you will get strings containing, for example, -10- or .10. but also -10abc or -10123

If you use the Create field operation, the data type of the values in the new field is boolean (true or false).

Example

In the siem.logtrust.web.activity table, we want to get only the events that include the token language in the headers field strings (value language delimited by ASCII symbols). We want to ignore the case, so tokens including capital letters will be also considered. To do it, we will apply a Filter using the Contains tokens operation.

Active case insensitive

Some operations have a case sensitive and a case insensitive version, for example, Contains - case insensitive (weakhas) and Contains (has, ->). Use these options to display only the sensitive or insensitive versions of these operations, or choose all to show both versions. Operations that don't have a sensitive and insensitive version will be visible regardless of the option selected. You can select the default option in your User preferences, and Admin users can do the same for all the users in the domain in their Domain preferences.

The arguments needed for the filter are:

Value - headers field
contains - Click the pencil icon and enter language

Click Filter data and you will see the following result. As you can see, events containing the token language appear, but also Language.

Click Create field and follow the same steps to add a new Boolean field that shows true when the conditions explained above are met.

How does it work in the search window?

In all data tables, the entire event is logged in a Raw field displaying event data as a string. This string will be logged as various names depending on the table: rawMessage, rawSource, or raw.

Use the Filter on raw field to search for keywords throughout the entire raw data field, instead of filtering by specific field.

The LINQ expression will differ as follows (in order of search priority):

where weaktoktains (rawMessage, "<value>")"
where weaktoktains (rawSource, "<value>")"
where weaktoktains (raw, "<value>")"

How does it work in LINQ?

Use the operator where... to apply the Filter operation and select... as... to apply the Create field operation. These are the valid formats of the Contains tokens operation:

weaktoktains(string1, string2)
weaktoktains(string1, string2, boolean_left)
weaktoktains(string1, string2, boolean_left, boolean_right)

Examples

You can copy the following LINQ script and try the above example on the siem.logtrust.web.activity table:

from siem.logtrust.web.activity
  where weaktoktains(headers, "language")

And this is the same example using the Create field operation:

from siem.logtrust.web.activity
  select weaktoktains(headers, "language") as token_language