Document toolboxDocument toolbox

Data types in Devo

When raw data arrives at Devo and is grouped into tables, each row represents an event and each field represents a different type of data. Devo automatically assigns one of its recognized data types to each field. There are two ways of checking a field data type.

In the search window, hover over a field header and a green icon will appear on the left. Hover over the icon until the cursor changes shape and a contextual menu will appear. You can see the name of the field and data type below the name.

If you want to check all data types of all fields in your query, open the Field viewer at the left of the window. A window with all fields and data types will appear.

When performing query operations such as applying filters or creating new fields, you must take into account the data types of the fields you add as arguments, since each function requires specific types of data.

For example, say you want to get the regions corresponding to a set of IP addresses. You must create a new field using the Geolocated Region Name operation, and add an Ip type field as an argument. The created field will return the corresponding regions as sentences of characters (String data type). 

The following table describes the available data types in Devo:

Data type

Description

Example

Data type

Description

Example

str

The string data type represents a sequence of characters.

"Hello user 25", "user@company.com"...

int4

The integer data type represents mathematical integers, that is, numbers that can be zero, positive or negative values but no decimal values.

21, 0, -4...

int8 / bigint

The integer data type represents mathematical integers, that is, numbers that can be zero, positive or negative values but no decimal values. Bigints are values superior to int64.

12345678901234567890987654321

float

The float data type represents a decimal (floating-point) number.

3.25907, 5.542...

boxar(int1)

The boxar(int1) data type represents a byte array in the form of a hexadecimal string.

41, 616263...

bool

The bool data type represents the values true and false.

true, false

duration

The duration data type represents an amount of time. It’s composed by an integer number followed by a suffix: d (for days), h (for hours), m (for minutes) and s (for seconds). Milliseconds have no suffix and are simply represented with the corresponding integer number.

1d, 20m, 340...

timestamp

The timestamp data type represents a date in 'yyyy-MM-dd HH:mm:ss.SSS' format.

2019-02-06 08:58:51.230

geocoord

The geocoord data type represents a valid geocoord, which may be either a set of sexagesimal coordinates based on latitude and longitude values (latlon), or a hash representation of coordinates (geohash).

40°24'46.3"N 3°41'43.8"W, ezjmguvj...

ip4

The ip4 data type represents IPv4 addresses in dot-decimal notation.

43.92.193.15

ip6

The ip6 data type represents IPv6 addresses.

2001:0db8:85a3:0000:0000:8a2e:0370:7334

net4

The net4 data type represents an IPv4 network in x.x.x.x/s format.

0.0.0.0/8

net6

The net6 data type represents an IPv6 network in x:x:x:x:x:x/s format.

2001:db8:abcd:12::/64

regexp

The regexp data type represents a regular expression.

[^\w]

template

The template data type represents a substitution string and is required as an input argument for the Substitute (subs) and Substitute all (subsall) operations.

The Template (template) operation outputs values in this data type.

***, \1

dc

The distinct count data type represents an estimated count of distinct elements in a data stream.

The HyperLogLog++ (hllpp) and HyperLogLog++ unpack (unpackhllpp) aggregation operations output values in this data type.

The Approximated estimation (estimation), and HyperLogLog++ pack (pack) operations require this data type as input.

2.00000005960464

image

 

The image data type represents a Base64 encoded image. 

The To image (image) operation processes a Base64 enconded string and outputs its results in this data type.

This data type is not required as input for any other operation.

mac

The mac data type represents a valid MAC address.

00:0a:95:9d:68:16

namepattern

The namepattern data type represents a part of a table name that can be used as a wildcard to filter sets of names in set(name) format using the Any name matches (anymatches) operation. You can transform strings to name patterns using the Glob pattern on names (nameglob) operation.

my.app, demo...

set(name)

The set(name) data type represents a set of table names. You can only get a field with this type of data after performing a global search. You will see a field called tables that returns a list of tables that contain the terms you looked for.

my.app.test, my.app.test2, demo.ecommerce.data...

json

This data type identifies content formatted in JSON format. This type is required as input for the Jq eval operation and is the output type for the jsonparse operation.

Learn more here.

json type fields cannot yet be used to define alerts, aggregation tasks or injections, or be added as grouping keys when you group your data. This will be possible in future versions of the application.

{"id":3451,"first_name":"John","last_name":"Doe","email":"john.doe@devo.com"}

jq

This data type identifies a jq filter which is required as input for the Jq eval operation. This data type is created using the Jq filter compile operation.

jq is a command-line JSON processor. Learn more here.

.email

set

This data type identifies a collection of unique values arranges in no particular order.

Learn more here.

{1:15,2:43,3:92,4:193}

 

map

A map data type represents an unordered set of key-value pairs. A map pair is a key and value mapped to one another.

Learn more here.

{1:null,2:"2.00000005960464 ",3:"my.app",null:"username"}

array

This data type identifies a collection of equal elements arranged as a series.

This type is required as input for the sort, dropnulls, join, bag, index of and sum operations and is the output type for the array and mkarray operations.

Learn more here.

["usernameJohn","usernameMary","usernameJoe", "usernameJane"]

tuple

A tuple is a collection of sorted elements of any type (repeated or not). The difference between an array and a tuple lies in the fact that in the array all the elements are internally converted to the same type, while in the tuple they are not (each tuple element retains its type).

Learn more here.

•(1.33, 400)
•(One, two, Number 3)
•(One, true)
•(self, 12)

table

This data type identifies a list of fields in table format.

This data type is used in subqueries and is only represented in query API.

-

Â