Data types in Devo
- Juan Tomás Alonso Nieto (Deactivated)
- Julio Vargas
- Tomás Lagomarsino
When raw data arrives at Devo and is grouped into tables, each row represents an event and each field represents a different type of data. Devo automatically assigns one of its recognized data types to each field. There are two ways of checking a field data type.
In the search window, hover over a field header and a green icon will appear on the left. Hover over the icon until the cursor changes shape and a contextual menu will appear. You can see the name of the field and data type below the name.
If you want to check all data types of all fields in your query, open the Field viewer at the left of the window. A window with all fields and data types will appear.
When performing query operations such as applying filters or creating new fields, you must take into account the data types of the fields you add as arguments, since each function requires specific types of data.
For example, say you want to get the regions corresponding to a set of IP addresses. You must create a new field using the Geolocated Region Name operation, and add an Ip type field as an argument. The created field will return the corresponding regions as sentences of characters (String data type).
The following table describes the available data types in Devo:
Data type | Description | Example |
|---|---|---|
str | The string data type represents a sequence of characters. | "Hello user 25", "user@company.com"... |
int4 | The integer data type represents mathematical integers, that is, numbers that can be zero, positive or negative values but no decimal values. | 21, 0, -4... |
int8 / bigint | The integer data type represents mathematical integers, that is, numbers that can be zero, positive or negative values but no decimal values. Bigints are values superior to int64. | 12345678901234567890987654321 |
float | The float data type represents a decimal (floating-point) number. | 3.25907, 5.542... |
boxar(int1) | The boxar(int1) data type represents a byte array in the form of a hexadecimal string. | 41, 616263... |
bool | The bool data type represents the values true and false. | true, false |
duration | The duration data type represents an amount of time. It’s composed by an integer number followed by a suffix: d (for days), h (for hours), m (for minutes) and s (for seconds). Milliseconds have no suffix and are simply represented with the corresponding integer number. | 1d, 20m, 340... |
timestamp | The timestamp data type represents a date in 'yyyy-MM-dd HH:mm:ss.SSS' format. | 2019-02-06 08:58:51.230 |
geocoord | The geocoord data type represents a valid geocoord, which may be either a set of sexagesimal coordinates based on latitude and longitude values (latlon), or a hash representation of coordinates (geohash). | 40°24'46.3"N 3°41'43.8"W, ezjmguvj... |
ip4 | The ip4 data type represents IPv4 addresses in dot-decimal notation. | 43.92.193.15 |
ip6 | The ip6 data type represents IPv6 addresses. | 2001:0db8:85a3:0000:0000:8a2e:0370:7334 |
net4 | The net4 data type represents an IPv4 network in x.x.x.x/s format. | 0.0.0.0/8 |
net6 | The net6 data type represents an IPv6 network in x:x:x:x:x:x/s format. | 2001:db8:abcd:12::/64 |
regexp | The regexp data type represents a regular expression. | [^\w] |
template | The template data type represents a substitution string and is required as an input argument for the Substitute (subs) and Substitute all (subsall) operations. The Template (template) operation outputs values in this data type. | ***, \1 |
dc | The distinct count data type represents an estimated count of distinct elements in a data stream. The HyperLogLog++ (hllpp) and HyperLogLog++ unpack (unpackhllpp) aggregation operations output values in this data type. The Approximated estimation (estimation), and HyperLogLog++ pack (pack) operations require this data type as input. | 2.00000005960464 |
image
| The image data type represents a Base64 encoded image. The To image (image) operation processes a Base64 enconded string and outputs its results in this data type. This data type is not required as input for any other operation. | |
mac | The mac data type represents a valid MAC address. | 00:0a:95:9d:68:16 |
namepattern | The namepattern data type represents a part of a table name that can be used as a wildcard to filter sets of names in set(name) format using the Any name matches (anymatches) operation. You can transform strings to name patterns using the Glob pattern on names (nameglob) operation. | my.app, demo... |
set(name) | The set(name) data type represents a set of table names. You can only get a field with this type of data after performing a global search. You will see a field called tables that returns a list of tables that contain the terms you looked for. | my.app.test, my.app.test2, demo.ecommerce.data... |
json | This data type identifies content formatted in JSON format. This type is required as input for the Jq eval operation and is the output type for the jsonparse operation. json type fields cannot yet be used to define alerts, aggregation tasks or injections, or be added as grouping keys when you group your data. This will be possible in future versions of the application. | {"id":3451,"first_name":"John","last_name":"Doe","email":"john.doe@devo.com"} |
jq | This data type identifies a jq filter which is required as input for the Jq eval operation. This data type is created using the Jq filter compile operation. jq is a command-line JSON processor. Learn more here. | |
set | This data type identifies a collection of unique values arranges in no particular order. | {1:15,2:43,3:92,4:193}
|
map | A map data type represents an unordered set of key-value pairs. A map pair is a key and value mapped to one another. | {1:null,2:"2.00000005960464 ",3:"my.app",null:"username"} |
array | This data type identifies a collection of equal elements arranged as a series. This type is required as input for the sort, dropnulls, join, bag, index of and sum operations and is the output type for the array and mkarray operations. | ["usernameJohn","usernameMary","usernameJoe", "usernameJane"] |
tuple | A tuple is a collection of sorted elements of any type (repeated or not). The difference between an array and a tuple lies in the fact that in the array all the elements are internally converted to the same type, while in the tuple they are not (each tuple element retains its type). | •(1.33, 400) |
table | This data type identifies a list of fields in table format. This data type is used in subqueries and is only represented in query API. | - |