Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

titleWatch video tutorial: Devo Query Lookups

...

Widget Connector
overlayyoutube
_templatecom/atlassian/confluence/extra/widgetconnector/templates/youtube.vm
width600px
urlhttps://www.youtube.com/watch?v=2M_wa-EtRrc
height300px

...

titleWatch video tutorial: Time Range Lookups

...

...

Table of Contents
minLevel1
maxLevel2
typeflat

...

To be able to create lookup tables from queries, you need to have management permission on Query lookups.Take into account that this the Manage version of the Query lookups permission. Having the User resources permission is equivalent to having the Manage version of the Query lookups permission.

This permission is part of a hierarchy with the Lookups permission in a higher level, meaning you need to have Lookups assigned in order to be able to have Query lookups.

...

Notice at least the View version of the Finders permission is required to enable the Lookup permission (know more about permissions here).

...

Note

Feature not enabled by default

Note that this feature is only enabled in certain domains by default. If you need to use it and is not enabled in your domain, contact the Devo support team.

...

These lookup tables are also created with query data, but they differ from static query lookups in that they are constantly populated with new sets of data. By default, they are updated every 5 minutes since the creation date, but this time might increase depending on the load of the system. If a new event arrives where the key field value has different row values, the old ones will be overwritten. Be aware that events remain unaffected if there are not any updates.

If your query groups data, the dynamic query lookup will be updated according to the grouping time indicated. For example, if your query groups data every 1 hour, the lookup will be populated with a new set of data every 1 hour. Note that if the grouping period is less than 5 minutes, the lookup will be updated every 5 minutes.

...

Before start creating your lookups, there are some limitations you must take into account.

  • Creation duration: if it exceeds 1 hour, a timeout error appears and the creation process is canceled. Whether it reaches the time limit or not is closely related to the other two restrictions described below.

  • Lookup size: in dynamic lookups, each iteration of new data must not exceed 8 GB downloaded from the server, otherwise, the lookup is aborted. Note that this limit does not apply to the total size of the dynamic lookup, which can exceed 8 GB.

  • Number of rows: in dynamic lookups, each iteration of new data must not exceed a certain number of rows downloaded from the server, which may vary depending on your environment. The number of rows downloaded is determined by the number of changes of a single key in each petition, which can be up to a maximum of 33,554,432 by default.

...

Rw ui steps macro
Rw step

Go to Data Searchsearch and access the required table.

Rw step

Choose the required period of time using the time range selector and click Apply Interval.

Rw step

In the data table toolbar, click the gear icon options menu and then select New → Query lookup.

Rw step

The New Query Lookuplookup window appears. Enter the required information.

Image RemovedImage Added

Name

Give a name to the lookup table.

Data fieldsFields

Drag to this field the data fields you want to add to the lookup table.

Info

Eventdate field not allowed with non-time-based groupings

If you used a non-time-based grouping when building your query, you cannot add the eventdate field to the lookup. If you try, you will get an error message explaining this.

ModeQuery mode

Choose either Static query or Dynamic query.

Key field

Select one of the fields added as key field. Learn more about this here.

Info

Repeated values in the key field

If the field selected as key upon the creation of a static query lookup contains repeated values, only the last value in the field will be kept, and the rest of the events with that value repeated will be discarded. In the case of a dynamic query lookup, repeated key values will be replaced by the most recent ones.

Time range lookup

Check this option to use a timestamp type field in your table to determine the value to be matched with the key field according to the date. Learn more about time range lookups in the section below.

Rw step

Click Save settingsCreate. You will receive a notification when the file is uploaded.

Now you can add fields from your lookup to a query in order to enrich it with additional information. See Add lookup values to your query to learn how to do it.

Query lookup tables are shown along with uploaded lookup tables, in Data Search search → Lookup Managementmanagement. The query lookup will be ready when the indicator in the Status field turns green. The Type field shows Upload, Static Query or Dynamic Query to indicate the lookup table type. Learn more in Manage and edit lookup tables.

...

Both static and dynamic query lookups can be created as a time range lookup. When using this option, the same entry of your key field will be matched with different results depending on the corresponding time range between the dates specified in a timestamp field of your query. This way, you can match the same value in your query with different values in your lookup, which may come in handy in different situations.

To create a time range lookup, you must check the Time range lookup checkbox in the creation process of the query lookup, as explained above. Once you do this, the Time range field dropdown menu will appear, and you must choose the timestamp-type field you want to use among the ones added into the field on the left.

...

Rw ui textbox macro
typeinfo

Choosing your timestamp field

If the type of the field is not timestamp-typeit will not appear in the Time range field dropdown. Remember that you can check the type of field by clicking the icon that appears when hovering over a field header. You can use the Timestamp (timestamp) operation to transform strings representing dates or epoch times in milliseconds (integers) into timestamp data type.

Image RemovedImage Added

Time range lookup example

For example, imagine you have a query that shows the cities to which a user (Mike) has been calling in different time ranges.

As you can see in the picture below, Mike talked to colleagues in different cities at different times. Let’s say we want to consider the time from one call to the next as a time slot assigned to that specific city. For that purpose, we will define a time range lookup as described above.

...

After defining a time range lookup based on this data, we want to define a new field in another query to match the user with the city that corresponds to them according to the time slot mentioned before and recorded in the lookup.

As you can see in the picture below, the new field shows the corresponding city according to the time in the eventdate field. For example, you can see the value New York in events sent at 16:00 because the time range lookup matches New York to any event received from 13:00 to 17:00.

...