Introduction
Limited availability
This feature has limited availability. If you have any doubts or questions do not hesitate to reach out to us.
Data in Devo is indexed by ingestion time (eventdate) by default, which means that filtering and grouping by the event date are optimized. With this new feature, we have added a new time index over the creation time of the events (creationdate field), so queries over this field are also optimized.
The event date is when Devo receives and ingests data. On the other hand, the creation date is considered as when data is generated at its source. For this reason, the event date is always after the creation date, although in many cases there is a negligible difference between both. Sometimes there is a large difference between both dates and a user is interested in the creation date being the time reference for their data. For example, imagine a machine that generates events, which is then switched off during a whole day and the events arrive into Devo the next day: in this case the event date and creation date would differ.
What permissions do I need?
To be able to see the time reference options and use the creation date, you need the Time control permission.
Time reference options
The time control button is located to the left of the date picker in the search window (see picture below) and serves as a visual indicator of whether time control is activated, as well as which time reference you are using for your table.
Once activated, use this button to select the time reference for your current search:
Creation date mode (CrD): This sets the creation date as the time reference for the current search.
Ingestion date mode (InD): This sets the event date as the time reference for the current search. Note that this is the default in Devo.
Switching not recommended
We recommend you refrain from switching between both time reference options when running queries as the results returned might not be correct. If you need to use the other option, we recommend running a new search instead.
As mentioned above, the creation date refers to when data is generated at the source, whereas the event date is when Devo receives and ingests the data. Note that the selected time reference affects the following features:
Data search date picker.
Data search (regardless of whether the server mode is activated).
Reception delay
The Reception delay option only becomes available when you select the creation date as the time reference. It allows you to set the maximum expected delay between the creation date and the ingestion date for your chosen table. It is important to note that the delay does not function as a filter, but rather compiles all of the data specified in the time range plus the selected delay period. Setting a reception delay allows the system to increase performance as it only looks for the corresponding events.
The available delay options are:
1 hour
2 hours
4 hours
12 hours
1 day
2 days
4 days
7 days
Until now (maximum)
Default value
If there is no known maximum delay period then the Until now (maximum) is set by default.
Switching not recommended
We recommend you refrain from changing the reception delay when running queries. The idea behind the reception delay is that it is adapted and set to each of your data tables, taking into consideration the anticipated delay between the creation and ingestion of data for each table.
Example
Imagine you select the creation date in the time control and set the time range to display all events with a creation date between the following period:
2021-12-15 00.00
2021-12-15 12.00
However, you know from experience that there is a delay in some events being ingested into Devo, so you select a delay of 1 hour. This means your query with the delay selected is like this:
where "2021-12-15 00:00" <= creationdate < "2021-12-15 12:00" where "2021-12-15 00:00" <= eventdate < "2021-12-15 13:00"
Events compiled in this search:
Those with creationdate between 00:00 and 12:00
Those with eventdate between 00:00 and 13:00
Events not compiled in this search:
Those with creationdate and eventdate prior to 00:00
Those with creationdate after 12:00
Those with eventdate after 13:00
LINQ syntax
When in creation date mode it is important to be aware of a quirk in the LINQ syntax when working with groupings. This is because in Devo's system the every is still linked to the event date, which is the default time reference.
Example with ingestion date mode
from test.keep_cd.free group every 5m by creationdate, facility every 5m
Example with creation date mode
from test.keep_cd.free select period(creationdate, 5m) as creationdate_1 group every 5m by creationdate_1, facility every -
For this reason we strongly recommend that you do not switch between the different time reference dates once you have started with a grouping. If you must change the time reference date, we strongly suggest that you go back to the first breadcrumb and create the grouping from scratch.
Default time reference
There are two different ways that you can set the default time reference for your domain for tables that contain both creation and event dates.
Domain preferences
You can set the reference time in your domain preferences.
Search window
You can set the time reference used for your current table as the default for either the current user or for all domain users, provided the table has time control activated and the user has time control permissions. Any changes made here override whatever previous settings have been made in the domain preferences area.
Click on the additional tools icon in the search window toolbar.
Navigate to Source table and click Set reference time.
There are four options available to you:
Save view timestamp configuration | Saves the current time reference for this table. |
|---|---|
Reset view timestamp configuration | Resets the time reference for this table. |
Save domain timestamp configuration | Saves the current time reference for the domain. |
Reset domain timestamp configuration | Resets the time reference for the domain. |
