| Table of Contents | ||||||
|---|---|---|---|---|---|---|
|
...
| Anchor | ||||
|---|---|---|---|---|
|
When you access the search window after running a search, your browser is in charge of processing some of your query operations in addition to the server. In order to prevent browser exhaustion, switching to server mode is recommended when dealing with queries that require computationally heavy calculations. For the same reason, it is also recommended before grouping your data when dealing with tables that contain columns fields with a very large number of distinct values (that is, high cardinality and/or variability). In cases other than the aforementioned, it is recommended to continue running queries using the default search window mode.
...
Query-building can sometimes involve quite a bit of trial and error with the operations you apply to the data. So before starting to build and refine your query, apply a briefer time period and make sure that real-time event flow is off. This provides better performance because there will be fewer events to apply the operations to when you apply filters, create columnsfields, and so on. Once you are satisfied with your query, you can set the time range and real-time event flow as you require.
...
All filters should be applied early in the query, and certainly before grouping and aggregation. This reduces the memory and computation required for the later operations. The following describes the recommended order of operations:
Filters
Create columns fields (data enrichment)
Filters of new columnsfields
Groups
Aggregations
| Anchor | ||||
|---|---|---|---|---|
|
...
Consider applying a filter to the field before grouping to limit the cardinality.
If the field contains numeric values, enrich the data with a new column field that identifies a numeric range to which the event belongs, then group by the numeric range instead of the individual values.
| Anchor | ||||
|---|---|---|---|---|
|
...
fields in a table
If you use a Finder to open a data table, you can pre-select only the columns fields with data that is of interest to you. This reduces the amount of data that your browser needs to load into memory. Here's how.
If you have a query already open in the search window, you can use the Column Field manager tool to pick the columns fields you want to show or hide. Here's how.
...
Sometimes it can be visually difficult to examine an event's data, especially when the number of columns fields necessitates a horizontal scrollbar. This is when this tool comes in really handy.
...
This window lists each event's fields and values on its own page so that you can thoroughly examine the events one at a time. The Rich views toggle is activated by default and when activated, correctly reads and displays fields with values in JSON format. Use the Type column field to see the type of data of each Event. Image data is shown directly as an image. You can also copy an event's data to paste it elsewhere, or download the event in CSV, JSON, or TXT formats.
...
Once a query’s events have been grouped, there are some limitations you should keep in mind if you want to apply additional operations.
Applying filters or creating new
...
fields in grouped queries
You will only be able to use aggregations or grouping keys as arguments. Grouping keys are the columns fields you used to group the events. Other columns fields in the original table will not be available to use as arguments.
...
Bonus tips
Sparse verses dense
Suppose you have to search for a specific tree in both a sparse forest and a dense forest. The spare forest will have a small number of trees and it will be really easy to spot a special tree. However, the dense forest will have too many trees that could possibly match the one you're looking for; you would have to manually inspect every tree.
This concept directly relates to the frequency of values and the number of events in your searches.
| Info |
|---|
Tip When running a search in Devo, it's best to use sparse terms, that is, a word, a number or a value that is found relatively infrequently. |
Ordering of clauses
The order of clauses is important to achieve optimal performance in your queries. See the following example, where 99% of the logs in the table include the term "INFO". This query:
from application.log.log where toktains(raw, "INFO"), service="test"
...
| Info |
|---|
Tip When adding several |
Be careful when using the Not operation
The Not (not) operation does not use the token index, so it is recommended to push Not clauses down your query and apply additional filter operations before.
| Info |
|---|
Tip Use Not operators as last clauses if you have values you can index on. Otherwise, use them at the beginning of the query if you have to go through every line. |
Use logical operators in the proper order
When using the logical operators And (and), Or (or) and Not (not), it is important to place them properly in the query to get the required results. Always keep in mind the De Morgan's law:
not (A or B) → (not A) and (not B)
not (A and B) → (not A) or (not B)
...