Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 21 Current »

 Watch video

About the autoparser

When data enters Devo through a non-standardized method, it cannot be automatically parsed at that moment of reception due to the lack of Devo tags to interpret its structure. However, it can still be parsed automatically thanks to the Autoparser. This functionality analyzes the data patterns in search of possible ways to parse your data, allowing you to choose the one you consider most adequate to resemble the originally intended structure.

How does it work?

The Autoparser uses internal logic to analyze the content of up to 200 sample events in an unparsed table in order to identify:

  • Non-alphanumeric ASCII characters that might be field delimiters. 

  • Units of data that follow a common and fixed syntax; i.e. IPv4 and IPv6 addresses, strings enclosed in quotation marks, floating-point decimal values, and more.

Based on this analysis, it recommends a selection of characters that are likely (and less likely) to be delimiters. You can then use the Autoparser controls to select the delimiters you need or analyze a different sample.

Delimiter analysis

The Autoparser will only offer as delimiters those characters that appear in all the 200 logs analyzed. For example, the comma will not be offered as a delimiter in the following set because the third element does not have any:
a, b, c, d // x, y, z // sampleLog

What data do I need?

Tables

The table must be one of the following:

  • my.app → tables created by sending data from a new, proprietary data source.

  • my.upload → tables created by manually uploading a file containing data.

Unless…

  • They have less than 4 tag levels, which makes them ineligible for the use of the autoparser.

  • They have been created by injecting data from another table, which are already properly parsed.

Events

Events must comply with the following conditions:

  • Events must contain the same number of fields in the same order.

  • Events must use delimiters in the same pattern in every event.

  • Events must use valid formats.

  • Timestamp data will be parsed as a string when presenting an invalid format. Check valid formats below:

    • ddd MMM DD HH:mm:ss YYYY → Thu Mar 29 00:21:05 2012

    • DD-MMM-YYYY HH:mm:ss.SSS → 27-Aug-2012 09:44:09.378

    • DD/MM/YYYY HH:mm:ss → 23/07/2019 07:55:00

    • DD/MM/YYYY H:mm:ss → 14/09/2012 9:42:05

    • YYYY-MM-DD HH:mm:ss.SSS → 2000-12-17 01:01:01.123

    • YYYY/MM/DD HH:mm:ss → 2012/08/25 06:48:18

    • YYYY/MM/DD → 2012/08/25

    • yyyy-MM-dd:hh:mm:ss+gmt → 2012-08-16:10:29:17+0200

    • yyyy-MM-dd hh:mm:ss.micros → 2000-12-17 01:01:01.123456

    • Epoch.millis (recognized as a float, you need to change it) → 1234567890.123

    • Millis (recognized as an integer, you need to change it) → 1584100816544

Disparate data

If the events do not conform to these requirements, it may be imposible to extract a pattern and an error will pop up to further specify the reason. You can then parse it manually using operations such as Split (split) and later create a custom table. For my.app data, you can also contact customer support to request a custom parser.

What permissions do I need?

To use this feature, you need two different permissions:

  • View level of the Finders permission. This permission is required to access a search, which is necessary to use the autoparser.

  • The Log autoparser permission. This permission is required to enable the option in the toolbar menu of the search window.

Using the autoparser

  • No labels