Selecting unrevealed fields

[ 1 Unrevealed fields explained ] [ 2 How can I reveal these fields? ] [ 3 Are there any limitations? ]

Unrevealed fields explained

Along with the fields available in a table, each table has a unique set of unrevealed fields. They may contain a variety of information related to data ingestion and structural metadata. A subset of them is now available, however, they are not shown by default and need to be explicitly requested in the query using the procedures explained in Selecting specific fields in LINQ or Selecting specific fields with the Finder. You can find them marked as Extra so they can be easily identified.

Although the unrevealed fields are not the same for all tables, the possibilities are limited to the following:

raw: this field contains events as they are stored in Devo. This is perhaps the most significant field because it provides you with the data as it was originally generated.
rawSource or rawMessage: these fields contain the exact same information, the payload of the events (events without headers or metadata). However, the former will be progressively deprecated in favor of the latter.
rawHostName, rawHostIp, rawHostchain, hostchain: these fields make reference to different aspects of data origin (hostName and hostIp) and the path traveled from origin to Devo (hostchain).
rawTag, tag: these fields make reference to the full tag of the table.
rawTechnology, rawBrand, rawPhylum, rawFamily, rawGenus, rawSpecies: these fields make reference to each of the individual tag levels of the table. They apply only to the my.app tables.

Unique set of unrevealed fields

Be aware that each table presents a unique set of fields marked as Extra, and only those are eligible to be revealed. Trying to show one that is not part of that set will return an error. For example, if the list of Extra fields includes raw and rawSource, trying to show rawMessage will not work.

How can I reveal these fields?

Regardless of the method used to show them (Finders or Free text query), you will recognize these fields because they display an Extra sign:

Using finders

In the case of the Finders, all you have to do after clicking the Select fields button is select or deselect them at will before running the query.

Using free text query

In the case of the Free text query, the Show table fields button will show the names of the available fields so you can define the correct syntax to include them.

from demo.ecommerce.data
select hostchain
select tag
select raw
select rawSource

Are there any limitations?

Be aware that these special fields have some limitations. They cannot be used in custom tables, union tables and injections unless they are renamed. This is because they are intrinsic to the data creation process itself, so even if they are not explicitly shown, they exist with that name.

Custom and union tables

If you try to include these fields in a custom table or a union table without renaming them, you will receive an error message explaining the reason and the table will not be created. To know more about creating custom and union tables, check the following articles: Create a custom table, Create a union table.

Injections

If you try to include these fields in an injection without renaming them, the injection form will display a warning informing you that these fields will be ignored if they are not renamed. The injection will be therefore created without these fields if you proceed anyway. To know more about injecting data, check the following article: Inject data to a new table.