Selecting unrevealed columns

Along with the columns available in a table, each table has a unique set of unrevealed columns. They may contain a variety of information related to data ingestion and structural metadata. A subset of them is now available, however, they are not shown by default and need to be explicitly requested in the query using the procedures explained in Selecting specific columns in LINQ or Selecting specific columns with the Finder. You can find them marked as Extra so they can be easily identified.

Although the unrevealed columns are not the same for all tables, the possibilities are limited to the following:

• raw: this column contains events as they are stored in Devo. This is perhaps the most significant column because it provides you with the data as it was originally generated.

• rawSource or rawMessage: these columns contain the exact same information, the payload of the events (events without headers or metadata). However, the former will be progressively deprecated in favor of the latter.

• rawHostName, rawHostIp, rawHostchain, hostchain: these columns make reference to different aspects of data origin (hostName and hostIp) and the path traveled from origin to Devo (hostchain).

• rawTag, tag: these columns make reference to the full tag of the table.

• rawTechnology, rawBrand, rawPhylum, rawFamily, rawGenus, rawSpecies: these columns make reference to each of the individual tag levels of the table. They apply only to the my.app tables.

Procedure

Regardless of the method used to show them (Finders or Free Text Query), you will recognize these columns because they display an Extra sign:

• In the case of the Finders, all you have to do after clicking the Select Columns button is select or deselect them at will before running the query.

  • In the case of the Free Text Query, the Show Table Fields button will show the names of the available columns so you can define the correct syntax to include them.
    
    An example of a valid syntax for the Free Text Query to show the columns shown in the picture above would be the following:
    from demo.ecommerce.data
select hostchain
select tag
select raw
select rawSource

    Open

    

Limitations

Be aware that these special columns have some limitations. They cannot be used in custom tables, union tables and injections unless they are renamed. This is because they are intrinsic to the data creation process itself, so even if they are not explicitly shown, they exist with that name.

• If you try to include these columns in a custom table or a union table without renaming them, you will receive an error message explaining the reason and the table will not be created. To know more about creating custom and union tables, check the following articles: Create a custom table, Create a union table.

  Open

  

• If you try to include these columns in an injection without renaming them, the injection form will display a warning informing that these columns will be ignored if they are not renamed. The injection will be therefore created without these columns if you proceed anyway. To know more about injecting data, check the following article: Inject data to a new table