Versions Compared

Version Old Version 8 New Version Current
Changes made by

Jonas Gonzalez

Jonas Gonzalez

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
maxLevel2
minLevel2
typeflat

...

Endpoints and methods

Anchor
path1
path1
GET /lookup/{domain}

Display information on the lookups existing on a given domain.

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the lookups you want to list.

Query string parameters

Query string parameters are optionally added after the path parameters, preceded by a question mark (?) and separated by an ampersand (&)

Parameter

Type

Description

max_length

integer

Maximum number of values to return. The default value is 100. The maximum value is 1000 and the minimum value is 1.

Lookups are shown from oldest to newest.

pageToken

integer

Use this parameter to consider a previously requested list on your next request. This should be set to the value shown in the nextPageToken parameter included in the response you want to consider, if successful.

For example, you may perform a request and ask for information about the oldest 5 lookups in your domain. Then, you can perform a second request including the value in the nextPageToken parameter returned with the previous response and requesting only 2 lookups. This will return the 2 lookups created after the first 5 previously requested.

The default and minimum value is 0.

owner

string

You can filter the lookups in the response based on their domain owner, that is, the domain where the lookup was created. Supported values are:

  • THIS_DOMAIN - Returns only lookups created in the domain indicated in the request URL.

  • OTHER_DOMAINS - Returns only lookups owned by other domains but visible by the domain indicated in the request URL. This might be the case of domains that belong to a multitenant structure and have access to lookups defined in other domains that belong to that structure. You can decide the visibility level of a new lookup upon its creation.

  • ANY_DOMAIN - Returns both lookups owned by the domain indicated in the request URL and those visible by them in other domains.

The default value is THIS_DOMAIN

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can decide to make a lookup visible in all the domains within the structure when you create it using POST/lookup/{domain}/{name}/deploy-config. See how to do it here.

Example

Find below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article.

Code Block
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?max_length=10

And this request will return information about all the lookups owned by the user domain and also the ones visible to them in other domains:

Code Block
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?owner=ANY_DOMAIN
Rw expand
titleResponses

Code

Description

200

Successful response. The response includes the list of lookups in the given domain with their details.

Code Block
{
  "type": "LookupMetaInfoListResponse",
  "cid": "72078e04ee84",
  "code": 200,
  "context": null,
  "id": "xxxxxxxxxx-0f74-11ee-b13b-4fc634871e5f",
  "msg": "tutorial lookups.",
  "lookups": [
    {
      "name": "Lookup_test_t",
      "domain": "tutorial",
      "fileSize": 12288,
      "numEntries": 136,
      "creationDate": "2023-03-15T13:45:57.63",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "domain",
          "type": "str",
          "key": true
        },
        {
          "column": "CDNProvider",
          "type": "str",
          "key": false
        }
      ],
      "owner": "user@devo.com"
    },
    {
      "name": "CDN_Providers",
      "domain": "tutorial",
      "fileSize": 12288,
      "numEntries": 136,
      "creationDate": "2023-03-15T13:45:57.806",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "domain",
          "type": "str",
          "key": true
        },
        {
          "column": "CDNProvider",
          "type": "str",
          "key": false
        }
      ],
      "shared": false,
      "owner": "user@devo.com"
    },
    {
      "name": "test_101",
      "domain": "tutorial",
      "fileSize": 40960,
      "numEntries": 307,
      "creationDate": "2023-03-15T13:45:58.338",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "alertName",
          "type": "str",
          "key": true
        },
        {
          "column": "alertType",
          "type": "str",
          "key": false
        },
        {
          "column": "alertMitreTactics",
          "type": "str",
          "key": false
        },
        {
          "column": "alertMitreTechniques",
          "type": "str",
          "key": false
        },
        {
          "column": "alertPriority",
          "type": "int4",
          "key": false
        }
      ],
      "owner": "user@devo.com"
    },
    {
      "name": "d14022023api",
      "domain": "tutorial",
      "fileSize": 8192,
      "numEntries": 1,
      "creationDate": "2023-03-15T13:46:58.05",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "key",
          "type": "int4",
          "key": true
        },
        {
          "column": "fbool",
          "type": "bool",
          "key": false
        }
      ],
      "owner": null
    }
  ],
  "nextPageToken": -1
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupMetaInfoResponseError",
  "cid": "046ead1c8323",
  "code": 400,
  "context": null,
  "id": "xxxxxx-4fbb-11ed-859a-f19049cfd717"
}

401

Unsuccessful response. The user is unauthorized to list the domain's lookups.

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupMetaInfoResponseError",
  "cid": "e33f677a6981",
  "code": 404,
  "context": null,
  "id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
  "msg": "Not Found"
}

Anchor
path2
path2
GET /lookup/{domain}/{name}

Return information of a specific lookup.

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the lookup you want to retrieve.

name required

string

Enter the name of the lookup you want to get information about.

Example

Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.

Code Block
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup
Rw expand
titleResponses

Code

Description

200

Successful response. The response includes information about the requested lookup.

Code Block
{
  "type": "LookupMetaInfoResponse",
  "cid": "3c9bb672512c",
  "code": 200,
  "context": null,
  "id": "xxxxxxxxx-0f75-11ee-b13b-636d49ade562",
  "msg": "tutorial/test_101 meta information.",
  "lookupMetaInfo": {
    "name": "test_101",
    "domain": "tutorial",
    "fileSize": 40960,
    "numEntries": 307,
    "creationDate": "2023-03-15T13:45:58.14",
    "keyType": {
      "type": "first",
      "columns": null,
      "hasher": null
    },
    "deployConfig": null,
    "lastStatus": null,
    "fields": [
      {
        "column": "alertName",
        "type": "str",
        "key": true
      },
      {
        "column": "alertType",
        "type": "str",
        "key": false
      },
      {
        "column": "alertMitreTactics",
        "type": "str",
        "key": false
      },
      {
        "column": "alertMitreTechniques",
        "type": "str",
        "key": false
      },
      {
        "column": "alertPriority",
        "type": "int4",
        "key": false
      }
    ],
    "owner": "user@devo.com"
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupMetaInfoResponseError",
  "cid": "4b6cdd188a82",
  "code": 400,
  "context": null,
  "id": "xxxxxx-5052-11ed-859a-910aa97972a3",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to get information about the lookup.

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

404

Unsuccessful response. Lookup not found.

Code Block
{
  "type": "LookupMetaInfoResponseError",
  "cid": "e33f677a6981",
  "code": 404,
  "context": null,
  "id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
  "msg": "Not Found"
}

Anchor
path3
path3
GET /lookup/{domain}/{name}/job

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the required lookup.

name required

string

Enter the name of the required lookup.

Example

Find below a request example in cURL language. This request will return information about the last jobs generated by the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.

Code Block
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job
Rw expand
titleResponses

Code

Description

200

Successful response. Returns the job UUIDs of the given lookup.

Code Block
{
  "cid": "e47f4ab72ded",
  "code": 200,
  "context": null,
  "id": "xxxxxxxx-e37c-11ed-b5ea-0242ac120002",
  "msg": "Lookup job uuids",
  "jobs": [
    "xxxxxxx-c9a2-489c-8794-ea656a19b822",
    "xxxxxxx-9714-48a7-9976-73e41523edfd",
    "xxxxxxx-48a8-46ea-ab22-e0a5458e302b",
    "xxxxxxx-ad7e-4fe6-bb43-89f93e629d76"
  ]
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupMetaInfoResponseError",
  "cid": "6cde7fd6a1f4",
  "code": 400,
  "context": null,
  "id": "xxxxxxx-53a2-11ed-859a-33f9a0bb845a",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to access the lookup

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

Anchor
path4
path4
GET /lookup/{domain}/{name}/job/{id}

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the required lookup.

name required

string

Enter the name of the required lookup.

id required

string

Enter the UUID of the required lookup job. You can get a job UUID using the request explained above, under the jobs parameter in the response.

Example

Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.

Code Block
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job/123456
Rw expand
titleResponses

Code

Description

200

Successful response. Returns the job statuses.

Code Block
{
  "cid": "e47f4ab72ded",
  "code": 200,
  "context": null,
  "id": null,
  "msg": "Lookup job's statuses",
  "status": [
    {
      "eventdata": "2021-09-29T10:18:10.805",
      "domain": "galactic_empire",
      "lookup": "ImperialIntranetActivity",
      "msg": "Lookup successfully created"
      "code": "create.ok"
    },
    {
      "eventdata": "2021-09-29T10:18:12.472",
      "domain": "ImperialIntranetActivity",
      "lookup": "test-schedule",
      "msg": "Lookup ready to be executed"
      "code": "deploy.ok"
    }
  ]
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupJobsError",
  "cid": "4d1eb85a908d",
  "code": 400,
  "context": null,
  "id": "xxxxxx-53a4-11ed-859a-2d69d242a54f",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to access the lookup

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

Anchor
path5
path5
POST/lookup/{domain}/{name}/deploy-config

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the new lookup.

name required

string

Enter the name of the new lookup.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Anchor
id
id

id - Lookup ID object.

Parameter

Type

Description

creator

string

Domain of the lookup.

name

string

Name of the lookup.

Example value:

Code Block
"id": {
  "creator": "self",
  "name": "ForceSensitiveBeings"
}

Anchor
visibility
visibility

visibility - (string) Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

Anchor
recipe
recipe

recipe required - Recipe of the lookup to be created.

Parameter

Type

Description

recipeType required

string

Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article.

source required

object

Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field).

Note

Check more about uploading a lookup using a CSV through S3, see this article.

  • columns - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketName required (string) - Name of the S3 bucket where the CSV is located.

    • keyName required (string) - Path of the CSV in the bucket.

    • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

    • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • host - (string) - The S3 bucket host where the CSV is located.

    • port - (integer) - The S3 bucket port where the CSV is located.

    • region - (string) - The AWS region where the S3 bucket has been created.

  • query - (string) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

    • boolean

    • string

    • int4

    • int8

    • float4

    • float8

    • hex4

    • hex8

    • ip4

    • ip6

    • timestamp

    • duration

lookupType

object

Indicate the lookup type.

  • type required - (string) Valid values are normal (regular lookups) and historic (time range lookups).

  • instantPolicy - (string) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic. Possible values are:

    • natural - The associated source must be historical and the instant of each row will be used.

    • const - The instance of all rows will be the supplied constant value.

    • column - The instance will be extracted from the column with the given name. That column type must be timestamp.

  • instant - (number) Only for historic lookups of type const.

  • columnName - (string) Only for historic lookups of type column.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

columnFilter

array

If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete.

secondaryIndexes

object

Columns of a lookup indexed as secondary indexes.

  • type required - (string) Type of the secondary indexes to be applied to the lookup. Values can be all, none or by-name. The default value is none.

  • map - Map of column names to be applied as secondary indexes when the type selected is by-name.

refreshMillis

number

Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic.

startMillis

number

Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic.

requiresDate

boolean

If true, the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now(). Can only be used if the recipe type chosen is periodic. The default value is false.

Anchor
status
status

notifyStatus - (boolean) If true, a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false by default.

Check below an example of request body:

Code Block
{
  "id": {
    "creator": "rebel_alliance",
    "name": "TotallyNotFakeData"
  },
  "recipe": {
    "recipeType": "once",
    "source": {
      "query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
    },
    "lookupType": {
      "type": "normal"
    },
    "append": false,
    "key": {
      "type": "column",
      "column": "key"
    },
    "columnFilter": [
      "key",
      "IsDataFake",
      "RebelsImprisoned",
      "CreditsOnImperialBanks",
      "Hex4Emperor",
      "Hex8Vader",
      "EmperorClones",
      "Pi",
      "EmperorIP4",
      "EmperorIP6",
      "TimeAfterBattleOfYavin"
    ],
    "contribution": {
      "type": "add"
    },
    "requiresDate": false
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the creation request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d41c91a21d56",
  "code": 201,
  "context": null,
  "id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
  "msg": "Lookup sent to creation",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": "creator-only",
    "recipe": {
      "type": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      }
    }
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": "BAD_REQUEST",
  "id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
  "message": "Bad Request"
}

401

Unsuccessful response. The user is unauthorized to create a lookup.

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

403

Unsuccessful response. User credentials are correct but does not have permission to create this lookup.

Code Block
{
  "type": "LookupCreationError",
  "cid": "9a3eda1848d1",
  "code": 403,
  "context": "FORBIDDEN",
  "id": "xxxxxx5055-11ed-859a-7b524e50491a",
  "msg": "User is not authorized to perform operations in the domain."
}

Anchor
path6
path6
POST/lookup/{domain}/{name}/deploy-csv

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

columns required

array

Array of lookup column descriptors. Each column object may contain the following parameters:

  • name required - (string) Name of the column.

  • from - (integer) 0-based index of the column in the CSV.

  • type - (string) Data type of the column. Possible values are:

    • BOOLEAN

    • STRING

    • INT4

    • INT8

    • FLOAT4

    • FLOAT8

    • HEX4

    • HEX8

    • IP4

    • IP6

    • TIMESTAMP

    • DURATION

fileProvider

array

Info to get the CSV file from S3.

  • bucketName required (string) - Name of the S3 bucket where the CSV is located.

  • keyName required (string) - Path of the CSV in the bucket.

  • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

  • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • host - (string) - The S3 bucket host where the CSV is located.

  • port - (integer) - The S3 bucket port where the CSV is located.

  • region - (string) - The AWS region where the S3 bucket has been created.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

skipPreface

string

Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

hasHeader

boolean

Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

skipEmptyLines

boolean

Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

Code Block
{
  "columns": [
    {
      "name": "ID",
      "type": "INT8"
    },
    {
      "name": "Location",
      "type": "STRING"
    },
    {
      "name": "Age",
      "type": "FLOAT8"
    },
    {
      "name": "Species",
      "type": "STRING"
    },
    {
      "name": "MidiclorianLevel",
      "type": "FLOAT4"
    }
  ],
  "key": {
    "type": "column",
    "column": "ID"
  },
  "fileProvider": {
    "bucketName": "holocrons-bucket",
    "keyName": "secrets/data/force-sensitive-beings.csv",
    "transferOwnership": true
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path7
path7
POST/lookup/{domain}/{name}/deploy-csv

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request multipart/form-data body must include the CSV file and an associated JSON configuration object. Click them in the following list to see its details:

Parameter

Type

Description

deploy-config

object

Deploy CSV configuration of the lookup to be created. This object states the visibility of the created lookup, the fileProvider indicating the CSV file stored in AWS S3, the column descriptors and key. It also includes metainfo from the CSV.

  • visibility - (string) Visibility level of the lookup. Allowed values are:

    • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

    • all-subdomains - (string) The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

  • columns required - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketNamerequired - (string) The name of the S3 bucket where the CSV is located.

    • keyNamerequired- (string) The path of the CSV inside the bucket.

    • transferOwnershiprequired - (boolean) This parameter represents whether Lomana should act as the owner of the CSV and delete it from the bucket once it has finished creating the lookup.

    • accessKey - (string) The access key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • secretKey - (string) The secret key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • host - (string) The S3 bucket host where to find the CSV.

    • port - (integer) The S3 bucket port where to find the CSV.

    • region - (string) The AWS region whether the S3 bucket has been created.

  • key - (string) In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

    • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

    • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

    • type required - (string). Type of key algorithm. Values can be:

      • first-column - The first column of the lookup will be used as key. This is the default value.

      • column - The column indicated in the column parameter will be used as key.

      • row-hash - A hash of the elements on the first row of the lookup will be used as key.

      • col-hash - A hash of the columns stated in the columns parameter will be used as key.

      • seq - The key of the lookup will be generated sequentially.

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • contribution - (array) Defines how a row contributes to the final result, normally used on incremental lookups.

    • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

    • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

  • keepHistory - (boolean) If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

  • columnTypeReference- (string) If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

  • append- (boolean) If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

csv

file

CSV file to upload. Only needed if the file was not uploaded to S3.

Note

Check more about uploading a lookup using a CSV through S3, see this article.

Check below an example of request body:

Code Block
{
  "deploy-config": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/deploy-csv.json"
  },
  "csv": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/simple_lookup.csv"
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path8
path8
POST/lookup/{domain}/{name}/deploy-static-query

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

Code Block
{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-1d < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  },
  "keepHistory": false,
  "columnTimeReference": null
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      }
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-static-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path9
path9
POST/lookup/{domain}/{name}/deploy-periodic-query

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queryable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

refreshPeriod

string

Refresh period of the lookup. Defaults to grouping period of the query in case it is a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo's duration type.

The minimum value accepted is 1 minute.

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

Check below an example of request body:

Code Block
{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-5m < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  },
  "refreshPeriod": "5m",
  "startDate": "2023-12-12T00:00:00.00Z",
  "append": true,
  "keepHistory": false,
  "columnTimeReference": null
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 300000,
      "startMillis": 1702339200000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-periodic-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path10
path10
POST/lookup/{domain}/{name}/deploy-sliding-window-query

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

keyrequired

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

windowSizerequired

string

Size of the sliding window. Every time the query is executed, the lookup will contain the size of the window. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 hour (Example: Xh, Xd).

refreshPeriod required

string

Lookup refresh period. Defaults to query’s grouping period in case of a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 minute (Example: Xm, Xh, Xd).

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

keepHistory

boolean

If set to true, the Lookup Manager will store all historic data in the lookup, enabling historic search.

columnTimeReference

string

If given, the Lookup Manager will use the column specified as time instant reference for the data provided.

Check below an example of request body:

Code Block
{
    "visibility": {
      "type": "creator-only"
    },
    "query": "from box.unix group by machine, srcIp",
    "key": {
        "type": "column",
        "column": "machine"
    },
    windowSize": "30d",
    "refreshPeriod" : "5m"
}
Rw expand
titleResponses

Code

Description

200

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 200,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select userid, domain from siem.logtrust.web.activity"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "userid"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 600000,
      "startMillis": 1702339200000,
      "slideWindow": 86400000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-sliding-window-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path11
path11
PUTlookup/{domain}/{name}/deploy-config

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where that contains the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Anchor
id
id

id  - Lookup ID object.

Parameter

Type

Description

creator

string

Domain of the lookup.

name

string

Name of the lookup.

Example value:

Code Block
"id": {
  "creator": "self",
  "name": "ForceSensitiveBeings"
}

Anchor
visibility
visibility

visibility - (string) Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain.. Only multitenant Admin users will be able to use this value.

Anchor
recipe
recipe

recipe required - Recipe of the lookup to be updated.

Parameter

Type

Description

recipeType required

string

Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article.

source required

object

Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field).

  • columns - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketName required (string) - Name of the S3 bucket where the CSV is located.

    • keyName required (string) - Path of the CSV in the bucket.

    • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

    • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • host - (string) - The S3 bucket host where the CSV is located.

    • port - (integer) - The S3 bucket port where the CSV is located.

    • region - (string) - The AWS region where the S3 bucket has been created.

  • query - (string) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

    • boolean

    • string

    • int4

    • int8

    • float4

    • float8

    • hex4

    • hex8

    • ip4

    • ip6

    • timestamp

    • duration

lookupType

object

Indicate the lookup type.

  • type required - (string) Valid values are normal (regular lookups) and historic (time range lookups).

  • instantPolicy - (string) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic. Possible values are:

    • natural - The associated source must be historical and the instant of each row will be used.

    • const - The instance of all rows will be the supplied constant value.

    • column - The instance will be extracted from the column with the given name. That column type must be timestamp.

  • instant - (number) Only for historic lookups of type const.

  • columnName - (string) Only for historic lookups of type column.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

columnFilter

array

If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete.

secondaryIndexes

object

Columns of a lookup indexed as secondary indexes.

  • type required - (string) Type of the secondary indexes to be applied to the lookup. Values can be all, none or by-name. The default value is none.

  • map - Map of column names to be applied as secondary indexes when the type selected is by-name.

refreshMillis

number

Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic.

startMillis

number

Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic.

requiresDate

boolean

If true, the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now(). Can only be used if the recipe type chosen is periodic. The default value is false.

Anchor
status
status

notifyStatus - (boolean) If true, a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false by default.

Check below an example of request body:

Code Block
{
  "id": {
    "creator": "rebel_alliance",
    "name": "TotallyNotFakeData"
  },
  "recipe": {
    "recipeType": "once",
    "source": {
      "query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
    },
    "lookupType": {
      "type": "normal"
    },
    "append": false,
    "key": {
      "type": "column",
      "column": "key"
    },
    "columnFilter": [
      "key",
      "IsDataFake",
      "RebelsImprisoned",
      "CreditsOnImperialBanks",
      "Hex4Emperor",
      "Hex8Vader",
      "EmperorClones",
      "Pi",
      "EmperorIP4",
      "EmperorIP6",
      "TimeAfterBattleOfYavin"
    ],
    "contribution": {
      "type": "add"
    },
    "requiresDate": false
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d41c91a21d56",
  "code": 201,
  "context": null,
  "id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
  "msg": "Lookup sent to creation",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": "creator-only",
    "recipe": {
      "type": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      }
    }
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": "BAD_REQUEST",
  "id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
  "message": "Bad Request"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

403

Unsuccessful response. User credentials are correct but does not have permission to update this lookup.

Code Block
{
  "type": "LookupCreationError",
  "cid": "9a3eda1848d1",
  "code": 403,
  "context": "FORBIDDEN",
  "id": "xxxxxx5055-11ed-859a-7b524e50491a",
  "msg": "User is not authorized to perform operations in the domain."
}

Anchor
path12
path12
PUTlookup/{domain}/{name}/deploy-csv

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the domain name of the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

columns required

array

Array of lookup column descriptors. Each column object may contain the following parameters:

  • name required - (string) Name of the column.

  • from - (integer) 0-based index of the column in the CSV.

  • type - (string) Data type of the column. Possible values are:

    • BOOLEAN

    • STRING

    • INT4

    • INT8

    • FLOAT4

    • FLOAT8

    • HEX4

    • HEX8

    • IP4

    • IP6

    • TIMESTAMP

    • DURATION

fileProvider

array

Info to get the CSV file from S3.

  • bucketName required (string) - Name of the S3 bucket where the CSV is located.

  • keyName required (string) - Path of the CSV in the bucket.

  • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

  • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • host - (string) - The S3 bucket host where the CSV is located.

  • port - (integer) - The S3 bucket port where the CSV is located.

  • region - (string) - The AWS region where the S3 bucket has been created.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

skipPreface

string

Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

hasHeader

boolean

Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

skipEmptyLines

boolean

Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

Code Block
{
  "columns": [
    {
      "name": "ID",
      "type": "INT8"
    },
    {
      "name": "Location",
      "type": "STRING"
    },
    {
      "name": "Age",
      "type": "FLOAT8"
    },
    {
      "name": "Species",
      "type": "STRING"
    },
    {
      "name": "MidiclorianLevel",
      "type": "FLOAT4"
    }
  ],
  "key": {
    "type": "column",
    "column": "ID"
  },
  "fileProvider": {
    "bucketName": "holocrons-bucket",
    "keyName": "secrets/data/force-sensitive-beings.csv",
    "transferOwnership": true
  },
  "contribution": {
    "type": "add"
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupUpdateResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

Anchor
path13
path13
PUTlookup/{domain}/{name}/deploy-csv

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request multipart/form-data body must include the CSV file and an associated JSON configuration object. Click them in the following list to see its details:

Parameter

Type

Description

deploy-config

object

Deploy CSV configuration of the lookup to be created. This object states the visibility of the created lookup, the fileProvider indicating the CSV file stored in AWS S3, the column descriptors and key. It also includes metainfo from the CSV.

  • visibility - (string) Visibility level of the lookup. Allowed values are:

    • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

    • all-subdomains - (string) The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

  • columns required - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketNamerequired - (string) The name of the S3 bucket where the CSV is located.

    • keyNamerequired- (string) The path of the CSV inside the bucket.

    • transferOwnershiprequired - (boolean) This parameter represents whether Lomana should act as the owner of the CSV and delete it from the bucket once it has finished creating the lookup.

    • accessKey - (string) The access key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • secretKey - (string) The secret key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • host - (string) The S3 bucket host where to find the CSV.

    • port - (integer) The S3 bucket port where to find the CSV.

    • region - (string) The AWS region whether the S3 bucket has been created.

  • key - (string) In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

    • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

    • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

    • type required - (string). Type of key algorithm. Values can be:

      • first-column - The first column of the lookup will be used as key. This is the default value.

      • column - The column indicated in the column parameter will be used as key.

      • row-hash - A hash of the elements on the first row of the lookup will be used as key.

      • col-hash - A hash of the columns stated in the columns parameter will be used as key.

      • seq - The key of the lookup will be generated sequentially.

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • contribution - (array) Defines how a row contributes to the final result, normally used on incremental lookups.

    • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

    • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

  • keepHistory - (boolean) If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

  • columnTypeReference- (string) If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

  • append- (boolean) If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

csv

file

CSV file to upload. Only needed if the file was not uploaded to S3.

Note

Check more about uploading a lookup using a CSV through S3, see this article.

Check below an example of request body:

Code Block
{
  "deploy-config": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/deploy-csv.json"
  },
  "csv": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/simple_lookup.csv"
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path14
path14
PUTlookup/{domain}/{name}/deploy-static-query

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the domain name of the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

Code Block
{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-1d < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  }
}
Rw expand
titleResponses

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      }
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-static-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

Anchor
path15
path15
PUTlookup/{domain}/{name}/deploy-periodic-query

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the domain name of the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queryable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

refreshPeriod

string

Refresh period of the lookup. Defaults to grouping period of the query in case it is a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo's duration type.

The minimum value accepted is 1 minute.

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

Check below an example of request body:

Code Block
{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-5m < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  },
  "refreshPeriod": "5m",
  "startDate": "now",
  "append": true
}
Rw expand
titleResponses

Code

Description

200

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 300000,
      "startMillis": 1702339200000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-periodic-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

Anchor
path16
path16
PUTlookup/{domain}/{name}/deploy-sliding-window-query

...

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

  • all-domains - To create a lookup visible by all domains. It is only allowed when the domain of the user is self.

Info

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

keyrequired

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

windowSizerequired

string

Size of the sliding window. Every time the query is executed, the lookup will contain the size of the window. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 hour (Example: Xh, Xd).

refreshPeriod required

string

Lookup refresh period. Defaults to query’s grouping period in case of a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 minute (Example: Xm, Xh, Xd).

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

keepHistory

boolean

If set to true, the Lookup Manager will store all historic data in the lookup, enabling historic search.

columnTimeReference

string

If given, the Lookup Manager will use the column specified as time instant reference for the data provided.

Check below an example of request body:

Code Block
{
    "visibility": {
      "type": "creator-only"
    },
    "query": "from box.unix group by machine, srcIp",
    "key": {
        "type": "column",
        "column": "machine"
    },
    "windowSize": "30d",
    "refreshPeriod" : "5m"
}
Rw expand
titleResponses

Code

Description

200

Successful response. Request submitted. The response includes the ID of the update request.

Code Block
{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 200,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select userid, domain from siem.logtrust.web.activity"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "userid"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 600000,
      "startMillis": 1702339200000,
      "slideWindow": 86400000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

Code Block
{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

Code Block
{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-sliding-window-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

Code Block
{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

Code Block
{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

Anchor
path17
path17
DELETE/lookup/{domain}/{name}

Send a request to delete a specific lookup.

Rw ui expands macro
Rw expand
titleRequest

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the lookup you want to delete.

name required

string

Enter the name of the lookup you want to delete.

Example

Find below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article.

Code Block
curl -H "standAloneToken:YOUR_TOKEN" -X DELETE   https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup
Rw expand
titleResponses

Code

Description

200

Successful response. Deletion request sent. The response includes the ID of the request.

Code Block
{
  "type": "LookupDeletionResponse",
  "cid": "f44f458f7c32",
  "code": 200,
  "context": null,
  "id": "xxxxxx-5052-11ed-b24b-85c623a0cbd8",
  "msg": "Lookup sent to deletion"
}

401

Unsuccessful response. The user is unauthorized to delete this lookup.

Code Block
{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}