Overview

Check the list of available endpoints and methods in the Lookups API:

Endpoints and methods

Description

1

 /lookup/{domain} see below

Display information on the lookups existing on a given domain.

2

 /lookup/{domain}/{name} see below

Return information of a specific lookup.

3

 /lookup/{domain}/{name}/job see below

Query the job UUIDs of a specific lookup.

4

/lookup/{domain}/{name}/job/{id} see below

Query the statuses of a lookup job.

5

Advanced mode

 lookup/{domain}/{name}/deploy-config see below

Advanced mode

Create a new lookup.

6

 lookup/{domain}/{name}/deploy-csv see below

Create a new lookup using a CSV (application/json request body)

7

 lookup/{domain}/{name}/deploy-csv see below

Create a new lookup using a CSV (multipart/form-data request body)

8

 lookup/{domain}/{name}/deploy-static-query see below

Create a new lookup based on a static query.

9

 lookup/{domain}/{name}/deploy-periodic-query see below

Create a new lookup based on a periodic query.

10

 lookup/{domain}/{name}/deploy-sliding-window-query see below

Create a new lookup based on a sliding window query.

11

Advanced mode

lookup/{domain}/{name}/deploy-config see below

Advanced mode

Update a specific lookup.

12

lookup/{domain}/{name}/deploy-csv see below

Update a CSV lookup (application/json request body)

13

lookup/{domain}/{name}/deploy-csv see below

Update a CSV lookup (multipart/form-data request body)

14

lookup/{domain}/{name}/deploy-static-query see below

Update a lookup based on a static query.

15

lookup/{domain}/{name}/deploy-periodic-query see below

Update a lookup based on a periodic query.

16

lookup/{domain}/{name}/deploy-sliding-window-query see below

Update a lookup based on a sliding window query.

17

 /lookup/{domain}/{name} see below

Send a request to delete a specific lookup.

Endpoints and methods

GET /lookup/{domain}

Display information on the lookups existing on a given domain.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the lookups you want to list.

Query string parameters

Query string parameters are optionally added after the path parameters, preceded by a question mark (?) and separated by an ampersand (&)

Parameter

Type

Description

max_length

integer

Maximum number of values to return. The default value is 100. The maximum value is 1000 and the minimum value is 1.

Lookups are shown from oldest to newest.

pageToken

integer

Use this parameter to consider a previously requested list on your next request. This should be set to the value shown in the nextPageToken parameter included in the response you want to consider, if successful.

For example, you may perform a request and ask for information about the oldest 5 lookups in your domain. Then, you can perform a second request including the value in the nextPageToken parameter returned with the previous response and requesting only 2 lookups. This will return the 2 lookups created after the first 5 previously requested.

The default and minimum value is 0.

owner

string

You can filter the lookups in the response based on their domain owner, that is, the domain where the lookup was created. Supported values are:

  • THIS_DOMAIN - Returns only lookups created in the domain indicated in the request URL.

  • OTHER_DOMAINS - Returns only lookups owned by other domains but visible by the domain indicated in the request URL. This might be the case of domains that belong to a multitenant structure and have access to lookups defined in other domains that belong to that structure. You can decide the visibility level of a new lookup upon its creation.

  • ANY_DOMAIN - Returns both lookups owned by the domain indicated in the request URL and those visible by them in other domains.

The default value is THIS_DOMAIN

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can decide to make a lookup visible in all the domains within the structure when you create it using POST /lookup/{domain}/{name}/deploy-config. See how to do it here.

Example

Find below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article.

curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?max_length=10

And this request will return information about all the lookups owned by the user domain and also the ones visible to them in other domains:

curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?owner=ANY_DOMAIN

Code

Description

200

Successful response. The response includes the list of lookups in the given domain with their details.

{
  "type": "LookupMetaInfoListResponse",
  "cid": "72078e04ee84",
  "code": 200,
  "context": null,
  "id": "xxxxxxxxxx-0f74-11ee-b13b-4fc634871e5f",
  "msg": "tutorial lookups.",
  "lookups": [
    {
      "name": "Lookup_test_t",
      "domain": "tutorial",
      "fileSize": 12288,
      "numEntries": 136,
      "creationDate": "2023-03-15T13:45:57.63",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "domain",
          "type": "str",
          "key": true
        },
        {
          "column": "CDNProvider",
          "type": "str",
          "key": false
        }
      ],
      "owner": "user@devo.com"
    },
    {
      "name": "CDN_Providers",
      "domain": "tutorial",
      "fileSize": 12288,
      "numEntries": 136,
      "creationDate": "2023-03-15T13:45:57.806",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "domain",
          "type": "str",
          "key": true
        },
        {
          "column": "CDNProvider",
          "type": "str",
          "key": false
        }
      ],
      "shared": false,
      "owner": "user@devo.com"
    },
    {
      "name": "test_101",
      "domain": "tutorial",
      "fileSize": 40960,
      "numEntries": 307,
      "creationDate": "2023-03-15T13:45:58.338",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "alertName",
          "type": "str",
          "key": true
        },
        {
          "column": "alertType",
          "type": "str",
          "key": false
        },
        {
          "column": "alertMitreTactics",
          "type": "str",
          "key": false
        },
        {
          "column": "alertMitreTechniques",
          "type": "str",
          "key": false
        },
        {
          "column": "alertPriority",
          "type": "int4",
          "key": false
        }
      ],
      "owner": "user@devo.com"
    },
    {
      "name": "d14022023api",
      "domain": "tutorial",
      "fileSize": 8192,
      "numEntries": 1,
      "creationDate": "2023-03-15T13:46:58.05",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "key",
          "type": "int4",
          "key": true
        },
        {
          "column": "fbool",
          "type": "bool",
          "key": false
        }
      ],
      "owner": null
    }
  ],
  "nextPageToken": -1
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "046ead1c8323",
  "code": 400,
  "context": null,
  "id": "xxxxxx-4fbb-11ed-859a-f19049cfd717"
}

401

Unsuccessful response. The user is unauthorized to list the domain's lookups.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "e33f677a6981",
  "code": 404,
  "context": null,
  "id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
  "msg": "Not Found"
}

GET /lookup/{domain}/{name}

Return information of a specific lookup.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the lookup you want to retrieve.

name required

string

Enter the name of the lookup you want to get information about.

Example

Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.

curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup

Code

Description

200

Successful response. The response includes information about the requested lookup.

{
  "type": "LookupMetaInfoResponse",
  "cid": "3c9bb672512c",
  "code": 200,
  "context": null,
  "id": "xxxxxxxxx-0f75-11ee-b13b-636d49ade562",
  "msg": "tutorial/test_101 meta information.",
  "lookupMetaInfo": {
    "name": "test_101",
    "domain": "tutorial",
    "fileSize": 40960,
    "numEntries": 307,
    "creationDate": "2023-03-15T13:45:58.14",
    "keyType": {
      "type": "first",
      "columns": null,
      "hasher": null
    },
    "deployConfig": null,
    "lastStatus": null,
    "fields": [
      {
        "column": "alertName",
        "type": "str",
        "key": true
      },
      {
        "column": "alertType",
        "type": "str",
        "key": false
      },
      {
        "column": "alertMitreTactics",
        "type": "str",
        "key": false
      },
      {
        "column": "alertMitreTechniques",
        "type": "str",
        "key": false
      },
      {
        "column": "alertPriority",
        "type": "int4",
        "key": false
      }
    ],
    "owner": "user@devo.com"
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "4b6cdd188a82",
  "code": 400,
  "context": null,
  "id": "xxxxxx-5052-11ed-859a-910aa97972a3",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to get information about the lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

404

Unsuccessful response. Lookup not found.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "e33f677a6981",
  "code": 404,
  "context": null,
  "id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
  "msg": "Not Found"
}

GET /lookup/{domain}/{name}/job

Query the job UUIDs (Universally Unique IDentifier) of a specific lookup. A new job is created for each create, update or delete process that is initiated. Each job is identified by a unique ID, or UUID.

You will get the last 10 UUIDs for the requested lookup. The list is ordered chronologically being the first job UUID the oldest and the last one the newest.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the required lookup.

name required

string

Enter the name of the required lookup.

Example

Find below a request example in cURL language. This request will return information about the last jobs generated by the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.

curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job

Code

Description

200

Successful response. Returns the job UUIDs of the given lookup.

{
  "cid": "e47f4ab72ded",
  "code": 200,
  "context": null,
  "id": "xxxxxxxx-e37c-11ed-b5ea-0242ac120002",
  "msg": "Lookup job uuids",
  "jobs": [
    "xxxxxxx-c9a2-489c-8794-ea656a19b822",
    "xxxxxxx-9714-48a7-9976-73e41523edfd",
    "xxxxxxx-48a8-46ea-ab22-e0a5458e302b",
    "xxxxxxx-ad7e-4fe6-bb43-89f93e629d76"
  ]
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "6cde7fd6a1f4",
  "code": 400,
  "context": null,
  "id": "xxxxxxx-53a2-11ed-859a-33f9a0bb845a",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to access the lookup

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

GET /lookup/{domain}/{name}/job/{id}

Query the statuses of a lookup job given its UUID. You can get a job UUID using the request explained above.

You will get the last 50 statuses for the requested lookup job. The list is ordered chronologically being the first status the oldest and the last one the newest.

These are the possible status codes you may get in your response:

Code

Status

create.start

Lookup creation started.

create.ok

Lookup successfully created.

create.err

Error creating lookup.

push.start

Lookup synchronous upload started.

push.ok

Successful lookup synchronous upload.

push.err

Lookup synchronous upload failed.

deploy.start

Lookup asynchronous upload started.

deploy.ok

Successful lookup asynchronous upload started.

deploy.err

Lookup asynchronous upload failed.

del.start

Lookup deletion started.

del.ok

Lookup successfully deleted.

del.err

Error deleting lookup.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the required lookup.

name required

string

Enter the name of the required lookup.

id required

string

Enter the UUID of the required lookup job. You can get a job UUID using the request explained above, under the jobs parameter in the response.

Example

Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.

curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job/123456

Code

Description

200

Successful response. Returns the job statuses.

{
  "cid": "e47f4ab72ded",
  "code": 200,
  "context": null,
  "id": null,
  "msg": "Lookup job's statuses",
  "status": [
    {
      "eventdata": "2021-09-29T10:18:10.805",
      "domain": "galactic_empire",
      "lookup": "ImperialIntranetActivity",
      "msg": "Lookup successfully created"
      "code": "create.ok"
    },
    {
      "eventdata": "2021-09-29T10:18:12.472",
      "domain": "ImperialIntranetActivity",
      "lookup": "test-schedule",
      "msg": "Lookup ready to be executed"
      "code": "deploy.ok"
    }
  ]
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupJobsError",
  "cid": "4d1eb85a908d",
  "code": 400,
  "context": null,
  "id": "xxxxxx-53a4-11ed-859a-2d69d242a54f",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to access the lookup

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

POST /lookup/{domain}/{name}/deploy-config

Create a new lookup.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the new lookup.

name required

string

Enter the name of the new lookup.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

id - Lookup ID object.

Parameter

Type

Description

creator

string

Domain of the lookup.

name

string

Name of the lookup.

Example value:

"id": {
  "creator": "self",
  "name": "ForceSensitiveBeings"
}

visibility - (string) Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

recipe required - Recipe of the lookup to be created.

Parameter

Type

Description

recipeType required

string

Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article.

source required

object

Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field).

Check more about uploading a lookup using a CSV through S3, see this article.

  • columns - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketName required (string) - Name of the S3 bucket where the CSV is located.

    • keyName required (string) - Path of the CSV in the bucket.

    • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

    • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • host - (string) - The S3 bucket host where the CSV is located.

    • port - (integer) - The S3 bucket port where the CSV is located.

    • region - (string) - The AWS region where the S3 bucket has been created.

  • query - (string) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

    • boolean

    • string

    • int4

    • int8

    • float4

    • float8

    • hex4

    • hex8

    • ip4

    • ip6

    • timestamp

    • duration

lookupType

object

Indicate the lookup type.

  • type required - (string) Valid values are normal (regular lookups) and historic (time range lookups).

  • instantPolicy - (string) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic. Possible values are:

    • natural - The associated source must be historical and the instant of each row will be used.

    • const - The instance of all rows will be the supplied constant value.

    • column - The instance will be extracted from the column with the given name. That column type must be timestamp.

  • instant - (number) Only for historic lookups of type const.

  • columnName - (string) Only for historic lookups of type column.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

columnFilter

array

If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete.

secondaryIndexes

object

Columns of a lookup indexed as secondary indexes.

  • type required - (string) Type of the secondary indexes to be applied to the lookup. Values can be all, none or by-name. The default value is none.

  • map - Map of column names to be applied as secondary indexes when the type selected is by-name.

refreshMillis

number

Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic.

startMillis

number

Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic.

requiresDate

boolean

If true, the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now(). Can only be used if the recipe type chosen is periodic. The default value is false.

notifyStatus - (boolean) If true, a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false by default.

Check below an example of request body:

{
  "id": {
    "creator": "rebel_alliance",
    "name": "TotallyNotFakeData"
  },
  "recipe": {
    "recipeType": "once",
    "source": {
      "query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
    },
    "lookupType": {
      "type": "normal"
    },
    "append": false,
    "key": {
      "type": "column",
      "column": "key"
    },
    "columnFilter": [
      "key",
      "IsDataFake",
      "RebelsImprisoned",
      "CreditsOnImperialBanks",
      "Hex4Emperor",
      "Hex8Vader",
      "EmperorClones",
      "Pi",
      "EmperorIP4",
      "EmperorIP6",
      "TimeAfterBattleOfYavin"
    ],
    "contribution": {
      "type": "add"
    },
    "requiresDate": false
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the creation request.

{
  "type": "LookupCreationResponse",
  "cid": "d41c91a21d56",
  "code": 201,
  "context": null,
  "id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
  "msg": "Lookup sent to creation",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": "creator-only",
    "recipe": {
      "type": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      }
    }
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": "BAD_REQUEST",
  "id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
  "message": "Bad Request"
}

401

Unsuccessful response. The user is unauthorized to create a lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

403

Unsuccessful response. User credentials are correct but does not have permission to create this lookup.

{
  "type": "LookupCreationError",
  "cid": "9a3eda1848d1",
  "code": 403,
  "context": "FORBIDDEN",
  "id": "xxxxxx5055-11ed-859a-7b524e50491a",
  "msg": "User is not authorized to perform operations in the domain."
}

POST /lookup/{domain}/{name}/deploy-csv

Simplified endpoint to create a new lookup from a CSV file (application/json request body).

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

columns required

array

Array of lookup column descriptors. Each column object may contain the following parameters:

  • name required - (string) Name of the column.

  • from - (integer) 0-based index of the column in the CSV.

  • type - (string) Data type of the column. Possible values are:

    • BOOLEAN

    • STRING

    • INT4

    • INT8

    • FLOAT4

    • FLOAT8

    • HEX4

    • HEX8

    • IP4

    • IP6

    • TIMESTAMP

    • DURATION

fileProvider

array

Info to get the CSV file from S3.

  • bucketName required (string) - Name of the S3 bucket where the CSV is located.

  • keyName required (string) - Path of the CSV in the bucket.

  • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

  • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • host - (string) - The S3 bucket host where the CSV is located.

  • port - (integer) - The S3 bucket port where the CSV is located.

  • region - (string) - The AWS region where the S3 bucket has been created.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

skipPreface

string

Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

hasHeader

boolean

Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

skipEmptyLines

boolean

Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

{
  "columns": [
    {
      "name": "ID",
      "type": "INT8"
    },
    {
      "name": "Location",
      "type": "STRING"
    },
    {
      "name": "Age",
      "type": "FLOAT8"
    },
    {
      "name": "Species",
      "type": "STRING"
    },
    {
      "name": "MidiclorianLevel",
      "type": "FLOAT4"
    }
  ],
  "key": {
    "type": "column",
    "column": "ID"
  },
  "fileProvider": {
    "bucketName": "holocrons-bucket",
    "keyName": "secrets/data/force-sensitive-beings.csv",
    "transferOwnership": true
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

POST /lookup/{domain}/{name}/deploy-csv

Simplified endpoint to create a new lookup from a CSV file (multipart/form-data request body).

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request multipart/form-data body must include the CSV file and an associated JSON configuration object. Click them in the following list to see its details:

Parameter

Type

Description

deploy-config

object

Deploy CSV configuration of the lookup to be created. This object states the visibility of the created lookup, the fileProvider indicating the CSV file stored in AWS S3, the column descriptors and key. It also includes metainfo from the CSV.

  • visibility - (string) Visibility level of the lookup. Allowed values are:

    • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

    • all-subdomains - (string) The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

  • columns required - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketNamerequired - (string) The name of the S3 bucket where the CSV is located.

    • keyNamerequired - (string) The path of the CSV inside the bucket.

    • transferOwnershiprequired - (boolean) This parameter represents whether Lomana should act as the owner of the CSV and delete it from the bucket once it has finished creating the lookup.

    • accessKey - (string) The access key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • secretKey - (string) The secret key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • host - (string) The S3 bucket host where to find the CSV.

    • port - (integer) The S3 bucket port where to find the CSV.

    • region - (string) The AWS region whether the S3 bucket has been created.

  • key - (string) In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

    • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

    • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

    • type required - (string). Type of key algorithm. Values can be:

      • first-column - The first column of the lookup will be used as key. This is the default value.

      • column - The column indicated in the column parameter will be used as key.

      • row-hash - A hash of the elements on the first row of the lookup will be used as key.

      • col-hash - A hash of the columns stated in the columns parameter will be used as key.

      • seq - The key of the lookup will be generated sequentially.

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • contribution - (array) Defines how a row contributes to the final result, normally used on incremental lookups.

    • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

    • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

  • keepHistory - (boolean) If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

  • columnTypeReference- (string) If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

  • append- (boolean) If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

csv

file

CSV file to upload. Only needed if the file was not uploaded to S3.

Check more about uploading a lookup using a CSV through S3, see this article.

Check below an example of request body:

{
  "deploy-config": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/deploy-csv.json"
  },
  "csv": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/simple_lookup.csv"
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

POST /lookup/{domain}/{name}/deploy-static-query

Simplified endpoint to create a new lookup based on a static query.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-1d < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  },
  "keepHistory": false,
  "columnTimeReference": null
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      }
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-static-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

POST /lookup/{domain}/{name}/deploy-periodic-query

Simplified endpoint to create a new lookup based on a periodic query.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queryable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

refreshPeriod

string

Refresh period of the lookup. Defaults to grouping period of the query in case it is a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo's duration type.

The minimum value accepted is 1 minute.

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

Check below an example of request body:

{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-5m < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  },
  "refreshPeriod": "5m",
  "startDate": "2023-12-12T00:00:00.00Z",
  "append": true,
  "keepHistory": false,
  "columnTimeReference": null
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 300000,
      "startMillis": 1702339200000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-periodic-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

POST /lookup/{domain}/{name}/deploy-sliding-window-query

Simplified endpoint to create a new lookup based on a sliding window query.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

keyrequired

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

windowSizerequired

string

Size of the sliding window. Every time the query is executed, the lookup will contain the size of the window. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 hour (Example: Xh, Xd).

refreshPeriod required

string

Lookup refresh period. Defaults to query’s grouping period in case of a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 minute (Example: Xm, Xh, Xd).

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

keepHistory

boolean

If set to true, the Lookup Manager will store all historic data in the lookup, enabling historic search.

columnTimeReference

string

If given, the Lookup Manager will use the column specified as time instant reference for the data provided.

Check below an example of request body:

{
    "visibility": {
      "type": "creator-only"
    },
    "query": "from box.unix group by machine, srcIp",
    "key": {
        "type": "column",
        "column": "machine"
    },
    windowSize": "30d",
    "refreshPeriod" : "5m"
}

Code

Description

200

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 200,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select userid, domain from siem.logtrust.web.activity"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "userid"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 600000,
      "startMillis": 1702339200000,
      "slideWindow": 86400000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-sliding-window-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

PUT lookup/{domain}/{name}/deploy-config

Update a specific lookup.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where that contains the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

id  - Lookup ID object.

Parameter

Type

Description

creator

string

Domain of the lookup.

name

string

Name of the lookup.

Example value:

"id": {
  "creator": "self",
  "name": "ForceSensitiveBeings"
}

visibility - (string) Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain.. Only multitenant Admin users will be able to use this value.

recipe required - Recipe of the lookup to be updated.

Parameter

Type

Description

recipeType required

string

Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article.

source required

object

Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field).

  • columns - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketName required (string) - Name of the S3 bucket where the CSV is located.

    • keyName required (string) - Path of the CSV in the bucket.

    • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

    • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

    • host - (string) - The S3 bucket host where the CSV is located.

    • port - (integer) - The S3 bucket port where the CSV is located.

    • region - (string) - The AWS region where the S3 bucket has been created.

  • query - (string) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

    • boolean

    • string

    • int4

    • int8

    • float4

    • float8

    • hex4

    • hex8

    • ip4

    • ip6

    • timestamp

    • duration

lookupType

object

Indicate the lookup type.

  • type required - (string) Valid values are normal (regular lookups) and historic (time range lookups).

  • instantPolicy - (string) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic. Possible values are:

    • natural - The associated source must be historical and the instant of each row will be used.

    • const - The instance of all rows will be the supplied constant value.

    • column - The instance will be extracted from the column with the given name. That column type must be timestamp.

  • instant - (number) Only for historic lookups of type const.

  • columnName - (string) Only for historic lookups of type column.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

columnFilter

array

If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete.

secondaryIndexes

object

Columns of a lookup indexed as secondary indexes.

  • type required - (string) Type of the secondary indexes to be applied to the lookup. Values can be all, none or by-name. The default value is none.

  • map - Map of column names to be applied as secondary indexes when the type selected is by-name.

refreshMillis

number

Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic.

startMillis

number

Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic.

requiresDate

boolean

If true, the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now(). Can only be used if the recipe type chosen is periodic. The default value is false.

notifyStatus - (boolean) If true, a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false by default.

Check below an example of request body:

{
  "id": {
    "creator": "rebel_alliance",
    "name": "TotallyNotFakeData"
  },
  "recipe": {
    "recipeType": "once",
    "source": {
      "query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
    },
    "lookupType": {
      "type": "normal"
    },
    "append": false,
    "key": {
      "type": "column",
      "column": "key"
    },
    "columnFilter": [
      "key",
      "IsDataFake",
      "RebelsImprisoned",
      "CreditsOnImperialBanks",
      "Hex4Emperor",
      "Hex8Vader",
      "EmperorClones",
      "Pi",
      "EmperorIP4",
      "EmperorIP6",
      "TimeAfterBattleOfYavin"
    ],
    "contribution": {
      "type": "add"
    },
    "requiresDate": false
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d41c91a21d56",
  "code": 201,
  "context": null,
  "id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
  "msg": "Lookup sent to creation",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": "creator-only",
    "recipe": {
      "type": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      }
    }
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": "BAD_REQUEST",
  "id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
  "message": "Bad Request"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

403

Unsuccessful response. User credentials are correct but does not have permission to update this lookup.

{
  "type": "LookupCreationError",
  "cid": "9a3eda1848d1",
  "code": 403,
  "context": "FORBIDDEN",
  "id": "xxxxxx5055-11ed-859a-7b524e50491a",
  "msg": "User is not authorized to perform operations in the domain."
}

PUT lookup/{domain}/{name}/deploy-csv

Update a CSV lookup (application/json request body).

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the domain name of the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

columns required

array

Array of lookup column descriptors. Each column object may contain the following parameters:

  • name required - (string) Name of the column.

  • from - (integer) 0-based index of the column in the CSV.

  • type - (string) Data type of the column. Possible values are:

    • BOOLEAN

    • STRING

    • INT4

    • INT8

    • FLOAT4

    • FLOAT8

    • HEX4

    • HEX8

    • IP4

    • IP6

    • TIMESTAMP

    • DURATION

fileProvider

array

Info to get the CSV file from S3.

  • bucketName required (string) - Name of the S3 bucket where the CSV is located.

  • keyName required (string) - Path of the CSV in the bucket.

  • transferOwnership required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

  • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • host - (string) - The S3 bucket host where the CSV is located.

  • port - (integer) - The S3 bucket port where the CSV is located.

  • region - (string) - The AWS region where the S3 bucket has been created.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

skipPreface

string

Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

hasHeader

boolean

Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

skipEmptyLines

boolean

Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

  • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

  • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

{
  "columns": [
    {
      "name": "ID",
      "type": "INT8"
    },
    {
      "name": "Location",
      "type": "STRING"
    },
    {
      "name": "Age",
      "type": "FLOAT8"
    },
    {
      "name": "Species",
      "type": "STRING"
    },
    {
      "name": "MidiclorianLevel",
      "type": "FLOAT4"
    }
  ],
  "key": {
    "type": "column",
    "column": "ID"
  },
  "fileProvider": {
    "bucketName": "holocrons-bucket",
    "keyName": "secrets/data/force-sensitive-beings.csv",
    "transferOwnership": true
  },
  "contribution": {
    "type": "add"
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupUpdateResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

PUT lookup/{domain}/{name}/deploy-csv

Update a CSV lookup (multipart/form-data request body).

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request multipart/form-data body must include the CSV file and an associated JSON configuration object. Click them in the following list to see its details:

Parameter

Type

Description

deploy-config

object

Deploy CSV configuration of the lookup to be created. This object states the visibility of the created lookup, the fileProvider indicating the CSV file stored in AWS S3, the column descriptors and key. It also includes metainfo from the CSV.

  • visibility - (string) Visibility level of the lookup. Allowed values are:

    • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

    • all-subdomains - (string) The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

  • columns required - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

    • name required - (string) Name of the column.

    • from - (integer) 0-based index of the column in the CSV.

    • type - (string) Data type of the column. Possible values are:

      • BOOLEAN

      • STRING

      • INT4

      • INT8

      • FLOAT4

      • FLOAT8

      • HEX4

      • HEX8

      • IP4

      • IP6

      • TIMESTAMP

      • DURATION

  • fileProvider - (array) Info to get the CSV file from S3.

    • bucketNamerequired - (string) The name of the S3 bucket where the CSV is located.

    • keyNamerequired - (string) The path of the CSV inside the bucket.

    • transferOwnershiprequired - (boolean) This parameter represents whether Lomana should act as the owner of the CSV and delete it from the bucket once it has finished creating the lookup.

    • accessKey - (string) The access key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • secretKey - (string) The secret key of a customer’s AWS user with permissions to access the bucket represented in bucketName.

    • host - (string) The S3 bucket host where to find the CSV.

    • port - (integer) The S3 bucket port where to find the CSV.

    • region - (string) The AWS region whether the S3 bucket has been created.

  • key - (string) In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

    • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

    • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

    • type required - (string). Type of key algorithm. Values can be:

      • first-column - The first column of the lookup will be used as key. This is the default value.

      • column - The column indicated in the column parameter will be used as key.

      • row-hash - A hash of the elements on the first row of the lookup will be used as key.

      • col-hash - A hash of the columns stated in the columns parameter will be used as key.

      • seq - The key of the lookup will be generated sequentially.

  • skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

  • contribution - (array) Defines how a row contributes to the final result, normally used on incremental lookups.

    • type required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

    • name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete

  • keepHistory - (boolean) If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

  • columnTypeReference- (string) If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

  • append- (boolean) If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

csv

file

CSV file to upload. Only needed if the file was not uploaded to S3.

Check more about uploading a lookup using a CSV through S3, see this article.

Check below an example of request body:

{
  "deploy-config": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/deploy-csv.json"
  },
  "csv": {
    "externalValue": "https://devo.com/main/files/simple-lookup-with-header/simple_lookup.csv"
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "columns": [
          {
            "name": "ID",
            "type": "INT8"
          },
          {
            "name": "Location",
            "type": "STRING"
          },
          {
            "name": "Age",
            "type": "FLOAT8"
          },
          {
            "name": "Species",
            "type": "STRING"
          },
          {
            "name": "MidiclorianLevel",
            "type": "FLOAT4"
          }
        ],
        "fileProvider": {
          "bucketName": "holocrons-bucket",
          "keyName": "secrets/data/force-sensitive-beings.csv",
          "transferOwnership": true
        },
        "skipPreface": null,
        "hasHeader": false,
        "skipEmptyLines": false
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": null,
      "startMillis": null,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

PUT lookup/{domain}/{name}/deploy-static-query

Update a lookup based on a static query.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the domain name of the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

Check below an example of request body:

{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-1d < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  }
}

Code

Description

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      }
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-static-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

PUT lookup/{domain}/{name}/deploy-periodic-query

Update a lookup based on a periodic query.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the domain name of the lookup you want to update.

name required

string

Enter the name of the lookup you want to update.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queryable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

refreshPeriod

string

Refresh period of the lookup. Defaults to grouping period of the query in case it is a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo's duration type.

The minimum value accepted is 1 minute.

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. If there is no previous lookup, or if the recipe is updated, this property is ignored. The default value is false.

keepHistory

boolean

If true, the Lookup Manager will store in the lookup all historic data, enabling historic search. The default value is false.

columnTimeReference

string

If given, the Lookup Manager will use the specified column as time instant reference for the data provided.

Check below an example of request body:

{
  "visibility": {
    "type": "creator-only"
  },
  "query": "select userid, domain from siem.logtrust.web.navigation where now()-5m < eventdate < now()",
  "key": {
    "type": "column",
    "column": "userid"
  },
  "refreshPeriod": "5m",
  "startDate": "now",
  "append": true
}

Code

Description

200

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 201,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 300000,
      "startMillis": 1702339200000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-periodic-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

PUT lookup/{domain}/{name}/deploy-sliding-window-query

Update a lookup based on a sliding window query.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain where you want to create the lookup.

name required

string

Enter the name of the lookup you want to create.

Request body

The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:

Parameter

Type

Description

visibility

string

Visibility level of the lookup. Allowed values are:

  • creator-only - The lookup will be visible only in the creator's domain. This is the default value.

  • all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.

  • all-domains - To create a lookup visible by all domains. It is only allowed when the domain of the user is self.

Lookup ownership VS visibility

  • Lookup ownership refers to the domain where the lookup is defined.

  • Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.

query required

string

Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

keyrequired

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

  • columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

  • column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

  • type required - (string) Type of key algorithm. Values can be:

    • first-column - The first column of the lookup will be used as key. This is the default value.

    • column - The column indicated in the column parameter will be used as key.

    • row-hash - A hash of the elements on the first row of the lookup will be used as key.

    • col-hash - A hash of the columns stated in the columns parameter will be used as key.

    • seq - The key of the lookup will be generated sequentially.

windowSizerequired

string

Size of the sliding window. Every time the query is executed, the lookup will contain the size of the window. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 hour (Example: Xh, Xd).

refreshPeriod required

string

Lookup refresh period. Defaults to query’s grouping period in case of a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo duration type. The minimum value accepted is 1 minute (Example: Xm, Xh, Xd).

startDate

string

Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch.

keepHistory

boolean

If set to true, the Lookup Manager will store all historic data in the lookup, enabling historic search.

columnTimeReference

string

If given, the Lookup Manager will use the column specified as time instant reference for the data provided.

Check below an example of request body:

{
    "visibility": {
      "type": "creator-only"
    },
    "query": "from box.unix group by machine, srcIp",
    "key": {
        "type": "column",
        "column": "machine"
    },
    "windowSize": "30d",
    "refreshPeriod" : "5m"
}

Code

Description

200

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d5ce4eb105b2",
  "code": 200,
  "context": null,
  "id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
  "msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": {
      "type": "creator-only"
    },
    "recipe": {
      "recipeType": "periodic",
      "source": {
        "query": "select userid, domain from siem.logtrust.web.activity"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": true,
      "key": {
        "type": "column",
        "column": "userid"
      },
      "columnFilter": null,
      "contribution": {
        "type": "add"
      },
      "secondaryIndexes": {
        "type": "none"
      },
      "refreshMillis": 600000,
      "startMillis": 1702339200000,
      "slideWindow": 86400000,
      "requiresDate": true
    },
    "notifyStatus": true
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": null,
  "id": "9f270aca-5151-11ed-859a-5d2974203ed5",
  "msg": "refreshPeriod should be bigger than 1 minute"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "code": 401,
  "msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-sliding-window-query' requires valid auth",
  "cid": "29fda52318ae"
}

403

Unsuccessful response. Forbidden access.

{
  "code": 5,
  "msg": "Token invalid or expired",
  "cid": "29fda52318ae"
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 404,
  "context": null,
  "id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
  "msg": "Domain not found."
}

409

Unsuccessful response. Lookup already exists.

{
  "type": "LookupCreationError",
  "cid": "68e612d824a5",
  "code": 409,
  "context": null,
  "id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
  "msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
}

DELETE /lookup/{domain}/{name}

Send a request to delete a specific lookup.

Path parameters

Add the following path parameters as part of the endpoint:

Parameter

Type

Description

domain required

string

Enter the name of the domain that contains the lookup you want to delete.

name required

string

Enter the name of the lookup you want to delete.

Example

Find below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article.

curl -H "standAloneToken:YOUR_TOKEN" -X DELETE   https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup

Code

Description

200

Successful response. Deletion request sent. The response includes the ID of the request.

{
  "type": "LookupDeletionResponse",
  "cid": "f44f458f7c32",
  "code": 200,
  "context": null,
  "id": "xxxxxx-5052-11ed-b24b-85c623a0cbd8",
  "msg": "Lookup sent to deletion"
}

401

Unsuccessful response. The user is unauthorized to delete this lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}