Page Comparison

domain ^required

string

Enter the name of the domain that contains the lookups you want to list.

max_length

integer

Maximum number of values to return. The default value is 100. The maximum value is 1000 and the minimum value is 1.

Lookups are shown from oldest to newest.

pageToken

integer (int32)

Use this parameter to consider a previously requested list on your next request. This should be set to the value shown in the nextPageToken parameter included in the response you want to consider, if successful.

For example, you may perform a request and ask for information about the oldest 5 lookups in your domain. Then, you can perform a second request including the value in the nextPageToken parameter returned with the previous response and requesting only 2 lookups. This will return the 2 lookups created after the first 5 previously requested.

The default and minimum value is 0.

owner

string

You can filter the lookups in the response based on their domain owner, that is, the domain where the lookup was created. Supported values are:

• THIS_DOMAIN - Returns only lookups created in the domain indicated in the request URL.

• OTHER_DOMAINS - Returns only lookups owned by other domains but visible by the domain indicated in the request URL. This might be the case of domains that belong to a multitenant structure and have access to lookups defined in other domains that belong to that structure. You can decide the visibility level of a new lookup upon its creation.

• ANY_DOMAIN - Returns both lookups owned by the domain indicated in the request URL and those visible by them in other domains.

The default value is THIS_DOMAIN

200

Successful response. The response includes the list of lookups in the given domain with their details.

{
  "type": "LookupMetaInfoListResponse",
  "cid": "72078e04ee84",
  "code": 200,
  "context": null,
  "id": "xxxxxxxxxx-0f74-11ee-b13b-4fc634871e5f",
  "msg": "tutorial lookups.",
  "lookups": [
    {
      "name": "Lookup_test_t",
      "domain": "tutorial",
      "fileSize": 12288,
      "numEntries": 136,
      "creationDate": "2023-03-15T13:45:57.63",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "domain",
          "type": "str",
          "key": true
        },
        {
          "column": "CDNProvider",
          "type": "str",
          "key": false
        }
      ],
      "owner": "user@devo.com"
    },
    {
      "name": "CDN_Providers",
      "domain": "tutorial",
      "fileSize": 12288,
      "numEntries": 136,
      "creationDate": "2023-03-15T13:45:57.806",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "domain",
          "type": "str",
          "key": true
        },
        {
          "column": "CDNProvider",
          "type": "str",
          "key": false
        }
      ],
      "shared": false,
      "owner": "user@devo.com"
    },
    {
      "name": "test_101",
      "domain": "tutorial",
      "fileSize": 40960,
      "numEntries": 307,
      "creationDate": "2023-03-15T13:45:58.338",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "alertName",
          "type": "str",
          "key": true
        },
        {
          "column": "alertType",
          "type": "str",
          "key": false
        },
        {
          "column": "alertMitreTactics",
          "type": "str",
          "key": false
        },
        {
          "column": "alertMitreTechniques",
          "type": "str",
          "key": false
        },
        {
          "column": "alertPriority",
          "type": "int4",
          "key": false
        }
      ],
      "owner": "user@devo.com"
    },
    {
      "name": "d14022023api",
      "domain": "tutorial",
      "fileSize": 8192,
      "numEntries": 1,
      "creationDate": "2023-03-15T13:46:58.05",
      "keyType": {
        "type": "first",
        "columns": null,
        "hasher": null
      },
      "deployConfig": null,
      "lastStatus": null,
      "fields": [
        {
          "column": "key",
          "type": "int4",
          "key": true
        },
        {
          "column": "fbool",
          "type": "bool",
          "key": false
        }
      ],
      "owner": null
    }
  ],
  "nextPageToken": -1
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "046ead1c8323",
  "code": 400,
  "context": null,
  "id": "xxxxxx-4fbb-11ed-859a-f19049cfd717"
}

401

Unsuccessful response. The user is unauthorized to list the domain's lookups.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

404

Unsuccessful response. Domain not found.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "e33f677a6981",
  "code": 404,
  "context": null,
  "id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
  "msg": "Not Found"
}

domain ^required

string

Enter the name of the domain that contains the lookup you want to retrieve.

name ^required

string

Enter the name of the lookup you want to get information about.

200

Successful response. The response includes information about the requested lookup.

{
  "type": "LookupMetaInfoResponse",
  "cid": "3c9bb672512c",
  "code": 200,
  "context": null,
  "id": "xxxxxxxxx-0f75-11ee-b13b-636d49ade562",
  "msg": "tutorial/test_101 meta information.",
  "lookupMetaInfo": {
    "name": "test_101",
    "domain": "tutorial",
    "fileSize": 40960,
    "numEntries": 307,
    "creationDate": "2023-03-15T13:45:58.14",
    "keyType": {
      "type": "first",
      "columns": null,
      "hasher": null
    },
    "deployConfig": null,
    "lastStatus": null,
    "fields": [
      {
        "column": "alertName",
        "type": "str",
        "key": true
      },
      {
        "column": "alertType",
        "type": "str",
        "key": false
      },
      {
        "column": "alertMitreTactics",
        "type": "str",
        "key": false
      },
      {
        "column": "alertMitreTechniques",
        "type": "str",
        "key": false
      },
      {
        "column": "alertPriority",
        "type": "int4",
        "key": false
      }
    ],
    "owner": "user@devo.com"
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "4b6cdd188a82",
  "code": 400,
  "context": null,
  "id": "xxxxxx-5052-11ed-859a-910aa97972a3",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to get information about the lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

404

Unsuccessful response. Lookup not found.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "e33f677a6981",
  "code": 404,
  "context": null,
  "id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
  "msg": "Not Found"
}

domain ^required

string

Enter the name of the domain that contains the required lookup.

name ^required

string

Enter the name of the required lookup.

200

Successful response. Returns the job UUIDs of the given lookup.

{
  "cid": "e47f4ab72ded",
  "code": 200,
  "context": null,
  "id": "xxxxxxxx-e37c-11ed-b5ea-0242ac120002",
  "msg": "Lookup job uuids",
  "jobs": [
    "xxxxxxx-c9a2-489c-8794-ea656a19b822",
    "xxxxxxx-9714-48a7-9976-73e41523edfd",
    "xxxxxxx-48a8-46ea-ab22-e0a5458e302b",
    "xxxxxxx-ad7e-4fe6-bb43-89f93e629d76"
  ]
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupMetaInfoResponseError",
  "cid": "6cde7fd6a1f4",
  "code": 400,
  "context": null,
  "id": "xxxxxxx-53a2-11ed-859a-33f9a0bb845a",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to access the lookup

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

domain ^required

string

Enter the name of the domain that contains the required lookup.

name ^required

string

Enter the name of the required lookup.

id ^required

string

Enter the UUID of the required lookup job. You can get a job UUID using the request explained above, under the jobs parameter in the response.

200

Successful response. Returns the job statuses.

{
  "cid": "e47f4ab72ded",
  "code": 200,
  "context": null,
  "id": null,
  "msg": "Lookup job's statuses",
  "status": [
    {
      "eventdata": "2021-09-29T10:18:10.805",
      "domain": "galactic_empire",
      "lookup": "ImperialIntranetActivity",
      "msg": "Lookup successfully created"
      "code": "create.ok"
    },
    {
      "eventdata": "2021-09-29T10:18:12.472",
      "domain": "ImperialIntranetActivity",
      "lookup": "test-schedule",
      "msg": "Lookup ready to be executed"
      "code": "deploy.ok"
    }
  ]
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupJobsError",
  "cid": "4d1eb85a908d",
  "code": 400,
  "context": null,
  "id": "xxxxxx-53a4-11ed-859a-2d69d242a54f",
  "msg": "Unsuccessful response. Bad Request."
}

401

Unsuccessful response. The user is unauthorized to access the lookup

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

domain ^required

string

Enter the name of the domain where you want to create the new lookup.

name ^required

string

Enter the name of the new lookup.

creator

string

Domain of the lookup.

name

string

Name of the lookup.

recipeType ^required

string

Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article.

source ^required

object

Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field).

• columns - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

  • name ^required - (string) Name of the column.

  • from - (integer) 0-based index of the column in the CSV.

  • type - (string) Data type of the column. Possible values are:

    • BOOLEAN

    • STRING

    • INT4

    • INT8

    • FLOAT4

    • FLOAT8

    • HEX4

    • HEX8

    • IP4

    • IP6

    • TIMESTAMP

    • DURATION

• skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

• hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

• skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

• fileProvider - (array) Info to get the CSV file from S3.

  • bucketName ^required (string) - Name of the S3 bucket where the CSV is located.

  • keyName ^required (string) - Path of the CSV in the bucket.

  • transferOwnership ^required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

  • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • host - (string) - The S3 bucket host where the CSV is located.

  • port - (integer) - The S3 bucket port where the CSV is located.

  • region - (string) - The AWS region where the S3 bucket has been created.

• query - (string) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

lookupType

object

Indicate the lookup type.

• type ^required - (string) Valid values are normal (regular lookups) and historic (time range lookups).

• instantPolicy - (string) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic. Possible values are:

  • natural - The associated source must be historical and the instant of each row will be used.

  • const - The instance of all rows will be the supplied constant value.

  • column - The instance will be extracted from the column with the given name. That column type must be timestamp.

• instant - (number) Only for historic lookups of type const.

• columnName - (string) Only for historic lookups of type column.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

• columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

• column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

• type ^required - (string) Type of key algorithm. Values can be:

  • first-column - The first column of the lookup will be used as key. This is the default value.

  • column - The column indicated in the column parameter will be used as key.

  • row-hash - A hash of the elements on the first row of the lookup will be used as key.

  • col-hash - A hash of the columns stated in the columns parameter will be used as key.

  • seq - The key of the lookup will be generated sequentially.

columnFilter

array

If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

• type ^required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

• name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete.

secondaryIndexes

object

Columns of a lookup indexed as secondary indexes.

• type ^required - (string) Type of the secondary indexes to be applied to the lookup. Values can be all, none or by-name. The default value is none.

• map - Map of column names to be applied as secondary indexes when the type selected is by-name.

refreshMillis

number

Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic.

startMillis

number

Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic.

requiresDate

boolean

If true, the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now(). Can only be used if the recipe type chosen is periodic. The default value is false.

201

Successful response. Request submitted. The response includes the ID of the creation request.

{
  "type": "LookupCreationResponse",
  "cid": "d41c91a21d56",
  "code": 201,
  "context": null,
  "id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
  "msg": "Lookup sent to creation",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": "creator-only",
    "recipe": {
      "type": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      }
    }
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": "BAD_REQUEST",
  "id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
  "message": "Bad Request"
}

401

Unsuccessful response. The user is unauthorized to create a lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

403

Unsuccessful response. User credentials are correct but does not have permission to create this lookup.

{
  "type": "LookupCreationError",
  "cid": "9a3eda1848d1",
  "code": 403,
  "context": "FORBIDDEN",
  "id": "xxxxxx5055-11ed-859a-7b524e50491a",
  "msg": "User is not authorized to perform operations in the domain."
}

domain ^required

string

Enter the name of the domain where that contains the lookup you want to update.

name ^required

string

Enter the name of the lookup you want to update.

creator

string

Domain of the lookup.

name

string

Name of the lookup.

recipeType ^required

string

Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article.

source ^required

object

Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field).

• columns - (array) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:

  • name ^required - (string) Name of the column.

  • from - (integer) 0-based index of the column in the CSV.

  • type - (string) Data type of the column. Possible values are:

    • BOOLEAN

    • STRING

    • INT4

    • INT8

    • FLOAT4

    • FLOAT8

    • HEX4

    • HEX8

    • IP4

    • IP6

    • TIMESTAMP

    • DURATION

• skipPreface - (string) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

• hasHeader - (boolean) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

• skipEmptyLines - (boolean) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.

• fileProvider - (array) Info to get the CSV file from S3.

  • bucketName ^required (string) - Name of the S3 bucket where the CSV is located.

  • keyName ^required (string) - Path of the CSV in the bucket.

  • transferOwnership ^required (boolean) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.

  • accessKey - (string) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • secretKey - (string) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.

  • host - (string) - The S3 bucket host where the CSV is located.

  • port - (integer) - The S3 bucket port where the CSV is located.

  • region - (string) - The AWS region where the S3 bucket has been created.

• query - (string) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:

  • boolean

  • string

  • int4

  • int8

  • float4

  • float8

  • hex4

  • hex8

  • ip4

  • ip6

  • timestamp

  • duration

lookupType

object

Indicate the lookup type.

• type ^required - (string) Valid values are normal (regular lookups) and historic (time range lookups).

• instantPolicy - (string) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic. Possible values are:

  • natural - The associated source must be historical and the instant of each row will be used.

  • const - The instance of all rows will be the supplied constant value.

  • column - The instance will be extracted from the column with the given name. That column type must be timestamp.

• instant - (number) Only for historic lookups of type const.

• columnName - (string) Only for historic lookups of type column.

append

boolean

If true, when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation.

However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated.

The default value is false.

key

object

In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type.

• columns - (string) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash

• column - (string) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column

• type ^required - (string) Type of key algorithm. Values can be:

  • first-column - The first column of the lookup will be used as key. This is the default value.

  • column - The column indicated in the column parameter will be used as key.

  • row-hash - A hash of the elements on the first row of the lookup will be used as key.

  • col-hash - A hash of the columns stated in the columns parameter will be used as key.

  • seq - The key of the lookup will be generated sequentially.

columnFilter

array

If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected.

contribution

object

Defines how a row contributes to the final result, normally used on incremental lookups.

• type ^required - (string) Type of the contribution policy. Values can be add, del or col. The default value is add.

• name - (string) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete.

secondaryIndexes

object

Columns of a lookup indexed as secondary indexes.

• type ^required - (string) Type of the secondary indexes to be applied to the lookup. Values can be all, none or by-name. The default value is none.

• map - Map of column names to be applied as secondary indexes when the type selected is by-name.

refreshMillis

number

Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic.

startMillis

number

Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic.

requiresDate

boolean

If true, the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now(). Can only be used if the recipe type chosen is periodic. The default value is false.

201

Successful response. Request submitted. The response includes the ID of the update request.

{
  "type": "LookupCreationResponse",
  "cid": "d41c91a21d56",
  "code": 201,
  "context": null,
  "id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
  "msg": "Lookup sent to creation",
  "lookupDeployConfig": {
    "id": {
      "creator": "rebel_alliance",
      "name": "GalacticEmpireActivity"
    },
    "visibility": "creator-only",
    "recipe": {
      "type": "once",
      "source": {
        "query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
      },
      "lookupType": {
        "type": "normal"
      },
      "append": false,
      "key": {
        "type": "column",
        "column": "key"
      },
      "columnFilter": [
        "eventdate",
        "level",
        "domain",
        "userid",
        "sessionid",
        "correlationId"
      ],
      "contribution": {
        "type": "add"
      }
    }
  }
}

400

Unsuccessful response. Bad request.

{
  "type": "LookupCreationError",
  "cid": "0cd289fa1b63",
  "code": 400,
  "context": "BAD_REQUEST",
  "id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
  "message": "Bad Request"
}

401

Unsuccessful response. The user is unauthorized to update the lookup.

{
  "error": {
    "code": 401,
    "message": "Unauthorized"
  }
}

403

Unsuccessful response. User credentials are correct but does not have permission to update this lookup.

{
  "type": "LookupCreationError",
  "cid": "9a3eda1848d1",
  "code": 403,
  "context": "FORBIDDEN",
  "id": "xxxxxx5055-11ed-859a-7b524e50491a",
  "msg": "User is not authorized to perform operations in the domain."
}