Table of Contents |
---|
maxLevel | 2 |
---|
minLevel | 2 |
---|
type | flat |
---|
|
Overview
Check the list of available endpoints and methods in the Lookups API:
...
Endpoints and methods
...
Description
...
/lookup/{domain}
see below...
Display information on the lookups existing on a given domain.
...
/lookup/{domain}/{name}
see below...
Return information of a specific lookup.
...
/lookup/{domain}/{name}/job
see below...
Query the job UUIDs of a specific lookup.
...
/lookup/{domain}/{name}/job/{id}
see below...
Query the statuses of a lookup job.
...
lookup/{domain}/{name}/deploy-config
see below...
Create a new lookup.
...
lookup/{domain}/{name}/deploy-config
see below...
Update a specific lookup.
...
/lookup/{domain}/{name}
see below...
Send a request to delete a specific lookup.
Endpoints and methods
...
Display information on the lookups existing on a given domain.
...
Path parameters
Add the following path parameters as part of the endpoint:
...
Parameter
...
Type
...
Description
...
domain
required
...
string
...
Enter the name of the domain that contains the lookups you want to list.
Query string parameters
Query string parameters are optionally added after the path parameters, preceded by a question mark (?
) and separated by an ampersand (&
)
...
Parameter
...
Type
...
Description
...
max_length
...
integer
...
Maximum number of values to return. The default value is 100
. The maximum value is 1000
and the minimum value is 1
.
Lookups are shown from oldest to newest.
...
pageToken
...
integer (int32)
...
Use this parameter to consider a previously requested list on your next request. This should be set to the value shown in the nextPageToken
parameter included in the response you want to consider, if successful.
For example, you may perform a request and ask for information about the oldest 5 lookups in your domain. Then, you can perform a second request including the value in the nextPageToken
parameter returned with the previous response and requesting only 2 lookups. This will return the 2 lookups created after the first 5 previously requested.
The default and minimum value is 0
.
...
owner
...
string
You can filter the lookups in the response based on their domain owner, that is, the domain where the lookup was created. Supported values are:
THIS_DOMAIN
- Returns only lookups created in the domain indicated in the request URL.
OTHER_DOMAINS
- Returns only lookups owned by other domains but visible by the domain indicated in the request URL. This might be the case of domains that belong to a multitenant structure and have access to lookups defined in other domains that belong to that structure. You can decide the visibility level of a new lookup upon its creation.
ANY_DOMAIN
- Returns both lookups owned by the domain indicated in the request URL and those visible by them in other domains.
The default value is THIS_DOMAIN
...
Lookup ownership VS visibility
...
Lookup ownership refers to the domain where the lookup is defined.
...
Example
Find below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article.
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?max_length=10 |
And this request will return information about all the lookups owned by the user domain and also the ones visible to them in other domains:
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?owner=ANY_DOMAIN |
...
Code
...
Description
...
200
...
Successful response. The response includes the list of lookups in the given domain with their details.
Code Block |
---|
{
"type": "LookupMetaInfoListResponse",
"cid": "72078e04ee84",
"code": 200,
"context": null,
"id": "xxxxxxxxxx-0f74-11ee-b13b-4fc634871e5f",
"msg": "tutorial lookups.",
"lookups": [
{
"name": "Lookup_test_t",
"domain": "tutorial",
"fileSize": 12288,
"numEntries": 136,
"creationDate": "2023-03-15T13:45:57.63",
"keyType": {
"type": "first",
"columns": null,
"hasher": null
},
"deployConfig": null,
"lastStatus": null,
"fields": [
{
"column": "domain",
"type": "str",
"key": true
},
{
"column": "CDNProvider",
"type": "str",
"key": false
}
],
"owner": "user@devo.com"
},
{
"name": "CDN_Providers",
"domain": "tutorial",
"fileSize": 12288,
"numEntries": 136,
"creationDate": "2023-03-15T13:45:57.806",
"keyType": {
"type": "first",
"columns": null,
"hasher": null
},
"deployConfig": null,
"lastStatus": null,
"fields": [
{
"column": "domain",
"type": "str",
"key": true
},
{
"column": "CDNProvider",
"type": "str",
"key": false
}
],
"shared": false,
"owner": "user@devo.com"
},
{
"name": "test_101",
"domain": "tutorial",
"fileSize": 40960,
"numEntries": 307,
"creationDate": "2023-03-15T13:45:58.338",
"keyType": {
"type": "first",
"columns": null,
"hasher": null
},
"deployConfig": null,
"lastStatus": null,
"fields": [
{
"column": "alertName",
"type": "str",
"key": true
},
{
"column": "alertType",
"type": "str",
"key": false
},
{
"column": "alertMitreTactics",
"type": "str",
"key": false
},
{
"column": "alertMitreTechniques",
"type": "str",
"key": false
},
{
"column": "alertPriority",
"type": "int4",
"key": false
}
],
"owner": "user@devo.com"
},
{
"name": "d14022023api",
"domain": "tutorial",
"fileSize": 8192,
"numEntries": 1,
"creationDate": "2023-03-15T13:46:58.05",
"keyType": {
"type": "first",
"columns": null,
"hasher": null
},
"deployConfig": null,
"lastStatus": null,
"fields": [
{
"column": "key",
"type": "int4",
"key": true
},
{
"column": "fbool",
"type": "bool",
"key": false
}
],
"owner": null
}
],
"nextPageToken": -1
} |
...
400
...
Unsuccessful response. Bad request.
Code Block |
---|
{
"type": "LookupMetaInfoResponseError",
"cid": "046ead1c8323",
"code": 400,
"context": null,
"id": "xxxxxx-4fbb-11ed-859a-f19049cfd717"
} |
...
401
...
Unsuccessful response. The user is unauthorized to list the domain's lookups.
Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
...
404
...
Unsuccessful response. Domain not found.
Code Block |
---|
{
"type": "LookupMetaInfoResponseError",
"cid": "e33f677a6981",
"code": 404,
"context": null,
"id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
"msg": "Not Found"
} |
...
Return information of a specific lookup.
...
Table of Contents |
---|
maxLevel | 2 |
---|
minLevel | 2 |
---|
type | flat |
---|
|
Overview
Check the list of available endpoints and methods in the Lookups API:
Endpoints and methods | Description |
---|
/lookup/{domain} see below | Display information on the lookups existing on a given domain. |
/lookup/{domain}/{name} see below | Return information of a specific lookup. |
/lookup/{domain}/{name}/job see below | Query the job UUIDs of a specific lookup. |
/lookup/{domain}/{name}/job/{id} see below | Query the statuses of a lookup job. |
lookup/{domain}/{name}/deploy-config see below | Create a new lookup (advanced method). |
lookup/{domain}/{name}/deploy-csv see below | Create a new lookup using a CSV. |
lookup/{domain}/{name}/deploy-static-query see below | Create a new lookup based on a static query. |
lookup/{domain}/{name}/deploy-periodic-query see below | Create a new lookup based on a periodic query. |
lookup/{domain}/{name}/deploy-config see below | Update a specific lookup. |
lookup/{domain}/{name}/deploy-csv see below | Update a CSV lookup. |
lookup/{domain}/{name}/deploy-static-query see below | Update a lookup based on a static query. |
lookup/{domain}/{name}/deploy-periodic-query see below | Update a lookup based on a periodic query. |
/lookup/{domain}/{name} see below | Send a request to delete a specific lookup. |
Endpoints and methods
GET /lookup/{domain}
Display information on the lookups existing on a given domain.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain that contains the | lookup retrievename requiredQuery string parametersQuery string | Enter the name of the lookup you want to get information about. | Example
Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup |
Code | Description |
---|
200 | Successful response. The response includes information about the requested lookup.
Code Block |
{
"type": "LookupMetaInfoResponse",
"cid": "3c9bb672512c",
"code": 200,
"context": null,
"id": "xxxxxxxxx-0f75-11ee-b13b-636d49ade562",
"msg": "tutorial/test_101 meta information.",
"lookupMetaInfo": {
"name": "test_101",
"domain": "tutorial",
"fileSize": 40960,
"numEntries": 307,
"creationDate": "2023-03-15T13:45:58.14",
"keyType": parameters are optionally added after the path parameters, preceded by a question mark (?
) and separated by an ampersand (&
)
Parameter | Type | Description |
---|
max_length
| integer
| Maximum number of values to return. The default value is 100 . The maximum value is 1000 and the minimum value is 1 . Lookups are shown from oldest to newest. |
pageToken
| integer
| Use this parameter to consider a previously requested list on your next request. This should be set to the value shown in the nextPageToken parameter included in the response you want to consider, if successful. For example, you may perform a request and ask for information about the oldest 5 lookups in your domain. Then, you can perform a second request including the value in the nextPageToken parameter returned with the previous response and requesting only 2 lookups. This will return the 2 lookups created after the first 5 previously requested. The default and minimum value is 0 . |
owner
| string
| You can filter the lookups in the response based on their domain owner, that is, the domain where the lookup was created. Supported values are: THIS_DOMAIN - Returns only lookups created in the domain indicated in the request URL.
OTHER_DOMAINS - Returns only lookups owned by other domains but visible by the domain indicated in the request URL. This might be the case of domains that belong to a multitenant structure and have access to lookups defined in other domains that belong to that structure. You can decide the visibility level of a new lookup upon its creation.
ANY_DOMAIN - Returns both lookups owned by the domain indicated in the request URL and those visible by them in other domains.
The default value is THIS_DOMAIN Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can decide to make a lookup visible in all the domains within the structure when you create it using POST/lookup/{domain}/{name}/deploy-config . See how to do it here.
|
|
Example
Find below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article.
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?max_length=10 |
And this request will return information about all the lookups owned by the user domain and also the ones visible to them in other domains:
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain?owner=ANY_DOMAIN |
Code | Description |
---|
200 | Successful response. The response includes the list of lookups in the given domain with their details. Code Block |
---|
{
"type": "LookupMetaInfoListResponse",
"cid": "72078e04ee84",
"code": 200,
"context": null,
"id": "xxxxxxxxxx-0f74-11ee-b13b-4fc634871e5f",
"msg": "tutorial lookups.",
"lookups": [
{
" |
|
typefirstcolumnsnullhashernull },deployConfignulllastStatusnull"2023-03-15T13:45:57.63",
" |
|
fields [
columnalertNametype"str"keytrue
{ column"alertType" type"str"
keyfalse}, { alertMitreTacticsdomain",
"type": "str",
"key": true
|
|
falsealertMitreTechniquesCDNProvider",
"type": "str",
"key": false
|
|
},{columnalertPriority"typeint4CDN_Providers",
"domain": |
|
key": falsetutorial",
"fileSize": 12288,
|
|
}]owneruser@devo.com"2023-03-15T13:45:57.806",
|
|
}}400 | Unsuccessful response. Bad request.
Code Block |
{type"LookupMetaInfoResponseError",cid4b6cdd188a82"code": 400,context"id":"xxxxxx-5052-11ed-859a-910aa97972a3",msg"Unsuccessfulresponse.BadRequest."
}401 | Unsuccessful response. The user is unauthorized to get information about the lookup.
Code Block |
{error{code401message "Unauthorized"}}404 | Unsuccessful response. Lookup not found.
Code Block |
{typeLookupMetaInfoResponseError"cide33f677a6981code404,"context":null,"id": "xxxxxx-504c-11ed-859a-99d2d0713c21""msg": "Not Found"
}
...
Query the job UUIDs (Universally Unique IDentifier) of a specific lookup. A new job is created for each create, update or delete process that is initiated. Each job is identified by a unique ID, or UUID.
You will get the last 10 UUIDs for the requested lookup. The list is ordered chronologically being the first job UUID the oldest and the last one the newest.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: |
Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain that contains the required lookup. |
name required
| string
| Enter the name of the required lookup. |
Example
Find below a request example in cURL language. This request will return information about the last jobs generated by the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job |
Code | Description |
---|
200 | Successful response. Returns the job UUIDs of the given lookup.
Code Block |
{
"cid": "e47f4ab72ded",
"code": 200,
"context {
"column": "CDNProvider",
"type": "str",
"key": false
}
],
"shared": false,
"owner": "user@devo.com"
},
{
"name": "test_101",
"domain": "tutorial",
"fileSize": 40960,
"numEntries": 307,
"creationDate": "2023-03-15T13:45:58.338",
"keyType": {
"type": "first",
"columns": null,
"hasher": null
},
"deployConfig": null,
|
|
"id":"xxxxxxxx-e37c-11ed-b5ea-0242ac120002",msg"Lookup job uuids",jobs "xxxxxxx-c9a2-489c-8794-ea656a19b822","xxxxxxx-9714-48a7-9976-73e41523edfd",xxxxxxx-48a8-46ea-ab22-e0a5458e302b"xxxxxxx-ad7e-4fe6-bb43-89f93e629d76"]}400 | Unsuccessful response. Bad request.
Code Block |
{ LookupMetaInfoResponseError"cid": "6cde7fd6a1f4",code400,"context":null,"id": "xxxxxxx-53a2-11ed-859a-33f9a0bb845a""msg": "Unsuccessful response. Bad Request."
}401 | Unsuccessful response. The user is unauthorized to access the lookup
Code Block |
"error": {code401,messageUnauthorized}
}
...
Query the statuses of a lookup job given its UUID. You can get a job UUID using the request explained above.
You will get the last 50 statuses for the requested lookup job. The list is ordered chronologically being the first status the oldest and the last one the newest.
These are the possible status codes you may get in your response:
...
Code
...
Status
...
create.start
...
Lookup creation started.
...
create.ok
...
Lookup successfully created.
...
create.err
...
Error creating lookup.
...
push.start
...
Lookup synchronous upload started.
...
push.ok
...
Successful lookup synchronous upload.
...
push.err
...
Lookup synchronous upload failed.
...
deploy.start
...
Lookup asynchronous upload started.
...
deploy.ok
...
Successful lookup asynchronous upload started.
...
deploy.err
...
Lookup asynchronous upload failed.
...
del.start
...
Lookup deletion started.
...
del.ok
...
Lookup successfully deleted.
...
del.err
...
Error deleting lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: |
Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain that contains the required lookup. |
name required
| string
| Enter the name of the required lookup. |
id required
| string
| Enter the UUID of the required lookup job. You can get a job UUID using the request explained above, under the jobs parameter in the response. |
Example
Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job/123456 |
Code | Description |
---|
200 | Successful response. Returns the job statuses.
Code Block |
{
"cid": "e47f4ab72ded",
"code": 200,
"context": null,
"id "key": false
},
{
"column": "alertMitreTactics",
"type": "str",
"key": false
},
{
"column": "alertMitreTechniques",
"type": "str",
"key": false
},
{
"column": "alertPriority",
"type": "int4",
"key": false
}
],
"owner": "user@devo.com"
},
{
"name": "d14022023api",
"domain": "tutorial",
"fileSize": 8192,
"numEntries": 1,
"creationDate": "2023-03-15T13:46:58.05",
"keyType": {
"type": "first",
"columns": null,
|
|
"msg":"Lookupjob'sstatuses",status[{eventdata"2021-09-29T10:18:10.805"domain"galactic_empire"lookup"ImperialIntranetActivity","msg":"Lookupsuccessfullycreated"codecreate.ok}{
eventdata"2021-09-29T10:18:12.472",domainImperialIntranetActivitylookuptest-schedulemsg "Lookup ready to be executed"code"deploy.ok"null
}
],
"nextPageToken": -1
} |
|
400 | Unsuccessful response. Bad request. |
LookupJobsErrorLookupMetaInfoResponseError",
"cid": " |
|
4d1eb85a908d046ead1c8323",
"code": 400,
"context": null,
"id": "xxxxxx- |
|
53a42d69d242a54f",
"msg": "Unsuccessful response. Bad Request."
}401 | Unsuccessful response. The user is unauthorized to access the lookup |
401 | Unsuccessful response. The user is unauthorized to list the domain's lookups. Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
|
404 | Unsuccessful response. Domain not found. |
errortype": "LookupMetaInfoResponseError",
|
|
{ "cid": "e33f677a6981",
"code": |
|
401messageUnauthorized"xxxxxx-504c-11ed-859a-99d2d0713c21",
|
|
}
...
...
...
GET /lookup/{domain}/{name}
...
Return information of a specific lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain | where that contains the lookup you want to | create the new lookupretrieve. | name required
| string
| Enter the name of the | new lookup.Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Anchor |
---|
| id | id | id
- Lookup ID object.
Parameter | Type | Description |
---|
creator
| string
| Domain of the lookup. |
name
| string
| Name of the lookup. |
Example value:
Code Block |
"id": {
"creator": "selflookup you want to get information about. |
Example
Find below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article.
Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup |
Code | Description |
---|
200 | Successful response. The response includes information about the requested lookup. Code Block |
---|
{
"type": "LookupMetaInfoResponse",
"cid": "3c9bb672512c",
" |
|
nameForceSensitiveBeings
} Anchor |
---|
visibility | visibility | visibility
- (string)
Visibility level of the lookup. Allowed values are:
creator-only
- The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains
- The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
Anchor |
---|
recipe | recipe | recipe
required - Recipe of the lookup to be created.
Parameter | Type | Description |
---|
recipeType required
| string
| Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article. |
source required
| object
| Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field). Note |
---|
Check more about uploading a lookup using a CSV through S3, see this article. |
columns - (array ) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:
name required - (string ) Name of the column.
from - (integer ) 0-based index of the column in the CSV.
type - (string ) Data type of the column. Possible values are:
BOOLEAN
STRING
INT4
INT8
FLOAT4
FLOAT8
HEX4
HEX8
IP4
IP6
TIMESTAMP
DURATION
skipPreface - (string ) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
hasHeader - (boolean ) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
skipEmptyLines - (boolean ) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
fileProvider - (array ) Info to get the CSV file from S3.
bucketName required (string ) - Name of the S3 bucket where the CSV is located.
keyName required (string ) - Path of the CSV in the bucket.
transferOwnership required (boolean ) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.
accessKey - (string ) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
secretKey - (string ) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
host - (string ) - The S3 bucket host where the CSV is located.
port - (integer ) - The S3 bucket port where the CSV is located.
region - (string ) - The AWS region where the S3 bucket has been created.
query - (string ) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:
boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
|
lookupType
| object
| Indicate the lookup type. type required - (string ) Valid values are normal (regular lookups) and historic (time range lookups).
instantPolicy - (string ) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic . Possible values are:
natural - The associated source must be historical and the instant of each row will be used.
const - The instance of all rows will be the supplied constant value.
column - The instance will be extracted from the column with the given name. That column type must be timestamp .
instant - (number ) Only for historic lookups of type const .
columnName - (string ) Only for historic lookups of type column .
|
append
boolean
| If true , when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated. The default value is false . | key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
|
columnFilter
| array
| If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected. |
contribution
| object
| Defines how a row contributes to the final result, normally used on incremental lookups. type required - (string ) Type of the contribution policy. Values can be add , del or col . The default value is add .
name - (string ) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete .
|
secondaryIndexes
| object
| Columns of a lookup indexed as secondary indexes. type required - (string ) Type of the secondary indexes to be applied to the lookup. Values can be all , none or by-name . The default value is none .
map - Map of column names to be applied as secondary indexes when the type selected is by-name .
|
refreshMillis
| number
| Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic . |
startMillis
| number
| Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic . |
requiresDate
| boolean
| If true , the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now() . Can only be used if the recipe type chosen is periodic . The default value is false . |
Anchor |
---|
status | status | notifyStatus
- (boolean
) If true
, a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false
by default.
Check below an example of request body:
Code Block |
{
"id": {
"creator": "rebel_alliance",
"name": "TotallyNotFakeData"
},
"recipe": {
"recipeType": "once",
"source": {
"query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"key",
"IsDataFake",
"RebelsImprisoned",
"CreditsOnImperialBanks: null,
"id": "xxxxxxxxx-0f75-11ee-b13b-636d49ade562",
"msg": "tutorial/test_101 meta information.",
"lookupMetaInfo": {
"name": "test_101",
"domain": "tutorial",
"fileSize": 40960,
"numEntries": 307,
"creationDate": "2023-03-15T13:45:58.14",
"keyType": {
"type": "first",
"columns": null,
"hasher": null
},
"deployConfig": null,
"lastStatus": null,
"fields": [
{
"column": "alertName",
"type": "str",
"key": true
},
{
"column": "alertType",
"type": "str",
"key": false
},
{
"column": "alertMitreTactics",
"type": "str",
"key": false
},
{
"column": "alertMitreTechniques",
"type": "str",
"key": false
},
{
"column": "alertPriority",
"type": "int4",
"key": false
}
],
"owner": "user@devo.com"
}
} |
|
400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupMetaInfoResponseError",
"cid": "4b6cdd188a82",
"code": 400,
"context": null,
"id": "xxxxxx-5052-11ed-859a-910aa97972a3",
"msg": "Unsuccessful response. Bad Request."
} |
|
401 | Unsuccessful response. The user is unauthorized to get information about the lookup. Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
|
404 | Unsuccessful response. Lookup not found. Code Block |
---|
{
"type": "LookupMetaInfoResponseError",
"cid": "e33f677a6981",
"code": 404,
"context": null,
"id": "xxxxxx-504c-11ed-859a-99d2d0713c21",
"msg": "Not Found"
} |
|
GET /lookup/{domain}/{name}/job
Query the job UUIDs (Universally Unique IDentifier) of a specific lookup. A new job is created for each create, update or delete process that is initiated. Each job is identified by a unique ID, or UUID.
You will get the last 10 UUIDs for the requested lookup. The list is ordered chronologically being the first job UUID the oldest and the last one the newest.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain that contains the required lookup. | name required
| string
| Enter the name of the required lookup. |
ExampleFind below a request example in cURL language. This request will return information about the last jobs generated by the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article. Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job |
Code | Description |
---|
200 | Successful response. Returns the job UUIDs of the given lookup. Code Block |
---|
{
"cid": "e47f4ab72ded",
"code": 200,
"context": null,
"id": "xxxxxxxx-e37c-11ed-b5ea-0242ac120002",
"msg": "Lookup job uuids",
"jobs": [
"xxxxxxx-c9a2-489c-8794-ea656a19b822",
"xxxxxxx-9714-48a7-9976-73e41523edfd",
"xxxxxxx-48a8-46ea-ab22-e0a5458e302b",
"xxxxxxx-ad7e-4fe6-bb43-89f93e629d76"
]
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupMetaInfoResponseError",
"cid": "6cde7fd6a1f4",
"code": 400,
"context": null,
"id": "xxxxxxx-53a2-11ed-859a-33f9a0bb845a",
"msg": "Unsuccessful response. Bad Request."
} |
| 401 | Unsuccessful response. The user is unauthorized to access the lookup Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
|
|
GET /lookup/{domain}/{name}/job/{id}
Query the statuses of a lookup job given its UUID. You can get a job UUID using the request explained above.
You will get the last 50 statuses for the requested lookup job. The list is ordered chronologically being the first status the oldest and the last one the newest.
These are the possible status codes you may get in your response:
Code | Status |
---|
create.start
| Lookup creation started. |
create.ok
| Lookup successfully created. |
create.err
| Error creating lookup. |
push.start
| Lookup synchronous upload started. |
push.ok
| Successful lookup synchronous upload. |
push.err
| Lookup synchronous upload failed. |
deploy.start
| Lookup asynchronous upload started. |
deploy.ok
| Successful lookup asynchronous upload started. |
deploy.err
| Lookup asynchronous upload failed. |
del.start
| Lookup deletion started. |
del.ok
| Lookup successfully deleted. |
del.err
| Error deleting lookup. |
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain that contains the required lookup. | name required
| string
| Enter the name of the required lookup. | id required
| string
| Enter the UUID of the required lookup job. You can get a job UUID using the request explained above, under the jobs parameter in the response. |
ExampleFind below a request example in cURL language. This request will return information about the lookup called myLookup created in the domain indicated. Learn how to authorize your request in this article. Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X GET https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup/job/123456 |
Code | Description |
---|
200 | Successful response. Returns the job statuses. Code Block |
---|
{
"cid": "e47f4ab72ded",
"code": 200,
"context": null,
"id": null,
"msg": "Lookup job's statuses",
"status": [
{
"eventdata": "2021-09-29T10:18:10.805",
"domain": "galactic_empire",
"lookup": "ImperialIntranetActivity",
" |
|
Hex4Emperor, "Hex8Vader",EmperorClones, "Pi""EmperorIP4", "EmperorIP6"2021-09-29T10:18:12.472",
" |
|
TimeAfterBattleOfYavindomain": "ImperialIntranetActivity",
|
|
], contribution{typeadd"Lookup ready to be executed"
|
|
},requiresDate falseCode | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the creation |
Update a specific lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: |
Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain where that contains the lookup you want to update. |
name required
| string
| Enter the name of the lookup you want to update. |
Request body
The request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:
Anchor |
---|
id | id | id
- Lookup ID object.
Parameter | Type | Description |
---|
creator
| string
| Domain of the lookup. |
name
| string
| Name of the lookup. |
Example value:
Code Block |
---|
"id": {
"creator": "self",
"name": "ForceSensitiveBeings"
} |
Anchor |
---|
visibility | visibility | visibility
- (string)
Visibility level of the lookup. Allowed values are:
creator-only
- The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains
- The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain.. Only multitenant Admin users will be able to use this value.
Anchor |
---|
recipe | recipe | recipe
required - Recipe of the lookup to be updated.
Parameter | Type | Description |
---|
recipeType required
| string
| Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article. |
source required
| object
| Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field). columns - (array ) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:
name required - (string ) Name of the column.
from - (integer ) 0-based index of the column in the CSV.
type - (string ) Data type of the column. Possible values are:
BOOLEAN
STRING
INT4
INT8
FLOAT4
FLOAT8
HEX4
HEX8
IP4
IP6
TIMESTAMP
DURATION
skipPreface - (string ) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
hasHeader - (boolean ) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
skipEmptyLines - (boolean ) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
fileProvider - (array ) Info to get the CSV file from S3.
bucketName required (string ) - Name of the S3 bucket where the CSV is located.
keyName required (string ) - Path of the CSV in the bucket.
transferOwnership required (boolean ) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.
accessKey - (string ) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
secretKey - (string ) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
host - (string ) - The S3 bucket host where the CSV is located.
port - (integer ) - The S3 bucket port where the CSV is located.
region - (string ) - The AWS region where the S3 bucket has been created.
query - (string ) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:
boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
|
lookupType
| object
| Indicate the lookup type. type required - (string ) Valid values are normal (regular lookups) and historic (time range lookups).
instantPolicy - (string ) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic . Possible values are:
natural - The associated source must be historical and the instant of each row will be used.
const - The instance of all rows will be the supplied constant value.
column - The instance will be extracted from the column with the given name. That column type must be timestamp .
instant - (number ) Only for historic lookups of type const .
columnName - (string ) Only for historic lookups of type column .
|
append
boolean
| If true , when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated. The default value is false . | key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
|
columnFilter
| array
| If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected. |
contribution
| object
| Defines how a row contributes to the final result, normally used on incremental lookups. type required - (string ) Type of the contribution policy. Values can be add , del or col . The default value is add .
name - (string ) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete .
|
secondaryIndexes
| object
| Columns of a lookup indexed as secondary indexes. type required - (string ) Type of the secondary indexes to be applied to the lookup. Values can be all , none or by-name . The default value is none .
map - Map of column names to be applied as secondary indexes when the type selected is by-name .
|
refreshMillis
| number
| Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic . |
startMillis
| number
| Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic . |
requiresDate
| boolean
| If true , the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now() . Can only be used if the recipe type chosen is periodic . The default value is false . |
Anchor |
---|
status | status | notifyStatus
- (boolean
) If true
, a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false
by default.
Check below an example of request body:
Code Block |
{
"id": {
"creator": "rebel_alliance",
"name": "TotallyNotFakeData"
},
"recipe": {
"recipeType": "once",
"source": {
"query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"key",
"IsDataFake",
"RebelsImprisoned",
"CreditsOnImperialBanks",
"Hex4Emperor",
"Hex8Vader",
"EmperorClones",
"Pi",
"EmperorIP4",
"EmperorIP6",
"TimeAfterBattleOfYavin"
],
"contribution": {
"type": "add"
},
"requiresDate": false
}400 | Unsuccessful response. Bad request. |
LookupCreationResponseLookupJobsError",
"cid": " |
|
d41c91a21d56201400,
"context": null,
"id": "xxxxxx- |
|
220111ecb04a53c6289921cbLookupsenttocreation,
"lookupDeployConfig": |
401 | Unsuccessful response. The user is unauthorized to access the lookup |
id
creator"rebel_alliance",
nameGalacticEmpireActivity },
"visibility": "creator-only",
"recipe": {
"type": "once",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"eventdate",
"level",
"domain",
"userid",
"sessionid",
"correlationId"
],
"contribution": {
"type": "add"
}
}
}
}400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": "BAD_REQUEST",
"id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
"message": "Bad Request"
} |
|
401 | Unsuccessful response. The user is unauthorized to create a lookup. Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
|
403 | Unsuccessful response. User credentials are correct but does not have permission to create this lookup. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "9a3eda1848d1",
"code": 403,
"context": "FORBIDDEN",
"id": "xxxxxx5055-11ed-859a-7b524e50491a",
"msg": "User is not authorized to perform operations in the domain."
} |
|
...
POST/lookup/{domain}/{name}/deploy-config
Create a new lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain where you want to create the new lookup. | name required
| string
| Enter the name of the new lookup. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:
id - Lookup ID object.
Parameter | Type | Description |
---|
creator
| string
| Domain of the lookup. | name
| string
| Name of the lookup. |
Example value: Code Block |
---|
"id": {
"creator": "self",
"name": "ForceSensitiveBeings"
} |
visibility - (string) Visibility level of the lookup. Allowed values are:
creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
recipe required - Recipe of the lookup to be created.
Parameter | Type | Description |
---|
recipeType required
| string
| Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article. | source required
| object
| Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field). Note |
---|
Check more about uploading a lookup using a CSV through S3, see this article. |
columns - (array ) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:
name required - (string ) Name of the column.
from - (integer ) 0-based index of the column in the CSV.
type - (string ) Data type of the column. Possible values are:
BOOLEAN
STRING
INT4
INT8
FLOAT4
FLOAT8
HEX4
HEX8
IP4
IP6
TIMESTAMP
DURATION
skipPreface - (string ) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
hasHeader - (boolean ) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
skipEmptyLines - (boolean ) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
fileProvider - (array ) Info to get the CSV file from S3.
bucketName required (string ) - Name of the S3 bucket where the CSV is located.
keyName required (string ) - Path of the CSV in the bucket.
transferOwnership required (boolean ) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.
accessKey - (string ) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
secretKey - (string ) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
host - (string ) - The S3 bucket host where the CSV is located.
port - (integer ) - The S3 bucket port where the CSV is located.
region - (string ) - The AWS region where the S3 bucket has been created.
query - (string ) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:
boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
| lookupType
| object
| Indicate the lookup type. type required - (string ) Valid values are normal (regular lookups) and historic (time range lookups).
instantPolicy - (string ) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic . Possible values are:
natural - The associated source must be historical and the instant of each row will be used.
const - The instance of all rows will be the supplied constant value.
column - The instance will be extracted from the column with the given name. That column type must be timestamp .
instant - (number ) Only for historic lookups of type const .
columnName - (string ) Only for historic lookups of type column .
| append
| boolean
| If true , when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated. The default value is false . | key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
| columnFilter
| array
| If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected. | contribution
| object
| Defines how a row contributes to the final result, normally used on incremental lookups. type required - (string ) Type of the contribution policy. Values can be add , del or col . The default value is add .
name - (string ) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete .
| secondaryIndexes
| object
| Columns of a lookup indexed as secondary indexes. type required - (string ) Type of the secondary indexes to be applied to the lookup. Values can be all , none or by-name . The default value is none .
map - Map of column names to be applied as secondary indexes when the type selected is by-name .
| refreshMillis
| number
| Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic . | startMillis
| number
| Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic . | requiresDate
| boolean
| If true , the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now() . Can only be used if the recipe type chosen is periodic . The default value is false . |
notifyStatus - (boolean ) If true , a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false by default.
Check below an example of request body: Code Block |
---|
{
"id": {
"creator": "rebel_alliance",
"name": "TotallyNotFakeData"
},
"recipe": {
"recipeType": "once",
"source": {
"query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"key",
"IsDataFake",
"RebelsImprisoned",
"CreditsOnImperialBanks",
"Hex4Emperor",
"Hex8Vader",
"EmperorClones",
"Pi",
"EmperorIP4",
"EmperorIP6",
"TimeAfterBattleOfYavin"
],
"contribution": {
"type": "add"
},
"requiresDate": false
}
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the creation request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": "d41c91a21d56",
"code": 201,
"context": null,
"id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
"msg": "Lookup sent to creation",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": "creator-only",
"recipe": {
"type": "once",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"eventdate",
"level",
"domain",
"userid",
"sessionid",
"correlationId"
],
"contribution": {
"type": "add"
}
}
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": "BAD_REQUEST",
"id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
"message": "Bad Request"
} |
| 401 | Unsuccessful response. The user is unauthorized to create a lookup. Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
| 403 | Unsuccessful response. User credentials are correct but does not have permission to create this lookup. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "9a3eda1848d1",
"code": 403,
"context": "FORBIDDEN",
"id": "xxxxxx5055-11ed-859a-7b524e50491a",
"msg": "User is not authorized to perform operations in the domain."
} |
|
|
POST/lookup/{domain}/{name}/deploy-csv
Simplified endpoint to create a new lookup from a CSV file.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain where you want to create the lookup. | name required
| string
| Enter the name of the lookup you want to create. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Parameter | Type | Description |
---|
visibility
| string
| Visibility level of the lookup. Allowed values are: creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
| columns required
| array
| Array of lookup column descriptors. Each column object may contain the following parameters: name required - (string ) Name of the column.
from - (integer ) 0-based index of the column in the CSV.
type - (string ) Data type of the column. Possible values are:
BOOLEAN
STRING
INT4
INT8
FLOAT4
FLOAT8
HEX4
HEX8
IP4
IP6
TIMESTAMP
DURATION
| fileProvider required
| array
| Info to get the CSV file from S3. bucketName required (string ) - Name of the S3 bucket where the CSV is located.
keyName required (string ) - Path of the CSV in the bucket.
transferOwnership required (boolean ) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.
accessKey - (string ) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
secretKey - (string ) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
host - (string ) - The S3 bucket host where the CSV is located.
port - (integer ) - The S3 bucket port where the CSV is located.
region - (string ) - The AWS region where the S3 bucket has been created.
| key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
| skipPreface
| string
| Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty. | hasHeader
| boolean
| Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false . | skipEmptyLines
| boolean
| Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false . | contribution
| object
| Defines how a row contributes to the final result, normally used on incremental lookups. type required - (string ) Type of the contribution policy. Values can be add , del or col . The default value is add .
name - (string ) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete
|
Check below an example of request body: Code Block |
---|
{
"columns": [
{
"name": "ID",
"type": "INT8"
},
{
"name": "Location",
"type": "STRING"
},
{
"name": "Age",
"type": "FLOAT8"
},
{
"name": "Species",
"type": "STRING"
},
{
"name": "MidiclorianLevel",
"type": "FLOAT4"
}
],
"key": {
"type": "column",
"column": "ID"
},
"fileProvider": {
"bucketName": "holocrons-bucket",
"keyName": "secrets/data/force-sensitive-beings.csv",
"transferOwnership": true
}
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": "d5ce4eb105b2",
"code": 201,
"context": null,
"id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
"msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": {
"type": "creator-only"
},
"recipe": {
"recipeType": "periodic",
"source": {
"columns": [
{
"name": "ID",
"type": "INT8"
},
{
"name": "Location",
"type": "STRING"
},
{
"name": "Age",
"type": "FLOAT8"
},
{
"name": "Species",
"type": "STRING"
},
{
"name": "MidiclorianLevel",
"type": "FLOAT4"
}
],
"fileProvider": {
"bucketName": "holocrons-bucket",
"keyName": "secrets/data/force-sensitive-beings.csv",
"transferOwnership": true
},
"skipPreface": null,
"hasHeader": false,
"skipEmptyLines": false
},
"lookupType": {
"type": "normal"
},
"append": true,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": null,
"contribution": {
"type": "add"
},
"secondaryIndexes": {
"type": "none"
},
"refreshMillis": null,
"startMillis": null,
"requiresDate": true
},
"notifyStatus": true
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": null,
"id": "9f270aca-5151-11ed-859a-5d2974203ed5",
"msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"code": 401,
"msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
"cid": "29fda52318ae"
} |
| 403 | Unsuccessful response. Forbidden access. Code Block |
---|
{
"code": 5,
"msg": "Token invalid or expired",
"cid": "29fda52318ae"
} |
| 404 | Unsuccessful response. Domain not found. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 404,
"context": null,
"id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
"msg": "Domain not found."
} |
| 409 | Unsuccessful response. Lookup already exists. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 409,
"context": null,
"id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
"msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
} |
|
|
POST/lookup/{domain}/{name}/deploy-static-query
Simplified endpoint to create a new lookup based on a static query.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain where you want to create the lookup. | name required
| string
| Enter the name of the lookup you want to create. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Parameter | Type | Description |
---|
visibility
| string
| Visibility level of the lookup. Allowed values are: creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
| query required
| string
| Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are: boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
| key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
|
Check below an example of request body: Code Block |
---|
{
"visibility": {
"type": "creator-only"
},
"query": "select userid, domain from siem.logtrust.web.navigation where now()-1d < eventdate < now()",
"key": {
"type": "column",
"column": "userid"
}
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": "d5ce4eb105b2",
"code": 201,
"context": null,
"id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
"msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": {
"type": "creator-only"
},
"recipe": {
"recipeType": "once",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"eventdate",
"level",
"domain",
"userid",
"sessionid",
"correlationId"
],
"contribution": {
"type": "add"
},
"secondaryIndexes": {
"type": "none"
}
},
"notifyStatus": true
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": null,
"id": "9f270aca-5151-11ed-859a-5d2974203ed5",
"msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"code": 401,
"msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-static-query' requires valid auth",
"cid": "29fda52318ae"
} |
| 403 | Unsuccessful response. Forbidden access. Code Block |
---|
{
"code": 5,
"msg": "Token invalid or expired",
"cid": "29fda52318ae"
} |
| 404 | Unsuccessful response. Domain not found. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 404,
"context": null,
"id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
"msg": "Domain not found."
} |
| 409 | Unsuccessful response. Lookup already exists. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 409,
"context": null,
"id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
"msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
} |
|
|
POST/lookup/{domain}/{name}/deploy-periodic-query
Simplified endpoint to create a new lookup based on a periodic query.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain where you want to create the lookup. | name required
| string
| Enter the name of the lookup you want to create. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Parameter | Type | Description |
---|
visibility
| string
| Visibility level of the lookup. Allowed values are: creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queryable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
| query required
| string
| Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are: boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
| key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
| refreshPeriod
| string
| Refresh period of the lookup. Defaults to grouping period of the query in case it is a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo's duration type. The minimum value accepted is 1 minute. | startDate
| string
| Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch. | append
| boolean
| If true , when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated. The default value is false . |
Check below an example of request body: Code Block |
---|
{
"visibility": {
"type": "creator-only"
},
"query": "select userid, domain from siem.logtrust.web.navigation where now()-5m < eventdate < now()",
"key": {
"type": "column",
"column": "userid"
},
"refreshPeriod": "5m",
"startDate": "2023-12-12T00:00:00.000Z",
"append": true
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": "d5ce4eb105b2",
"code": 201,
"context": null,
"id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
"msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": {
"type": "creator-only"
},
"recipe": {
"recipeType": "periodic",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": true,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": null,
"contribution": {
"type": "add"
},
"secondaryIndexes": {
"type": "none"
},
"refreshMillis": 300000,
"startMillis": 1702339200000,
"requiresDate": true
},
"notifyStatus": true
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": null,
"id": "9f270aca-5151-11ed-859a-5d2974203ed5",
"msg": "refreshPeriod should be bigger than 1 minute"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"code": 401,
"msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-periodic-query' requires valid auth",
"cid": "29fda52318ae"
} |
| 403 | Unsuccessful response. Forbidden access. Code Block |
---|
{
"code": 5,
"msg": "Token invalid or expired",
"cid": "29fda52318ae"
} |
| 404 | Unsuccessful response. Domain not found. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 404,
"context": null,
"id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
"msg": "Domain not found."
} |
| 409 | Unsuccessful response. Lookup already exists. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 409,
"context": null,
"id": "9b6b0f0e-b162-11ee-b3ab-77fd619670ef",
"msg": "Lookup with domain galactic_empire and name destroyed_planets already exists"
} |
|
|
PUTlookup/{domain}/{name}/deploy-config
Update a specific lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain where that contains the lookup you want to update. | name required
| string
| Enter the name of the lookup you want to update. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details:
id - Lookup ID object.
Parameter | Type | Description |
---|
creator
| string
| Domain of the lookup. | name
| string
| Name of the lookup. |
Example value: Code Block |
---|
"id": {
"creator": "self",
"name": "ForceSensitiveBeings"
} |
visibility - (string) Visibility level of the lookup. Allowed values are:
creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain.. Only multitenant Admin users will be able to use this value.
recipe required - Recipe of the lookup to be updated.
Parameter | Type | Description |
---|
recipeType required
| string
| Type of the lookup recipe. Allowed values are once (static lookup) and periodic (dynamic lookup). Learn more about lookup types in this article. | source required
| object
| Source data of the lookup to be created. This object states if the lookup will be created using a query (using the query field), or through a CSV file stored in AWS S3 (using the fileProvider field). columns - (array ) Array of lookup column descriptors. Only used and required in CSV sources with fileProvider parameter informed and query parameter empty. Each column object may contain the following parameters:
name required - (string ) Name of the column.
from - (integer ) 0-based index of the column in the CSV.
type - (string ) Data type of the column. Possible values are:
BOOLEAN
STRING
INT4
INT8
FLOAT4
FLOAT8
HEX4
HEX8
IP4
IP6
TIMESTAMP
DURATION
skipPreface - (string ) Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
hasHeader - (boolean ) Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
skipEmptyLines - (boolean ) Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty.
fileProvider - (array ) Info to get the CSV file from S3.
bucketName required (string ) - Name of the S3 bucket where the CSV is located.
keyName required (string ) - Path of the CSV in the bucket.
transferOwnership required (boolean ) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.
accessKey - (string ) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
secretKey - (string ) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
host - (string ) - The S3 bucket host where the CSV is located.
port - (integer ) - The S3 bucket port where the CSV is located.
region - (string ) - The AWS region where the S3 bucket has been created.
query - (string ) Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are:
boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
| lookupType
| object
| Indicate the lookup type. type required - (string ) Valid values are normal (regular lookups) and historic (time range lookups).
instantPolicy - (string ) Defines how to calculate the instant of each row. Only used when the generated lookup type is historic . Possible values are:
natural - The associated source must be historical and the instant of each row will be used.
const - The instance of all rows will be the supplied constant value.
column - The instance will be extracted from the column with the given name. That column type must be timestamp .
instant - (number ) Only for historic lookups of type const .
columnName - (string ) Only for historic lookups of type column .
| append
| boolean
| If true , when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated. The default value is false . | key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
| columnFilter
| array
| If not null, a white list of columns will be projected. All elements of the list must be defined by the source. Columns whose name is not on this list won't be projected. | contribution
| object
| Defines how a row contributes to the final result, normally used on incremental lookups. type required - (string ) Type of the contribution policy. Values can be add , del or col . The default value is add .
name - (string ) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete .
| secondaryIndexes
| object
| Columns of a lookup indexed as secondary indexes. type required - (string ) Type of the secondary indexes to be applied to the lookup. Values can be all , none or by-name . The default value is none .
map - Map of column names to be applied as secondary indexes when the type selected is by-name .
| refreshMillis
| number
| Refresh time of the lookup. Can only be used and is required if the recipe type chosen is periodic . | startMillis
| number
| Millis since Epoch. Can only be used and is required if the recipe type chosen is periodic . | requiresDate
| boolean
| If true , the source query will be enriched with a closed date range restriction. The lower bound will be the maximum between startMillis and the last job instant, and the upper bound will be always now() . Can only be used if the recipe type chosen is periodic . The default value is false . |
notifyStatus - (boolean ) If true , a notification will be sent to the Devo app once the lookup is executed. If you do not include this parameter, it will be false by default.
Check below an example of request body: Code Block |
---|
{
"id": {
"creator": "rebel_alliance",
"name": "TotallyNotFakeData"
},
"recipe": {
"recipeType": "once",
"source": {
"query": "select 0 as key, false as IsDataFake, 2147483647 as RebelsImprisoned, 9223372036854775807 as CreditsOnImperialBanks, hex4('fffffff') as Hex4Emperor, hex8('fffffffffffffff') as Hex8Vader, 2.718281828459045 as EmperorClones, 3.141592653589793 as Pi, 87.219.9.157 as EmperorIP4, ip6('fe80::4492:bc4b:7a53:c0d5') as EmperorIP6, 0m as TimeAfterBattleOfYavin from siem.logtrust.web.navigation where now()-1m < eventdate < now() limit 1"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"key",
"IsDataFake",
"RebelsImprisoned",
"CreditsOnImperialBanks",
"Hex4Emperor",
"Hex8Vader",
"EmperorClones",
"Pi",
"EmperorIP4",
"EmperorIP6",
"TimeAfterBattleOfYavin"
],
"contribution": {
"type": "add"
},
"requiresDate": false
}
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": "d41c91a21d56",
"code": 201,
"context": null,
"id": "xxxxxx-2201-11ec-b04a-53c6289921cb",
"msg": "Lookup sent to creation",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": "creator-only",
"recipe": {
"type": "once",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"eventdate",
"level",
"domain",
"userid",
"sessionid",
"correlationId"
],
"contribution": {
"type": "add"
}
}
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": "BAD_REQUEST",
"id": "xxxxxx-5151-11ed-859a-5d2974203ed5",
"message": "Bad Request"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
| 403 | Unsuccessful response. User credentials are correct but does not have permission to update this lookup. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "9a3eda1848d1",
"code": 403,
"context": "FORBIDDEN",
"id": "xxxxxx5055-11ed-859a-7b524e50491a",
"msg": "User is not authorized to perform operations in the domain."
} |
|
|
PUTlookup/{domain}/{name}/deploy-csv
Update a CSV lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the domain name of the lookup you want to update. | name required
| string
| Enter the name of the lookup you want to update. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Parameter | Type | Description |
---|
visibility
| string
| Visibility level of the lookup. Allowed values are: creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
| columns required
| array
| Array of lookup column descriptors. Each column object may contain the following parameters: name required - (string ) Name of the column.
from - (integer ) 0-based index of the column in the CSV.
type - (string ) Data type of the column. Possible values are:
BOOLEAN
STRING
INT4
INT8
FLOAT4
FLOAT8
HEX4
HEX8
IP4
IP6
TIMESTAMP
DURATION
| fileProvider required
| array
| Info to get the CSV file from S3. bucketName required (string ) - Name of the S3 bucket where the CSV is located.
keyName required (string ) - Path of the CSV in the bucket.
transferOwnership required (boolean ) - Set this parameter to true if you want to transfer the CSV ownership to Devo and delete it from the bucket once the lookup is created.
accessKey - (string ) - The access key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
secretKey - (string ) - The secret key of a customer’s AWS user with permissions to access the bucket represented in the bucketName parameter.
host - (string ) - The S3 bucket host where the CSV is located.
port - (integer ) - The S3 bucket port where the CSV is located.
region - (string ) - The AWS region where the S3 bucket has been created.
| key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
| skipPreface
| string
| Enter a regular expression to ignore rows of data in a CSV used as source that follow the structure indicated in the regex. Only used in CSV sources with fileProvider parameter informed and query parameter empty. | hasHeader
| boolean
| Indicate if the source CSV file has a header row or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false . | skipEmptyLines
| boolean
| Indicate if you want to skip empty lines in the source CSV or not. Only used in CSV sources with fileProvider parameter informed and query parameter empty. Default value is false . | contribution
| object
| Defines how a row contributes to the final result, normally used on incremental lookups. type required - (string ) Type of the contribution policy. Values can be add , del or col . The default value is add .
name - (string ) The contribution will be extracted from the column with the given name. That column type must be string and its valid values are add and delete
|
Check below an example of request body: Code Block |
---|
{
"columns": [
{
"name": "ID",
"type": "INT8"
},
{
"name": "Location",
"type": "STRING"
},
{
"name": "Age",
"type": "FLOAT8"
},
{
"name": "Species",
"type": "STRING"
},
{
"name": "MidiclorianLevel",
"type": "FLOAT4"
}
],
"fileProvider": {
"bucketName": "holocrons-bucket",
"keyName": "secrets/data/force-sensitive-beings.csv",
"transferOwnership": true
}
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupUpdateResponse",
"cid": "d5ce4eb105b2",
"code": 201,
"context": null,
"id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
"msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": {
"type": "creator-only"
},
"recipe": {
"recipeType": "periodic",
"source": {
"columns": [
{
"name": "ID",
"type": "INT8"
},
{
"name": "Location",
"type": "STRING"
},
{
"name": "Age",
"type": "FLOAT8"
},
{
"name": "Species",
"type": "STRING"
},
{
"name": "MidiclorianLevel",
"type": "FLOAT4"
}
],
"fileProvider": {
"bucketName": "holocrons-bucket",
"keyName": "secrets/data/force-sensitive-beings.csv",
"transferOwnership": true
},
"skipPreface": null,
"hasHeader": false,
"skipEmptyLines": false
},
"lookupType": {
"type": "normal"
},
"append": true,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": null,
"contribution": {
"type": "add"
},
"secondaryIndexes": {
"type": "none"
},
"refreshMillis": null,
"startMillis": null,
"requiresDate": true
},
"notifyStatus": true
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": null,
"id": "9f270aca-5151-11ed-859a-5d2974203ed5",
"msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"code": 401,
"msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-csv' requires valid auth",
"cid": "29fda52318ae"
} |
| 403 | Unsuccessful response. Forbidden access. Code Block |
---|
{
"code": 5,
"msg": "Token invalid or expired",
"cid": "29fda52318ae"
} |
| 404 | Unsuccessful response. Domain not found. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 404,
"context": null,
"id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
"msg": "Domain not found."
} |
|
|
PUTlookup/{domain}/{name}/deploy-static-query
Update a lookup based on a static query.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the domain name of the lookup you want to update. | name required
| string
| Enter the name of the lookup you want to update. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Parameter | Type | Description |
---|
visibility
| string
| Visibility level of the lookup. Allowed values are: creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queriable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
| query required
| string
| Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are: boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
| key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
|
Check below an example of request body: Code Block |
---|
{
"query": "select eventdate, domain, userid from siem.logtrust.web.navigation where now()-1m < eventdate < now()"
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": "d5ce4eb105b2",
"code": 201,
"context": null,
"id": "c6b1e939-a57c-11ee-b1a9-a124bba45b9b",
"msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": {
"type": "creator-only"
},
"recipe": {
"recipeType": "once",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": false,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": [
"eventdate",
"level",
"domain",
"userid",
"sessionid",
"correlationId"
],
"contribution": {
"type": "add"
},
"secondaryIndexes": {
"type": "none"
}
},
"notifyStatus": true
}
} |
| 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context": null,
"id": "9f270aca-5151-11ed-859a-5d2974203ed5",
"msg": "KeyAlgorithm values can be 'first-column', 'column', 'row-hash', 'col-hash' and 'seq'"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"code": 401,
"msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-static-query' requires valid auth",
"cid": "29fda52318ae"
} |
| 403 | Unsuccessful response. Forbidden access. Code Block |
---|
{
"code": 5,
"msg": "Token invalid or expired",
"cid": "29fda52318ae"
} |
| 404 | Unsuccessful response. Domain not found. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "68e612d824a5",
"code": 404,
"context": null,
"id": "901c6154-b16b-11ee-b3ab-47fc28cf5888",
"msg": "Domain not found."
} |
|
|
PUTlookup/{domain}/{name}/deploy-periodic-query
Update a lookup based on a periodic query.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the domain name of the lookup you want to update. | name required
| string
| Enter the name of the lookup you want to update. |
Request bodyThe request JSON body must include the following objects and key-value pairs. Click them in the following list to see its details: Parameter | Type | Description |
---|
visibility
| string
| Visibility level of the lookup. Allowed values are: creator-only - The lookup will be visible only in the creator's domain. This is the default value.
all-subdomains - The lookup will be queryable in all the subdomains in a multitenant domain. It will only be visible in the lookup management web page in the root domain. Only multitenant Admin users will be able to use this value.
Info |
---|
Lookup ownership VS visibility Lookup ownership refers to the domain where the lookup is defined. Lookup visibility refers to the domain(s) where a lookup is visible. Note that lookups are created in a specific Devo domain, but they can be visible in other domains as well. If your domain belongs to a multitenant structure, you can use the all-subdomains parameter defined above to make it visible in all the domains within the structure.
|
| query required
| string
| Query to generate the lookup from. It must be written using LINQ syntax. Allowed field data types are: boolean
string
int4
int8
float4
float8
hex4
hex8
ip4
ip6
timestamp
duration
| key
| object
| In case the key of the lookup is not the name of a column, it can be computed through an algorithm using the elements on this data type. columns - (string ) List of columns to be used to generate the key for the lookup. This is only used when the type of the object is col-hash
column - (string ) Name of the column to be used as the key for the lookup. This is only used when the type of the object is column
type required - (string ) Type of key algorithm. Values can be:
first-column - The first column of the lookup will be used as key. This is the default value.
column - The column indicated in the column parameter will be used as key.
row-hash - A hash of the elements on the first row of the lookup will be used as key.
col-hash - A hash of the columns stated in the columns parameter will be used as key.
seq - The key of the lookup will be generated sequentially.
| refreshPeriod
| string
| Refresh period of the lookup. Defaults to grouping period of the query in case it is a grouping query, or 5 minutes if not. Accepted values are the ones accepted by Devo's duration type. The minimum value accepted is 1 minute. | startDate
| string
| Lookup creation start date. Defaults to the time of the request. Can either be an ISO-8601 date time string or a number of milliseconds from Epoch. | append
| boolean
| If true , when this recipe is evaluated, the content of its source will be appended to the lookup created on the previous evaluation. However, note that if there is no previous lookup or the recipe is updated (that is to say, the query is modified), this property will be ignored and the lookup will be recreated. The default value is false . |
Check below an example of request body: Code Block |
---|
{
"query": "select eventdate, domain, userid from siem.logtrust.web.navigation where now()-1m < eventdate < now()"
} |
Code | Description |
---|
201 | Successful response. Request submitted. The response includes the ID of the update request. Code Block |
---|
{
"type": "LookupCreationResponse",
"cid": " |
| d41c91a21d56d5ce4eb105b2",
"code": 201,
"context": null,
"id": " |
| xxxxxx220111ecb04a53c6289921cba124bba45b9b",
"msg": "Lookup sent to creation. You can check the creation status using the provided id: /lookup/{domain}/{name}/job/{id}",
"lookupDeployConfig": {
"id": {
"creator": "rebel_alliance",
"name": "GalacticEmpireActivity"
},
"visibility": {
"type": "creator-only"
},
"recipe": {
" |
| typeonceperiodic",
"source": {
"query": "select eventdate, level, domain, userid, sessionid, correlationId from siem.logtrust.web.activity where now()-1m < eventdate < now()"
},
"lookupType": {
"type": "normal"
},
"append": |
| falsetrue,
"key": {
"type": "column",
"column": "key"
},
"columnFilter": |
| [
eventdate,level, "domain" userid,sessionid,"correlationId"]contribution{ type"add"}} | 400 | Unsuccessful response. Bad request. Code Block |
---|
{
"type": "LookupCreationError",
"cid": "0cd289fa1b63",
"code": 400,
"context |
| ": "BAD_REQUEST",
"id": "xxxxxx-5151-11ed-859a-5d2974203ed5"": null,
"id": "9f270aca-5151-11ed-859a-5d2974203ed5",
"msg": "refreshPeriod should be bigger than 1 minute"
} |
| 401 | Unsuccessful response. The user is unauthorized to update the lookup. Code Block |
---|
{
"code": 401,
"msg": "Access to 'https://api.us.devo.com/lookup-api/lookup/galactic_empire/destroyed_planets/deploy-periodic-query' requires valid auth",
" |
| messageBad Request401403 | Unsuccessful response. | The user is unauthorized to update the lookuperror{ code401,messageUnauthorized
}403404 | Unsuccessful response. | User credentials are correct but does not have permission to update this lookupDomain not found. Code Block |
---|
{
"type": "LookupCreationError",
"cid": " |
| 9a3eda1848d1403"FORBIDDEN"xxxxxx505511ed859a7b524e50491aUser isauthorized to perform operations in the domain |
...
...
DELETE/lookup/{domain}/{name}
Send a request to delete a specific lookup.
Rw ui expands macro |
---|
Path parametersAdd the following path parameters as part of the endpoint: Parameter | Type | Description |
---|
domain required
| string
| Enter the name of the domain that contains the lookup you want to delete. | name required
| string
| Enter the name of the lookup you want to delete. |
ExampleFind below a request example in cURL language. This request will return information about the 10 oldest lookups created in the domain indicated. Learn how to authorize your request in this article. Code Block |
---|
curl -H "standAloneToken:YOUR_TOKEN" -X DELETE https://api-us.devo.com/lookup-api/lookup/myDomain/myLookup |
Code | Description |
---|
200 | Successful response. Deletion request sent. The response includes the ID of the request. Code Block |
---|
{
"type": "LookupDeletionResponse",
"cid": "f44f458f7c32",
"code": 200,
"context": null,
"id": "xxxxxx-5052-11ed-b24b-85c623a0cbd8",
"msg": "Lookup sent to deletion"
} |
| 401 | Unsuccessful response. The user is unauthorized to delete this lookup. Code Block |
---|
{
"error": {
"code": 401,
"message": "Unauthorized"
}
} |
|
|