Versions Compared

Version Old Version 1 New Version 2
Changes made by

Virginia Camuñas Núñez

Virginia Camuñas Núñez

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
maxLevel2
minLevel1
typeflat

...

Endpoints and methods

Anchor
path1
path1
POST /query

Use this endpoint tu run a query through the API.

Rw ui expands macro
Rw expand
titleRequest

Request body

The request JSON body must include an object with the following key-value pairs.

Parameter

Type

Description

query

string

This is the query that you want to run, expressed in LINQ. To find the query's LINQ script in the Devo app, open the query in the Data search area, then choose Query code editor from the toolbar.

The body of the request must contain either the query or the queryId parameter.

queryId

string

This is the ID of the query that you want to run. To find the query ID, open the query in the Data search area, then open the search window menu and select Current query → Get ID.

The body of the request must contain either the query or the queryId parameter.

from required

integer

The start date as a UTC timestamp in seconds. See the Relative dates section below to learn more about this parameter.

to

integer

The end date as a UTC timestamp in seconds. If this parameter is left out, the query will be continuous. See the Relative dates section below to learn more about this parameter.

mode

object

This object contains the type parameter (string) to specify the format of the response. If left out of the request body, the default response type json will be used.

The possible values are:

  • json

  • json/compact

  • json/simple

  • json/simple/compact

  • msgpack

  • csv

  • tsv

  • xls

These response formats are fully described later in this article. When you indicate a response format other than json, you must include the dateFormat and timeZone parameters.

destination

object

This object specifies where the response should be sent. If this object is left out of the request body, the response will be sent back to the request source.

It must include the following parameters:

  • type - (string) This is the type of system to which the response should be sent. The possible values are:

    • hdfs

    • s3

    • email

    • snmp

    • kafka

Depending on the type, additional parameters will be required. See the related HDFS, S3, email, SNMP and Kafka articles.

  • params - (string) Destination parameters, depend on the type selected. Check the HDFS, Kafka, S3, email and SNMP articles to see the parameters required for each destination.

dateFormat

string

This is only required when you specify a type other than json. The possible formats are:

  • default - yyyy-MM-dd' 'HH:mm:ss.SSS

  • sql - yyyy-MM-dd' 'HH:mm:ss.SSS

  • iso - yyyy-MM-dd'T'HH:mm:ss.SSSXX

timeZone

string

Change the timezone of the query, only for mode types other than json. This parameter supports any positive or negative GMT timezones, like GMT-2 or GMT+1

timeUnit

string

When from or to params are timestamps, you can use this parameter to specify the time unit. Only MILLISECONDS or SECONDS are valid values.

skip/offset

integer

You can use either the skip or offset parameters to skip the first X elements of the query.

limit

integer

Limit the results of the query. The query will stop after returning the first X elements of the query or reaching its end.

ipAsString

boolean

Set this parameter to true if you want to get IP addresses as dotted strings (for example, 94.2.23.1). If you don't add this parameter in your request or set it to false, IP addresses will be returned as numbers (for example, 1577195265).

vaultName

string

Query priority. Allowed values are:

  • low

  • normal

  • high

  • urgent

progressInfo

boolean

Set this parameter to true if you want to get progress info about the requested query. You will get an entry p with the timestamp of the event that is being processed at that moment. Progress info will be sent at most once every 5s. 

This is only available if you set type as json/simple/compact. See an example below.

allowPartialResults

boolean

Specifies whether partial results are allowed or not (true by default). If partial results are not allowed, the query will be interrupted every time some results are missing due to a failure.

keepAlive

object

Object with information about the keep alive message for CSV, TSV and XLS modes. If not provided, live queries aren't accepted for the mentioned modes. Note that this object doesn't apply to any json mode.

This object must contain the following parameters:

  • type - (string) Allowed values are:

    • empty - An empty event will be sent as a keep alive message.

    • token - A configurable token set in the parameter explained below will be sent as a keep alive message.

  • token - (string) Configurable token to be sent as a keep alive message for the token type. The default value is \n

Examples:

Code Block
"keepAlive": {
    "type":"empty"
}
Code Block
"keepAlive": {
    "type":"token", 
    "token":"myToken"
}

timeRangeFilter

object

Add this object to use time control in your query. Time control allows you to choose either the event date (time when Devo receives the data) or creation date (time when the events are generated in its source) as reference time when you query data.

Check this article to learn more about time control and how to use it in the Devo app.

This object may include the following parameters:

  • by - (string) Choose the reference time to be used in your query. Allowed values are eventdate and creationdate. Default value is eventdate.

  • allowedLateness - (string) This option can only be used if you enter creationdate in the by parameter above. Enter the maximum expected delay between the creation date and the ingestion date of the table queried. It admits duration type expressions (1h, 1d, etc). Default value is now.

Info

Setting a reception delay using the allowedLateness parameter allows the system to increase performance as it only looks for the corresponding events.

Example

Code Block
curl --location 'https://apiv2-us.devo.com/search/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 2e672c8182f67a3bc6cd7a6815864589' \
--data '{
 "query": "from siem.logtrust.web.activity",
 "limit": 30,
 "from": "1s",
 "keepAlive": {
    "type":"empty"                
  }
}'

The following are examples of responses:

Code Block
{
    "query": "from demo.ecommerce.data select *",
    "from": 1519645036,
    "to": 1519645136,
    "mode": {
        "type": "json"
    },
    "destination": {
      "type": "hdfs",
      "params": {
        "param1": "value1",
        "param1": "value2"
      }
    }
}
Code Block
{
    "from": 1519989362,
    "mode": {
        "type": "json/simple"
    },
    "query": "from my.synthesis.vec00.suricataalert group every 30s every 30s select *",
    "to": 1519989392
}
Code Block
{ 
 "query": "from demo.ecommerce.data select *",
 "limit": 10, 
 "from": 1528306922,
 "to": 1528306952,
 "mode": {
 "type": "tsv" 
 }
}

Anchor
relativedates
relativedates

Relative time-ranges

The Query API supports the same language for relative days as the one used in the Data Search section.

Let's suppose the current time (which we refer to as "now()") is Sunday, 05 February 2017, 13:37:05. The table below shows the resulting time when different expressions are applied.

Time expression

Description

Resulting time

now() - 60m

60 minutes ago

Sunday, 05 February 2017, 12:37:05

now() @ 1h

Now (rounded to the beginning of the hour)

Sunday, 05 February 2017, 13:00:00

now() - 24h

24 hours ago

Saturday, 04 February 2017, 13:37:05

(now() - 1d) @ 1d

Yesterday (rounded to the beginning of the day)

Saturday, 04 February 2017, 00:00:00

(now() - 2d) @ 1d

2 days ago (rounded to the beginning of the day)

Friday, 03 February 2017, 00:00:00

(now() - 2d) @ 1m

2 days ago (rounded to the beginning of the minute)

Friday, 03 February 2017, 13:37:00

((now() - 2d) @ 1d) - 2h

2 days ago (rounded to the beginning of the day minus 2 hours)

Thursday, 02 February 2017, 22:00:00

now() @ 1w

Locale week

Sunday, 05 February 2017, 00:00:00

now() @ 1W

ISO week

Monday, 30 January 2017, 00:00:00

now() ^ 6d

Replace the day with 6

Monday, 06 February 2017, 13:37:05

now() ^ 2018y3M6d15h30m20s

 

Tuesday, 06 March 2018, 15:30:20

 

...

 Relatives dates

Note

Deprecated

Relatives dates

A relative date range is a period of time that is relative to the current date (last week, last month, etc). You can add different operators to the from and to parameters of your query request to indicate specific time ranges. Note that the date you enter in the to parameter must always be greater than or equal to the from date.

Rw ui expands macro
Rw expand
titleDates

Operator

Description

today

Get the current day at 00:00:00. Note that the timeZone parameter affects the date settings.

  • "from": "today"
    This sets the starting date to 08-10-2018, 00:00:00 UTC

  • "to": "today"
    This sets the ending date to 08-10-2018, 00:00:00 UTC

  • "from": "today"
    "timeZone": "GMT+2"
    This sets the starting date to 08-10-2018, 00:00:00 GMT+2 (07-10-2018, 22:00:00 UTC)

  • "to": "today"
    "timeZone": "GMT+2"
    This sets the ending date to 08-10-2018, 00:00:00 GMT+2 (07-10-2018, 22:00:00 UTC)

now

Get the current day and time

  • "from": "now"
    This sets the starting date to 08-10-2018, 14:33:12 UTC

  • "to": "now"
    This sets the ending date to 08-10-2018, 14:33:12 UTC

endday

If you use this in the from field you will get the current day and the last second of the day. If you use it in the to field you will get the from date and the last second of that day. Note that the timeZone parameter affects the date settings.

  • "from": "endday"
    This sets the starting date to 08-10-2018, 23:59:59 UTC

  • "from": 1515500531 (this timestamp corresponds to 01/09/2018 12:22:11 UTC)
    "to": "endday"
    This sets the ending date to 01-09-2018, 23:59:59 UTC
    .

  • "from": "endday"
    "timeZone": "GMT+2"
    This sets the ending date to 08-10-2018, 23:59:59 GMT+2 (08-10-2018, 21:59:59 UTC)

  • "from": 1515493331 (this timestamp corresponds to 01/09/2018, 12:22:11 GMT+2)
    "to": "endday"
    "timeZone": "GMT+2"
    This sets the ending date to 01-09-2018 23:59:59 GMT+2 (01-09-2018, 21:59:59 UTC)

  • "from": 1515452400 (this timestamp corresponds to 01/09/2018, 01:00:00 GMT+2)
    "to": "endday"
    "timeZone": "GMT+2"
    This sets the ending date to 01-09-2018 23:59:59 GMT+2 (01-09-2018, 21:59:59 UTC)

endmonth

If you use this in the from field you will get the last day of the current month and the last second of that day. If you use it in the to field, you will get last day of the month indicated in the date field and the last second of that day. Note that the timeZone parameter affects the date settings.

  • "from": "endmonth"
    This sets the starting date to 31-10-2018, 23:59:59 UTC

  • "to": "endmonth"
    This sets the ending date to 30-09-2018, 23:59:59 UTC
    .

  • "from": 1536150131 (this timestamp corresponds to 05/09/2018, 12:22:11 UTC)
    "to": "endmonth"
    This sets the ending date to 30-09-2018, 23:59:59 UTC

  • "from": 1536142931 (this timestamp corresponds to 05/09/2018, 12:22:11 GMT+2)
    "to": "endmonth"
    "timeZone": "GMT+2"
    This sets the ending date to 30-09-2018 23:59:59 GMT+2 (30-09-2018, 21:59:59 UTC)

Rw expand
titleDays

Operator

Description

d

Enter a number followed by d in the from parameter to substract N days from the current date. If you use it in the to field you will get the from date plus the indicated number of days.

  • "from": "2d"
    This sets the starting date to 06-10-2018, 14:33:12 UTC

  • "from": 1536150131 (this timestamp corresponds to 05-09-2018, 12:22:11 UTC)
    "to": "2d"
    This sets the ending date to 07-09-2018, 12:22:11 UTC

  • "from": "5d"
    "to": "2d"
    This sets the starting date to 03-10-2018, 14:33:12 UTC and the ending date to 05-10-2018, 14:33:12 UTC

ad

Enter a number followed by ad in the from parameter to subtract N days from the current date and set time to 00:00:00. If you use it in the to field you will get the from date plus the indicated number of days and set time to 00:00:00. Note that the timeZone parameter affects the date settings.

  • "from": "2ad"
    This sets the starting date to 06-10-2018, 00:00:00 UTC

  • "from": 1536150131 (this timestamp corresponds to 05-09-2018, 12:22:11 UTC)

    "to": "2ad"
    This sets the ending date to 07-09-2018, 00:00:00 UTC

  • "from":"5ad"
    "to": "2ad"
    This sets the starting date to 03-10-2018, 00:00:00 UTC and the ending date to 05-10-2018, 00:00:00 UTC

  • "from": 1536142931 (this timestamp corresponds to 05/09/2018, 12:22:11 GMT+2)
    "to": "2ad"
    "timeZone": "GMT+2"
    This sets the ending date to 07-09-2018, 00:00:00 GMT+2 (06-09-2018, 22:00:00 UTC)

  • "from": "5ad"
    "to": "2ad"
    "timeZone": "GMT+2"
    This sets the starting date to 03-10-2018, 00:00:00 GMT+2 (02-10-2018, 22:00:00 UTC), and the ending date to 05-10-2018, 00:00:00 GMT+2 (04-10-2018, 22:00:00 UTC)

Rw expand
titleHours

Operator

Description

h

Enter a number followed by h in the from parameter to subtract N hours from the current time. If you use it in the to field you will get the from time plus the indicated number of hours.

  • "from": "2h"
    This sets the starting date to 08-10-2018, 12:33:12 UTC

  • "from": "16h"
    This sets the starting date to 07-10-2018, 22:33:12 UTC

  • "from": 1536150131 (this timestamp corresponds to 05/09/2018, 12:22:11 UTC)

    "to": "2h"
    This sets the ending date to 05-09-2018, 14:22:11 UTC

  • "from": "5h"

    "to": "2h"
    This sets the starting date to 08-10-2018, 09:33:12 UTC and the ending date to 08-10-2018, 11:33:12 UTC

ah

Enter a number followed by ah in the from parameter to subtract N hours from the current date at 00:00:00. If you use it in the to field you will add the indicated number of hours to the from date at 00:00:00. Note that the timeZone parameter affects the date settings.

  • "from": "2ah"
    This sets the starting date to 07-10-2018, 22:00:00 UTC

  • "from": "2ah"
    "timeZone": "GMT+2"
    This sets the starting date to 07-10-2018, 22:00:00 GMT+2 (07-10-2018, 20:00:00 UTC)

  • "from": 1536114131 (this timestamp corresponds to 05-09-2018, 02:22:11 UTC)
    "to": "12ah"
    This sets the ending date to 05-09-2018, 12:00:00 UTC

  • "from": 1536106931 (this timestamp corresponds to 05-09-2018, 02:22:11 GMT+2)
    "to": "12ah"
    "timeZone": "GMT+2"
    This sets the ending date to 05-09-2018, 12:00:00 GMT+2 (05-09-2018, 10:00:00 UTC)

    .

  • "from": "5ah"
    "to": "21ah"
    This sets the starting date to 07-10-2018, 19:00:00 UTC and the ending date to 07-10-2018, 21:00:00 UTC

...

Anchor
queryresponseformats
queryresponseformats

Query response formats

Responses to your queries can be either returned to the source of the request, forwarded to an HDFS, S3, SNMP, or Kafka type system, or sent via email.

...