Page Comparison

...

The Devo Wiz collector allows customers to retrieve Wiz cloud security issues into Devo to query, correlate, analyze, and visualize to enable Enterprise IT and Cybersecurity teams to take the most impactful decisions at the petabyte scale. The collector processes the Wiz API responses and sends them to the Devo platform, which then categorizes all data received on tables along rows and columns in your Devo domain.

Data sources

Data source	Description	API

endpoint

Endpoint	Collector service name	Devo table	Available from release
Issues	An issue in

Wiz

wiz is a vulnerability that is detected in the cloud infrastructure

/graphql

issues

cspm.wiz.issues.default

v1.0.0

Devo collector features

...

Feature

...

Details

...

Allow parallel downloading (multipod)

...

Not allowed

...

Running environments

...

Collector Server, On Premise

...

Populated Devo events

...

Table

...

Flattening preprocessing

...

Yes

Flattening preprocessing

...


Vulnerability	Vulnerabilities are weaknesses in computer systems that can be exploited by malicious attackers. Whether they are caused by bugs or design flaws, vulnerabilities can allow attackers to execute code in an environment or elevate privileges.	`/graphql`	`vulnerabilities`	`cspm.wiz.vulnerabilities.default`	`v1.5.0`
Audit Logs	The Audit Log records key events in Wiz, such as login, logout, and user update. The Audit Log is primarily used to investigate potentially suspicious activity or diagnose and troubleshoot errors.	`/graphql`	`auditLogs`	`cspm.wiz.audit.default`	`v1.5.0`
Cloud Configuration Findings	This returns the problems with configurations and the remediation solutions for the same.	`/graphql`	`cloudConfiguration`	`cspm.wiz.cloud_configuration.default`	`v1.5.0`
Custom Service	This provides an option to add custom graphql query in the config and ingest data.	`/graphql`	`custom_query`	`my.app.wiz.custom_query` (default) User can provide override tag in the config if the parser is deployed for their custom query or if they want a different table in my.app .	`v1.7.0`

Devo collector features

Feature	Details
Allow parallel downloading (`multipod`)	`not allowed`
Running environments	`collector server` `on-premise`
Populated Devo events	`table`
Flattening preprocessing	`yes`

Flattening preprocessing

In order to improve the data exploitation and enrichment, this collector applies some flattening actions to the collected data before delivering it to Devo:

Data source	Collector service	Optional	Flattening details
Issues	`issues`	`No`	The `control` key content is transferred to the first JSON level with the prefix `control_`. The `entity` key content is transferred to the first JSON level with the prefix `entity_`. The `entitySnapshot` key content is transferred to the first JSON level with the prefix `entitySnapshot_`.

...


Vulnerabilities	`vulnerabilities`	`Yes`	The `layer` key content is transferred to the first json level with the prefix `layer_`. The `vulnerable_asset` key content is transferred to the first json level with the prefix `asset_`.
Audit Logs	`auditLogs`	`Yes`	The `action_parameters` key content is transferred to the first json level with the prefix `action_`.
Cloud Configuration Findings	`cloudConfiguration`	`Yes`	The `resource` key content is transferred to the first json level with the prefix `resource_`.
Custom Service	`custom_query`	`Yes`	N/A

How to enable the collection in the vendor

...

Setting	Details
`override_api_base_url`	By default, the base URLis URLs `https://api.us1.app.wiz.io`. This parameter allows you to customize the base URL and is mandatory when the customer ULR URL is different than the given default value.
`client_id`	User Client ID to authenticate to the service.
`client_secret`	User Secret Key to authenticate to the service.

...

Rw ui tabs macro

Rw tab

title	Cloud collector

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

In the Collector Server GUI, access the domain in which you want this instance to be created
Click Add Collector and find the one you wish to add.
In the Version field, select the latest value.
In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).
In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.
In the Parameters section, establish the Collector Parameters as follows below:

Editing the JSON configuration

Code Block

{
  "global_overrides": {
    "debug": <debug_status>
  },
  "inputs": {
    "wiz_data_puller": {
      "id": "<short_unique_id>",
      "enabled": <input_status>,
      "override_api_base_url": <base_url>,
      "credentials": {
        "client_id": "<client_id>",
        "client_secret": "<client_secret>"
      },
      "services": {
        "issues": {
          "request_period_in_seconds": <request_period_in_seconds>,
          "historic_date_utc": <historic_date_utc>,
          "filters": {
            "type": <type_list>
          }
        },
        "vulnerabilities": {
  }     }   }
}

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Please replace the placeholders with real world values following the description table below:

Parameter

Data Type

Type

Value Range / Format

Details

debug_status

bool

Mandatory

false / true

If the value is true, the debug logging traces will be enabled when running the collector. If the value is false, only the info, warning and error logging levels will be printed.

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

input_status

bool

Mandatory

false / true

Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.

base_url

str

Optional

Valid URL following this regex:
^https:\/\/([a-z0-9]+[.]{1})([a-z0-9]+[.]{1})*[a-z]{2,}(:[0-9]{2,5})?$

By default, the base url is https://api.us1.app.wiz.io. This parameter allows you to customize the base url.

This parameter should be removed if it is not used.

historic_date_utc

str

Optional

UTC with format:
YYYY-mm-ddTHH:MM:SS.sssZ

This configuration allows you to set a custom date as the beginning of the period to download. This allows downloading historical data (1 month back for example) before downloading new events.

If this setting is not set, the default value is the current time.

This parameter should be removed if it is not used.

client_id

str

Mandatory

Any

User Client ID to authenticate to the service.

client_secret

str

Mandatory

Any

User Secret Key to authenticate to the service.

request_period_in_seconds

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (60 seconds)

This parameter should be removed if it is not used.

type_list

list

Optional

Possible values:

"TOXIC_COMBINATION"
"THREAT_DETECTION",
"CLOUD_CONFIGURATION"

Filter by Issue type. You can specify multiple values in an array.

Example 1:

Code Block
"type": ["THREAT_DETECTION"]

Example 2 (multiple values):

Code Block
"type": ["TOXIC_COMBINATION", "THREAT_DETECTION"]

This parameter should be removed if it is not used.

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

Code Block

<any_directory>
└── devo-collectors/
    └── <product_name>/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── state/
        └── config/ 
            └── config-<product_name>.yaml

Note
Replace `<product_name>` with the proper value.

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Image Removed

Replace <product_name> with the proper value.

Editing the config.yaml file

Code Block

globals:
  debug: <debug_status>
  id: <collector_id>
  name: <collector_name>
  persistence:
    type: filesystem
    config:
      directory_name: state
  multiprocessing: false
  queue_max_size_in_mb: 1024
  queue_max_size_in_messages: 1000
  queue_max_elapsed_time_in_sec: 60
  queue_wrap_max_size_in_messages: 100

outputs:
  devo_1:
    type: devo_platform
    config:
      address: <devo_address>
      port: 443
      type: SSL
      chain: <chain_filename>
      cert: <cert_filename>
      key: <key_filename>

inputs:
  wiz_data_puller:
    id: <short_unique_id>
    enabled: <input_status>
    override_api_base_url: <base_url>
    credentials:
      client_id: <client_id>
      client_secret: <client_secret>
    services:
      issues:
        request_period_in_seconds: <request_period_in_seconds>
        historic_date_utc: <historic_date_utc>
        filters:
          type: <type_list>

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Replace the placeholders with your required values following the description table below:

Parameter

Data type

Type

Value range

Details

debug_status

bool

Mandatory

false / true

If the value is true, the debug logging traces will be enabled when running the collector. If the value is false, only the info, warning and error logging levels will be printed.

collector_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this collector.

collector_name

str

Mandatory

Minimum length: 1
Maximum length: 10

Use this param to give a valid name to this collector.

devo_address

str

Mandatory

collector-us.devo.io
collector-eu.devo.io

Use this param to identify the Devo Cloud where the events will be sent.

chain_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the chain.cert file downloaded from your Devo domain. Usually this file's name is: chain.crt

cert_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the file.cert downloaded from your Devo domain.

key_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the file.key downloaded from your Devo domain.

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

input_status

bool

Mandatory

false / true

Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.

requests_per_seconds

int

Optional

Minimum value: 1

Customize the maximum number of API requests per second. If not used, the default setting will be used: 100000 requests/sec.

Info
This parameter can be left blank, removed or commented.

base_url

str

Optional

Valid URL following this regex:
^https:\/\/([a-z0-9]+[.]{1})([a-z0-9]+[.]{1})*[a-z]{2,}(:[0-9]{2,5})?$

By default, the base url is https://api.us1.app.wiz.io. This parameter allows you to customize the base url.

Info
This parameter can be left blank, removed or commented.

historic_date_utc

str

Optional

UTC with format:
YYYY-mm-ddTHH:MM:SS.sssZ

This configuration allows you to set a custom date as the beginning of the period to download. This allows downloading historical data (1 month back for example) before downloading new events.

If this setting is not set, the default value is the current time.

Note
Note that update this value triggers the clearing of the Collector’s persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.

Info
This parameter can be removed or commented.

client_id

str

Mandatory

Any

User Client ID to authenticate to the service.

client_secret

str

Mandatory

Any

User Secret Key to authenticate to the service.

request_period_in_seconds

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (60 seconds)

Info
This parameter can be removed or commented.

type_list

list

Optional

Possible values:

"TOXIC_COMBINATION"
"THREAT_DETECTION",
"CLOUD_CONFIGURATION"

Filter by Issue type. You can specify multiple values in an array.

Example 1:

Code Block
type: - THREAT_DETECTION

Example 2 (multiple values):

Code Block
type: - TOXIC_COMBINATION - THREAT_DETECTION

This parameter should be removed if it is not used.

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image

SHA-256 hash

collecto-wiz-docker-image-1.4.0.tgz

b9e82a00676ade05561e403f5ccaa7561b66dd384c74de76d29680c93a3262ce

Use the following command to add the Docker image to the system:

Code Block
gunzip -c <image_file>-<version>.tgz \| docker load

Note
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

Code Block

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

Code Block

version: '3'
services:
  collector-<product_name>:
    image: <image_name>:${IMAGE_VERSION:-latest}
    container_name: collector-<product_name>
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

Code Block
IMAGE_VERSION=<version> docker-compose up -d

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Collector service details

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.issues.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
    action {
      id
      type
    }
  }
}

Setup output

A successful run has the following output messages for the setup module:

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component

Description

Setup

The setup module is in charge of authenticating the service and managing the token expiration when needed.

Puller

The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination"request_period_in_seconds": <request_period_in_seconds>,
          "historic_date_utc": <historic_date_utc>
        },
        "auditLogs": {
          "request_period_in_seconds": <request_period_in_seconds>,
          "historic_date_utc": <historic_date_utc>
        },
        "cloudConfiguration": {
          "request_period_in_seconds": <request_period_in_seconds>,
          "historic_date_utc": <historic_date_utc>
        },
        "custom_query": {
          "types": [
            "custom_graphql_query"
          ],
          "request_period_in_seconds": "<request_period_in_seconds>",
          "historic_date_utc": "<start_date_utc>",
          "graphql_query": "<graphql_query>",
          "filter_by": "<filterBy_value_as_dict>",
          "filter_by_time_key": "<filter_by_time_key>",
          "response_time_key": "<response_time_key>",
          "override_devo_tag": "<override_devo_tag>"
        }
      }
    }
  }
}

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Please replace the placeholders with real world values following the description table below:

Parameter

Data type

Type

Value range / Format

Details

debug_status

bool

Mandatory

false / true

If the value is true, the debug logging traces will be enabled when running the collector. If the value is false, only the info, warning and error logging levels will be printed.

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

input_status

bool

Mandatory

false / true

Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.

base_url

str

Optional

Valid URL following this regex:
^https:\/\/([a-z0-9]+[.]{1})([a-z0-9]+[.]{1})*[a-z]{2,}(:[0-9]{2,5})?$

By default, the base url is https://api.us1.app.wiz.io. This parameter allows you to customize the base url.

This parameter should be removed if it is not used.

historic_date_utc

str

Optional

UTC with format:
YYYY-mm-ddTHH:MM:SS.sssZ

This configuration allows you to set a custom date as the beginning of the period to download. This allows downloading historical data (1 month back for example) before downloading new events.

If this setting is not set, the default value is the current time.

This parameter should be removed if it is not used.

client_id

str

Mandatory

Any

User Client ID to authenticate to the service.

client_secret

str

Mandatory

Any

User Secret Key to authenticate to the service.

request_period_in_seconds

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (60 seconds)

This parameter should be removed if it is not used.

type_list

list

Optional

Possible values:

"TOXIC_COMBINATION"
"THREAT_DETECTION",
"CLOUD_CONFIGURATION"

Filter by Issue type. You can specify multiple values in an array.

Example 1:

Code Block
"type": ["THREAT_DETECTION"]

Example 2 (multiple values):

Code Block
"type": ["TOXIC_COMBINATION", "THREAT_DETECTION"]

This parameter should be removed if it is not used.

override_tag_value

str

Optional

Devo Tag

Use this to override Devo tag. For custom_query service this is mandatory if the parser for the query is deployed.

graphql_query

str

Mandatory

Valid graphql query format

Used in custom_query service to provide custom query. Make sure to have three graphql variables → first , after and filterBy in the query . The query should also contain pageInfo parameter in it.

filterBy_value_as_dict

dict

Optional

Valid json

Filters can be added under the filter_by parameter as a dict.

filter_by_time_key

str

Mandatory

Minimum length: 1

You need to specify the datetime parameter which the qraphql query allows filtering on . For instance, for CloudEventsQuery after checking under CloudEventFilters , we find timestamp as the parameter.

response_time_key

str

Mandatory

Minimum length: 1

You need to specify the datetime parameter in the response of graphql query which the filter_by_time_key was applied on. For instance, for CloudEventsQuery it comes to be timestamp . Please note that response_time_key might be different from filter_by_time_key .

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

Code Block

<any_directory>
└── devo-collectors/
    └── <product_name>/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── state/
        └── config/ 
            └── config-<product_name>.yaml

Note
Replace `<product_name>` with the proper value.

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Image Added

Replace <product_name> with the proper value.

Editing the config.yaml file

Code Block

globals:
  debug: <debug_status>
  id: <collector_id>
  name: <collector_name>
  persistence:
    type: filesystem
    config:
      directory_name: state
  multiprocessing: false
  queue_max_size_in_mb: 1024
  queue_max_size_in_messages: 1000
  queue_max_elapsed_time_in_sec: 60
  queue_wrap_max_size_in_messages: 100

outputs:
  devo_1:
    type: devo_platform
    config:
      address: <devo_address>
      port: 443
      type: SSL
      chain: <chain_filename>
      cert: <cert_filename>
      key: <key_filename>

inputs:
  wiz_data_puller:
    id: <short_unique_id>
    enabled: <input_status>
    override_api_base_url: <base_url>
    credentials:
      client_id: <client_id>
      client_secret: <client_secret>
    services:
      issues:
        request_period_in_seconds: <request_period_in_seconds>
        historic_date_utc: <historic_date_utc>
        filters:
          type: <type_list>
        override_devo_tag : <override_tag_value>
      vulnerabilities:
        request_period_in_seconds: <request_period_in_seconds>
        historic_date_utc: <historic_date_utc>
        override_devo_tag: <override_tag_value>
      auditLogs:
        request_period_in_seconds: <request_period_in_seconds>
        historic_date_utc: <historic_date_utc>
        override_devo_tag: <override_tag_value>
      cloudConfiguration:
        request_period_in_seconds: <request_period_in_seconds>
        historic_date_utc: <historic_date_utc>
        override_devo_tag: <override_tag_value>
      custom_query:
        types:
          - custom_graphql_query
        request_period_in_seconds: <request_period_in_seconds>
        historic_date_utc: <start_date_in_utc>
        graphql_query: <graphql_query>
        filter_by: <filterBy_as_dict>
        filter_by_time_key: <filter_by_time_key_value>
        response_time_key: <response_time_key_value>
        override_devo_tag: <override_devo_tag>

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Replace the placeholders with your required values following the description table below:

Parameter

Data type

Type

Value range

Details

debug_status

bool

Mandatory

false / true

If the value is true, the debug logging traces will be enabled when running the collector. If the value is false, only the info, warning and error logging levels will be printed.

collector_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this collector.

collector_name

str

Mandatory

Minimum length: 1
Maximum length: 10

Use this param to give a valid name to this collector.

devo_address

str

Mandatory

collector-us.devo.io
collector-eu.devo.io

Use this param to identify the Devo Cloud where the events will be sent.

chain_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the chain.cert file downloaded from your Devo domain. Usually this file's name is: chain.crt

cert_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the file.cert downloaded from your Devo domain.

key_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the file.key downloaded from your Devo domain.

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

input_status

bool

Mandatory

false / true

Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.

requests_per_seconds

int

Optional

Minimum value: 1

Customize the maximum number of API requests per second. If not used, the default setting will be used: 100000 requests/sec.

This parameter can be left blank, removed or commented.

base_url

str

Optional

Valid URL following this regex:
^https:\/\/([a-z0-9]+[.]{1})([a-z0-9]+[.]{1})*[a-z]{2,}(:[0-9]{2,5})?$

By default, the base url is https://api.us1.app.wiz.io. This parameter allows you to customize the base url.

This parameter can be left blank, removed or commented.

historic_date_utc

str

Optional

UTC with format:
YYYY-mm-ddTHH:MM:SS.sssZ

This configuration allows you to set a custom date as the beginning of the period to download. This allows downloading historical data (1 month back for example) before downloading new events.

If this setting is not set, the default value is the current time.

Note
Note that update this value triggers the clearing of the Collector’s persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.

This parameter can be removed or commented.

client_id

str

Mandatory

Any

User Client ID to authenticate to the service.

client_secret

str

Mandatory

Any

User Secret Key to authenticate to the service.

request_period_in_seconds

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (60 seconds)

This parameter can be removed or commented.

type_list

list

Optional

Possible values:

"TOXIC_COMBINATION"
"THREAT_DETECTION",
"CLOUD_CONFIGURATION"

Filter by Issue type. You can specify multiple values in an array.

Example 1:

Code Block
type: - THREAT_DETECTION

Example 2 (multiple values):

Code Block
type: - TOXIC_COMBINATION - THREAT_DETECTION

This parameter should be removed if it is not used.

override_tag_value

str

Optional

Devo Tag

Use this to override Devo tag. For custom_query service this is mandatory if the parser for the query is deployed.

graphql_query

str

Mandatory

Valid graphql query format

Used in custom_query service to provide custom query. Make sure to have three graphql variables → first , after and filterBy in the query . The query should also contain pageInfo parameter in it.

filterBy_value_as_dict

dict

Optional

Valid json

Filters can be added under the filter_by parameter as a dict.

filter_by_time_key

str

Mandatory

Minimum length: 1

You need to specify the datetime parameter which the qraphql query allows filtering on . For instance, for CloudEventsQuery after checking under CloudEventFilters , we find timestamp as the parameter.

response_time_key

str

Mandatory

Minimum length: 1

You need to specify the datetime parameter in the response of graphql query which the filter_by_time_key was applied on. For instance, for CloudEventsQuery it comes to be timestamp . Please note that response_time_key might be different from filter_by_time_key .

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image	SHA-256 hash
collecto-wiz-docker-image-1.7.0.tgz	`a2c71efe0465eda6d1de9e44b70c92f68b4535decb0dcf5361b206176d3df0dd`

Use the following command to add the Docker image to the system:

Code Block
gunzip -c <image_file>-<version>.tgz \| docker load

Note
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

Code Block

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

Code Block

version: '3'
services:
  collector-<product_name>:
    image: <image_name>:${IMAGE_VERSION:-latest}
    container_name: collector-<product_name>
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

Code Block
IMAGE_VERSION=<version> docker-compose up -d

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Collector service details

Issue Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.issues.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
    action {
      id
      type
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Updating deduplication buffers content
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Terminated
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Data collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

Vulnerability Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.vulnerabilities.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues: issuesV2(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  type
  description
  sourceRule {
    __typename
    ... on Control {
      id
      name
      controlDescription: description
      resolutionRecommendation
      securitySubCategories {
        title
        category {
          name
          framework {
            name
          }
        }
      }
    }
    ... on CloudEventRule {
      id
      name
      cloudEventRuleDescription: description
      sourceType
      type
    }
    ... on CloudConfigurationRule {
      id
      name
      cloudConfigurationRuleDescription: description
      remediationInstructions
      serviceType
    }
  }
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#00001,vulnerabilities#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#00001,vulnerabilities#predefined) -> This is the first run of the collector. Generating the access token.
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#00001,vulnerabilities#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#00001,vulnerabilities#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#00001,vulnerabilities#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#00001,vulnerabilities#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

2024-02-15T06:48:00.286    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> PrePull Started.
2024-02-15T06:48:00.286    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> User has specified 2023-11-11 20:10:02 as the datetime. Historical polling will consider this datetime for creating the default values.
2024-02-15T06:48:00.286 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Historic datetime in the persistence object and in the configuration are different. Updating the value in state with the user specified datetime.
2024-02-15T06:48:00.286 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Next poll will consider 2023-11-11 20:10:02 to now as the date range.
2024-02-15T06:48:00.287 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
2024-02-15T06:48:00.287    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> PrePull Terminated
2024-02-15T06:48:00.287    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Starting data collection every 60 seconds
2024-02-15T06:48:00.287    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Pull Started
2024-02-15T06:48:00.288    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Fetching vulnerabilities from 2023-11-11T20:10:02Z
2024-02-15T06:48:00.288    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Requesting Wiz API for vulnerabilities
2024-02-15T06:48:03.155    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> successfully retried vulnerabilities from Wiz
2024-02-15T06:48:03.240    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Total number of events in this poll: 500
2024-02-15T06:48:03.241    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:03.250    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:03.255    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:03.485    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> 500 issues delivered
2024-02-15T06:48:03.507    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2023, 11, 13, 14, 38, 14, 622387), 'buffer_ids_with_duplication_risk': ['f7723a4c-3108-5149-8c5b-52582c2a6474']}.
2024-02-15T06:48:03.507    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.286006) so far: Number of requests made: 1; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 500.
2024-02-15T06:48:03.507    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Requesting Wiz API for vulnerabilities
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
2024-02-15T06:48:06.423    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> successfully retried vulnerabilities from Wiz
2024-02-15T06:48:06.454    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:06.457    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:06.459    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:06.509    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> 500 issues delivered
2024-02-15T06:48:06.510    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2023, 11, 17, 21, 54, 52, 964304), 'buffer_ids_with_duplication_risk': ['23d093be-97a8-59f9-a95d-36a841943235']}.
2024-02-15T06:48:06.510    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.286006) so far: Number of requests made: 2; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 1000.
2024-02-15T06:48:06.510    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Requesting Wiz API for vulnerabilities
2024-02-15T06:57:59.335    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> successfully retried vulnerabilities from Wiz
2024-02-15T06:57:59.336    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Removing the duplicate issues if present
2024-02-15T06:57:59.337    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:57:59.337    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Delivering issues to the SDK
2024-02-15T06:57:59.337    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> 0 issues delivered
2024-02-15T06:57:59.338    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Updating deduplication buffers content
2024-02-15T06:57:59.338    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1707979680.286006):Number of requests made: 196; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 97500; Average of events per second: 162.758.
2024-02-15T06:57:59.338    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Pull Terminated
2024-02-15T07:04:28.645    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> Data collection completed. Elapsed time: 0.626 seconds. Waiting for 59.374 second(s) until the next one

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

2024-02-15T07:03:39.203    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,vulnerabilities,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707980279.338879) so far: Number of requests made: 110; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 55000.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

AuditLogs Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.audit.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query AuditLogTable(
  $first: Int
  $after: String
  $filterBy: AuditLogEntryFilters
) {
  auditLogEntries(first: $first, after: $after, filterBy: $filterBy) {
    nodes {
      id
      action
      requestId
      status
      timestamp
      actionParameters
      userAgent
      sourceIP
      serviceAccount {
        id
        name
      }
      user {
        id
        name
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

2024-02-15T06:47:57.287    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,auditLogs#predefined) -> Puller Setup Started
2024-02-15T06:47:57.287    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,auditLogs#predefined) -> This is the first run of the collector. Generating the access token.
2024-02-15T06:47:57.288    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,auditLogs#predefined) -> Getting the auth token url based on provided api_base_url
2024-02-15T06:47:57.288    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,auditLogs#predefined) -> Getting the auth token url based on provided api_base_url
2024-02-15T06:47:59.494    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,auditLogs#predefined) -> Puller Setup Terminated
2024-02-15T06:47:59.494    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,auditLogs#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

2024-02-15T06:48:00.295    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> PrePull Started.
2024-02-15T06:48:00.295    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> User has specified 2023-11-11 20:10:02 as the datetime. Historical polling will consider this datetime for creating the default values.
2024-02-15T06:48:00.295 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Historic datetime in the persistence object and in the configuration are different. Updating the value in state with the user specified datetime.
2024-02-15T06:48:00.295 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Next poll will consider 2023-11-11 20:10:02 to now as the date range.
2024-02-15T06:48:00.296 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
2024-02-15T06:48:00.296    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> PrePull Terminated
2024-02-15T06:48:00.296    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Starting data collection every 60 seconds
2024-02-15T06:48:00.296    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Pull Started
2024-02-15T06:48:00.296    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Fetching auditLogs from 2023-11-11T20:10:02Z
2024-02-15T06:48:00.297    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Requesting Wiz API for auditLogs
2024-02-15T06:48:01.886    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> successfully retried auditLogs from Wiz
2024-02-15T06:48:01.918    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Total number of events in this poll: 500
2024-02-15T06:48:01.919    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:01.919    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:01.920    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:02.017    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> 500 issues delivered
2024-02-15T06:48:02.055    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2024, 2, 15, 6, 47, 59, 126416), 'buffer_ids_with_duplication_risk': []}.
2024-02-15T06:48:02.055    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.295365) so far: Number of requests made: 1; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 500.
2024-02-15T06:48:02.055    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Requesting Wiz API for auditLogs
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
2024-02-15T06:48:01.886    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> successfully retried auditLogs from Wiz
2024-02-15T06:48:01.918    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Total number of events in this poll: 500
2024-02-15T06:48:01.919    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:01.919    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:01.920    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:02.017    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> 500 issues delivered
2024-02-15T06:48:02.055    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2024, 2, 15, 6, 47, 59, 126416), 'buffer_ids_with_duplication_risk': []}.
2024-02-15T06:48:02.055    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.295365) so far: Number of requests made: 1; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 500.
2024-02-15T06:48:02.055    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Requesting Wiz API for auditLogs
2024-02-15T06:48:32.320    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> successfully retried auditLogs from Wiz
2024-02-15T06:48:32.326    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:32.328    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:32.328    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:32.343    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> 343 issues delivered
2024-02-15T06:48:32.344    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2023, 11, 16, 1, 16, 19, 908609), 'buffer_ids_with_duplication_risk': []}.
2024-02-15T06:48:32.344    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Updating deduplication buffers content
2024-02-15T06:48:32.344    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1707979680.295365):Number of requests made: 21; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 10343; Average of events per second: 322.735.
2024-02-15T06:48:32.345    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Pull Terminated
2024-02-15T06:48:32.345    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Data collection completed. Elapsed time: 32.050 seconds. Waiting for 27.950 second(s) until the next one

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

2024-02-15T06:48:32.344    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,auditLogs,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1707979680.295365):Number of requests made: 21; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 10343; Average of events per second: 322.735.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

CloudConfiguration Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.cloud_configuration.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query CloudConfigurationFindingsPage(
  $filterBy: ConfigurationFindingFilters
  $first: Int
  $after: String
  $orderBy: ConfigurationFindingOrder
) {
  configurationFindings(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      id
      targetExternalId
      targetObjectProviderUniqueId
      firstSeenAt
      severity
      result
      status
      remediation
      resource {
        id
        providerId
        name
        nativeType
        type
        region
        subscription {
          id
          name
          externalId
          cloudProvider
        }
        projects {
          id
          name
          riskProfile {
            businessImpact
          }
        }
        tags {
          key
          value
        }
      }
      rule {
        id
        graphId
        name
        description
        remediationInstructions
        functionAsControl
      }
      securitySubCategories {
        id
        title
        category {
          id
          name
          framework {
            id
            name
          }
        }
      }
      ignoreRules {
        id
        name
        enabled
        expiredAt
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

2024-02-15T06:47:57.290    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,cloudConfiguration#predefined) -> Puller Setup Started
2024-02-15T06:47:57.292    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,cloudConfiguration#predefined) -> This is the first run of the collector. Generating the access token.
2024-02-15T06:47:57.292    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,cloudConfiguration#predefined) -> Getting the auth token url based on provided api_base_url
2024-02-15T06:47:57.292    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,cloudConfiguration#predefined) -> Using default Authentication Domain auth.wiz.io for fetching Access Token
2024-02-15T06:47:59.891    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,cloudConfiguration#predefined) -> Puller Setup Terminated
2024-02-15T06:47:59.891    INFO InputProcess::WizDataPullerSetup(Wiz_test_nikhil,wiz_data_puller#00001,cloudConfiguration#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

2024-02-15T06:48:00.290    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> PrePull Started.
2024-02-15T06:48:00.291    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> User has specified 2023-11-11 20:10:02 as the datetime. Historical polling will consider this datetime for creating the default values.
2024-02-15T06:48:00.291 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Historic datetime in the persistence object and in the configuration are different. Updating the value in state with the user specified datetime.
2024-02-15T06:48:00.291 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Next poll will consider 2023-11-11 20:10:02 to now as the date range.
2024-02-15T06:48:00.291 WARNING InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
2024-02-15T06:48:00.292    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> PrePull Terminated
2024-02-15T06:48:00.292    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Starting data collection every 60 seconds
2024-02-15T06:48:00.292    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Pull Started
2024-02-15T06:48:00.292    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Fetching cloudConfiguration from 2023-11-11T20:10:02Z
2024-02-15T06:48:00.292    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Requesting Wiz API for cloudConfiguration
2024-02-15T06:48:29.044    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> successfully retried cloudConfiguration from Wiz
2024-02-15T06:48:29.127    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Total number of events in this poll: 500
2024-02-15T06:48:29.128    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:29.128    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:29.129    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:29.238    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> 500 issues delivered
2024-02-15T06:48:29.239    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2023, 11, 23, 6, 46, 39, 979510), 'buffer_ids_with_duplication_risk': ['498e0032-f76f-58d8-935b-dc20546c4d77']}.
2024-02-15T06:48:29.239    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.290731) so far: Number of requests made: 1; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 500.
2024-02-15T06:48:29.239    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Requesting Wiz API for cloudConfiguration
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
2024-02-15T06:48:40.162    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> successfully retried cloudConfiguration from Wiz
2024-02-15T06:48:40.207    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:40.216    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:40.217    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:40.297    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> 500 issues delivered
2024-02-15T06:48:40.299    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2023, 11, 24, 3, 29, 59, 755884), 'buffer_ids_with_duplication_risk': ['0e446865-0e54-567d-a3ab-f5bfedecefbc']}.
2024-02-15T06:48:40.299    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.290731) so far: Number of requests made: 2; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 1000.
2024-02-15T06:48:40.299    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Requesting Wiz API for cloudConfiguration
2024-02-15T06:48:40.162    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> successfully retried cloudConfiguration from Wiz
2024-02-15T06:48:40.207    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Removing the duplicate issues if present
2024-02-15T06:48:40.216    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
2024-02-15T06:48:40.217    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Delivering issues to the SDK
2024-02-15T06:48:40.297    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> 500 issues delivered
2024-02-15T06:48:40.299    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'last_polled_timestamp': datetime.datetime(2023, 11, 11, 20, 10, 2, tzinfo=<UTC>), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2023, 11, 24, 3, 29, 59, 755884), 'buffer_ids_with_duplication_risk': ['0e446865-0e54-567d-a3ab-f5bfedecefbc']}.
2024-02-15T06:48:40.299    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1707979680.290731) so far: Number of requests made: 2; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 1000.
2024-02-15T06:48:40.299    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Requesting Wiz API for cloudConfiguration
2024-02-15T07:01:14.752    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1707980301.689743):Number of requests made: 26; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 12779; Average of events per second: 73.841.
2024-02-15T07:01:14.752    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Pull Terminated

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

2024-02-15T07:01:14.752    INFO InputProcess::WizDataPuller(wiz_data_puller,00001,cloudConfiguration,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1707980301.689743):Number of requests made: 26; Number of events received: 500; Number of duplicated events filtered out: 0; Number of events generated and sent: 12779; Average of events per second: 73.841.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

Expand

title	Restart the persistence

This collector uses persistent storage to download events in an orderly fashion and avoid duplicates. In case you want to re-ingest historical data or recreate the persistence, you can restart the persistence of this collector by following these steps:

Edit the configuration file.
Change the value of the historical_date_utc parameter to a different one.
Save the changes.
Restart the collector.

The collector will detect this change and will restart the persistence using the parameters of the configuration file or the default configuration in case it has not been provided.

Note
Note that this action clears the persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.

Expand

title	Troubleshooting

Error type	Error ID	Error message	Cause	Solution
`InitVariablesError`	`1`	`Devo tag is the required field for sending events to Devo. Specify it in collector definitions`	This error is raised when `devo_tag` property is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`2`	`Required setting. devo_tag is not of expected type: str`	This error is raised when `devo_tag` is defined in `collector_definitions.yaml` but the format is not `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`3`	`Optional setting, override_devo_tag not of expected type: str`	This error is raised when optional value `override_devo_tag` added in `config.json` is not of type `str`.	Edit the value of `override_devo_tag` in `config.json` so it is of type `str`. Or leave it empty so it takes the default value.
`InitVariablesError`	`4`	`GraphQL query is the required field for querying issues from Wiz. Specify it in collector definitions`	This error is raised when `graphql_query` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`5`	`Required setting. graphql_query is not of expected type: str`	This error is raised when `graphql_query` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`6`	`user_agent is the required field for passing in headers of Wiz API calls. Specify it in collector definitions`	This error is raised when `user_agent` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`7`	`Required setting. user_agent is not of expected type: str`	This error is raised when `user_agent` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`8`	`Optional setting, flatten_data not of expected type: bool`	This error is raised when the optional value `flatten_data` defined in `config.json` is not of type `bool`.	Edit the value of `flatten_data` in `config.json` so it is of type `bool`. You can also remove `override_flatten_data` parameter from `config.json` so it takes the default value.
`InitVariablesError`	`9`	`Optional setting, requests_per_second not of expected type: int`	This error is raised when the optional value `requests_per_second` defined in `config.json` is not of type `int`.	Edit the value of `requests_per_second` in `config.json` so it is of type `int`. Or leave it empty so it takes the default value.
`InitVariablesError`	`10`	`Required setting. requested_page_size_in_items is not of expected type: int`	This error is raised when `requested_page_size_in_items` defined in `collector_definitions.yaml` is not of type `int`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`11`	`access_token_timeout is the required field for checking if the token is expired. Specify it in collector definitions`	This error is raised when `access_token_timeout` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`12`	`Required setting. access_token_timeout is not of expected type: int`	This error is raised when `access_token_timeout` defined in `collector_definitions.yaml` is not of type `int`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`13`	`default_historic_days is the required field in case historic_date_utc is not specified. Specify it in collector definitions`	This error is raised when `default_historic_days` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`14`	`Required setting. default_historic_days is not of expected type: int`	This error is raised when `default_historic_days` defined in `collector_definitions.yaml` is not of type `int`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`15`	`api_url_regex is the required field for validating the base url. Specify it in collector definitions`	This error is raised when `api_url_regex` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`16`	`Required setting. api_url_regex is not of expected type: str`	This error is raised when `api_url_regex` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`17`	`historic_date_time_format is the required field for validating datetime format. Specify it in collector definitions`	This error is raised when `api_url_regex` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`18`	`Required setting. historic_date_time_format is not of expected type: str`	This error is raised when `api_url_regex` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`19`	`api_base_url not of expected type: str`	This error could be raised for two reasons: `api_base_url` defined in `collector_definitions.yaml` is not of type `str`. `override_api_base_url` defined in `config.json` is not of type `str`.	Solutions for both cases, respectively: This is an internal issue. Contact with Devo Support team. Edit the value of `override_api_base_url` in `config.json` so it is of type `str`. Or remove the parameter definition, so it takes the default value.
`InitVariablesError`	`20`	`api_base_url must match regex: <regex>`	This error could be raised for two reasons: `api_base_url` defined in `collector_definitions.yaml` does not match the required regex. `override_api_base_url` defined in `config.json` does not match the required regex.	Solutions for both cases, respectively: This is an internal issue. Contact with Devo Support team. Edit the value of `override_api_base_url` in `config.json` so it matches the indicated regex. Or remove the parameter definition so it takes the default value. If default value was used it is an internal issue, so contact with Devo Support team.
`InitVariablesError`	`21`	`Required setting, credentials not found in user configuration`	This error is raised when the required property `credentials` is not found in `config.json`.	Add `credentials` dictionary in `config.json`, including `client_id` and `client_secret` fields.
`InitVariablesError`	`22`	`Required setting, credentials not of expected type: dict`	This error is raised when `credentials` is defined in `config.json` but the format is not `dict`.	Edit the value of `credentials` in `config.json` so it is of type `dict`.
`InitVariablesError`	`23`	`Required setting, client_id not found in user configuration`	This error is raised when the required property `client_id` is not found in `config.json`, into `credentials` dictionary.	Add `client_id` property in `config.json`, into `credentials` dictionary.
`InitVariablesError`	`24`	`Required setting, client_id not of expected type: str`	This error is raised when `client_id` is defined in `config.json` but the format is not `str`.	Edit the value of `client_id` in `config.json`, into `credentials` dictionary, so it is of type `str`.
`InitVariablesError`	`25`	`Required setting, client_secret not found in user configuration`	This error is raised when the required property `client_secret` is not found in `config.json`, into `credentials` dictionary.	Add `client_secret` property in `config.json`, into `credentials` dictionary.
`InitVariablesError`	`26`	`Required setting, client_secret not of expected type: str`	This error is raised when `client_secret` is defined in `config.json` but the format is not `str`.	Edit the value of `client_secret` in `config.json`, into `credentials` dictionary, so it is of type `str`.
`InitVariablesError`	`27`	`Required setting, historic_date_utc not of expected type: str`	This error is raised when the optional value `historic_date_utc` defined in `config.json` is not of type `str`.	Edit the value of `historic_date_utc` in `config.json` so it is of type `str`. Or leave it empty so the collector starts pulling data N days ago at the current time.
`InitVariablesError`	`28`	`Time format for historic date must be <time_format>. e.g. 2022-02-15T14:32:33.043Z`	This error is raised when the optional value `historic_date_utc` defined in `config.json` does not match the indicated format.	Make the value of `historic_date_utc` in `config.json` match the indicated format. Or leave it empty so the collector starts pulling data N days ago at the current time.
`InitVariablesError`	`29`	`historic datetime cannot be greater than the present UTC time`	This error is raised when the optional value `historic_date_utc` defined in `config.json` is higher than the current time. Time is indicated in UTC time.	Make the value of `historic_date_utc` in `config.json` be a past value in UTC time. Or leave it empty so the collector starts pulling data N days ago at the current time.
`SetupError`	`100`	`Error occurred while requesting access token from the Wiz server. Error message: <error_message>`	This error is raised when a generic error occurs during the request to get the token to authenticate the collector in the API.	This is an internal issue. Contact with Devo Support team.
`SetupError`	`101`	`The credentials provided in the config file are incorrect. Please provide the correct credentials.` `Status code: 401` `Error type: <error_type>` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are not valid. (401 Unauthorized error)	Add the correct `client_id` and `client_secret` fields in `config.json`, at `credentials` property.
`SetupError`	`102`	`The credentials provided in the config file does not have necessary permissions to create access token.` `Status code: 403` `Error type: <error_type>` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but the credentials used to access the API have no permissions to create a token. (403 Forbidden error)	Add some credentials in `config.json` with enough privileges to create a token from the API.
`SetupError`	`103`	`The requested URL <URL> is not found. The URL may have been deprecated.` `Status code: 404` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but the authentication endpoint that is being requested to get a token is not found.	This is an internal issue. Contact with Devo Support team.
`SetupError`	`104`	`Unexpected error occurred while getting access token from the Wiz server` `Status code: <status_code>` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but an unexpected response has been returned from the API.	This is an internal issue. Contact with Devo Support team.
`SetupError`	`105`	`The credentials does not have valid permissions to fetch issues from the Wiz server`	This error is raised when the credentials provided in `config.json` are valid, but the credentials used have no permissions to access the API endpoint to get issues.	Check that the provided credentials have enough permission to retrieve issues from the API.
`SetupError`	`106`	`Failed to check if the provided credentials have valid permissions.` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but there was an error in the process of checking if the provided credentials have permission to get issues.	This is an internal issue. Contact with Devo Support team.
`PullError`	`300`	`Error occurred while requesting issues from the Wiz server. Error message: <error_message>`	This error is raised when a generic error occurs during the request to get issues from the API.	This is an internal issue. Contact with Devo Support team.
`PullError`	`301`	`The token used to make this request is not valid anymore.` `Status code: 401` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is not valid anymore.	Check if the credentials need to be renewed. Add some valid credentials in `config.json`. If credentials are still valid, contact with Devo Support team.
`PullError`	`302`	`The access token does not have necessary permissions to fetch issues from Wiz.` `Status code: 403` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is valid, but it lost permissions to get issues from API.	Check what happened with credential permissions. Add some credentials in `config.json` with permissions to get issues. If credentials are valide and still have permissions, contact with Devo Support team.
`PullError`	`303`	`The requested URL <URL> is not found. The URL may have been depreciated` `Status code: 404` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is valid, but cannot find the endpoints to get issues.	This is an internal issue. Contact with Devo Support team.
`PullError`	`304`	`The server has returned <status_code> status code. The server may not be available for fetching issues. Try after sometime. Error message from server: <error_message>`	This error is raised when the token being used to make requests to the API is valid, but there has been an error on Wiz's API.	The error is on Wiz’s side. Wiz can be contacted for more info. It should work again when the incident at Wiz is solved.
`PullError`	`305`	`Unexpected error occurred while getting issues from the Wiz server` `Status code: <status_code>` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is valid, but there has been an unexpected return from the API.	This is an internal issue. Contact with Devo Support team.
`PullError`	`306`	`After <retry_count> retries still getting the too many requests error.`	This error is raised when the token being used to make requests to the API is valid, but we are constantly receiving a `429` error response (too many requests)	Check throttle limitations on Wiz API and change the value of `request_period_in_seconds` and xxxxxxxx from `config.json` to adapt to it.

Custom Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table my.app.wiz.custom_query by default. You need to provide override_tag if you need to change it.

Expand

title	GraphQL queries whose parsers are deployed

CloudEventsQuery
- Override Devo tag : cspm.wiz.cloud_event.default
- filter_by_time_key: timestamp
- response_time_key: timestamp
- filter_by: { "origin": { "equals": "WIZ_SENSOR" } }

Note
`filter_by` is just an example and can change as per requirement. All above fields need to be in user config.

Code Block

query CloudEventsQuery(
  $first: Int
  $after: String
  $filterBy: CloudEventFilters
) {
  cloudEvents(first: $first, after: $after, filterBy: $filterBy) {
    __typename
    totalCount
    nodes {
      ... on CloudEvent {
        id
        name
        kind
        origin
        severity
        externalId
        externalName
        cloudPlatform
        timestamp
        cloudNativeService
        category
        actor {
          id
          externalId
          providerUniqueId
          type
          cloudAccount {
            id
          }
          isExternalCloudAccount
          friendlyName
          name
          email
          userAgent
        }
        actorIP
        actorIPMeta {
          country
          countryCode
          city
          reputation
          reputationSource
          reputationDescription
        }
        isForeignActorIP
        subjectResource {
          id
          externalId
          providerUniqueId
          type
          nativeType
          name
          hostname
          cloudAccount {
            id
            externalId
            name
            cloudProvider
            linkedProjects {
              name
            }
          }
          region
          tags
          openToAllInternet
          hasSensitiveData
          kubernetesCluster {
            id
            name
            type
          }
          kubernetesNamespace {
            id
            name
            type
          }
          kubernetesFlavor
          containerService {
            id
            name
          }
        }
        cloudProviderUrl
        path
        hash
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

systemActivitiesQuery
- Override Devo tag : cspm.wiz.system_activity.default
- filter_by_time_key: createdAt
- response_time_key: createdAt
- filter_by: {}

Note
`filter_by` is just an example and can change as per requirement. All above fields need to be in user config.

Code Block

query systemActivitiesQuery(
  $first: Int
  $after: String
  $filterBy: SystemActivityFilters
) {
  systemActivities(first: $first, after: $after, filterBy: $filterBy) {
    __typename
    totalCount
    nodes {
      ... on SystemActivity {
        id
        name
        triggerType
        triggeredBy {
          ... on SystemActivityUserSnapshot {
            id
            name
          }
        }
        triggeredBy {
          ... on SystemActivitySystemTrigger {
            id
          }
        }
        triggeredBy {
          ... on SystemActivityServiceAccountSnapshot {
            id
            name
          }
        }
        createdAt
        startedAt
        endedAt
        status
        statusInfo
        summary
        groupId
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

2024-07-12T14:44:51.388    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> Puller Setup Started
2024-07-12T14:44:51.388    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> This is the first run of the collector. Generating the access token.
2024-07-12T14:44:51.388    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> Getting the auth token url based on provided api_base_url
2024-07-12T14:44:52.690    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> successfully generated new access token
2024-07-12T14:44:53.548    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
2024-07-12T14:44:53.549    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> Puller Setup Terminated
2024-07-12T14:44:53.549    INFO InputProcess::WizCustomDataPullerSetup(unknown,wiz_data_puller#3454335,custom_query#custom) -> Setup for module <WizCustomDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

2024-07-12T14:44:54.393    INFO InputProcess::WizCustomDataPuller(wiz_data_puller,3454335,custom_query,custom) -> PrePull Started.
2024-07-12T14:44:54.396    INFO InputProcess::WizCustomDataPuller(wiz_data_puller,3454335,custom_query,custom) -> User has specified 2024-07-05 06:03:52 as the datetime. Historical polling will consider this datetime for creating the default values.
2024-07-12T14:44:54.396    INFO InputProcess::WizCustomDataPuller(wiz_data_puller,3454335,custom_query,custom) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(20222024, 7, 15, 16, 03, 052), 'last_polled_timestamp': datetime.datetime(20222024, 7, 15, 16, 03, 052), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
2024-07-12T14:44:54.398 WARNING InputProcess::WizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> (Partial)Saved Statisticsstate for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.loaded: {'historic_date_utc': datetime.datetime(2024, 7, 5, 6, 3, 52), 'last_polled_timestamp': datetime.datetime(2024, 7, 5, 6, 3, 52), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
2024-07-12T14:44:54.398    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> PrePull Requesting Wiz API for issuesTerminated
2024-07-12T14:44:54.399    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> successfully retried issues from Wiz Starting data collection every 300 seconds
2024-07-12T14:44:54.399    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> RemovingPull the duplicate issues if presentStarted
2024-07-12T14:44:54.401    INFO InputProcessOutputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPullerConsoleSender(internal_senders,console_sender_0) -> {"message_timestamp": "2024-07-12 09:14:54.399", "message_tag": "devo.collectors.out.local.info", "message_content": "{\"msg\": \"Starting data collection every 300 seconds\", \"time\": \"2024-07-12T09:14:54.399167Z\", \"level\": \"info\", \"collector_name\": \"unknown\", \"collector_version\": \"unknown\", \"collector_image\": null, \"job_id\": \"unknown\", \"input_name\": \"wiz_data_puller\", \"service_name\": \"custom_query\", \"module_name\": \"WizCustomDataPuller\"}"}
2024-07-12T14:44:54.401    INFO InputProcess::WizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> StateFetching hascustom_query been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.from 2024-07-05T00:33:52Z
2024-07-12T14:44:54.401    INFO InputProcess::WizCustomDataPuller(wiz_data_puller,3454335,custom_query,custom) -> Requesting Wiz API for custom_query
2024-07-12T14:44:58.123    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> Updating deduplication buffers content successfully retried custom_query from Wiz
2024-07-12T14:44:58.136    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> StatisticsTotal number forof thisevents pullin cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.this poll: 500
2024-07-12T14:44:58.202    INFO InputProcess::WizCustomDataPuller(wiz_data_puller,3454335,custom_query,custom) -> Flatten data is set to False. Adding 'devo_pulling_id' to events
2024-07-12T14:44:58.202    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> Pull Terminated Delivering issues to the SDK
2024-07-12T14:44:58.236    INFO InputProcess::WizDataPullerWizCustomDataPuller(wiz_data_puller,000113454335,issuescustom_query,predefinedcustom) -> Data500 collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)issues delivered

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

2024-02-15T06:48:32.344    INFO InputProcess::WizDataPuller(wiz_data_puller,0001100001,issuesauditLogs,predefined) -> Statistics for this pull cycle (@devo_pulling_id=16566027931707979680.044179295365):Number of requests made: 321; Number of events received: 45500; Number of duplicated events filtered out: 0; Number of events generated and sent: 4510343; Average of events per second: 33322.797735.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

...

INFO MainProcess::MainThread -> (CollectorMultiprocessingQueue) standard_queue_multiprocessing -> max_size_in_messages: 1000, max_size_in_mb: 1024, max_wrap_size_in_items: 100 INFO MainProcess

Expand

title	Troubleshooting

Expand

title	Verify collector operations

Initialization

The initialization module is in charge of setup and running the input (pulling logic) and output (delivering logic) services and validating the given configuration.

A successful run has the following output messages for the initializer module:

Code Block

Error type	Error ID	Error message	Cause	Solution
`InitVariablesError`	`1`	`Devo tag is the required field for sending events to Devo. Specify it in collector definitions`	This error is raised when `devo_tag` property is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`2`	`Required setting. devo_tag is not of expected type: str`	This error is raised when `devo_tag` is defined in `collector_definitions.yaml` but the format is not `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`3`	`Optional setting, override_devo_tag not of expected type: str`	This error is raised when optional value `override_devo_tag` added in `config.json` is not of type `str`.	Edit the value of `override_devo_tag` in `config.json` so it is of type `str`. Or leave it empty so it takes the default value.
`InitVariablesError`	`4`	`GraphQL query is the required field for querying issues from Wiz. Specify it in collector definitions`	This error is raised when `graphql_query` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`5`	`Required setting. graphql_query is not of expected type: str`	This error is raised when `graphql_query` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`6`	`user_agent is the required field for passing in headers of Wiz API calls. Specify it in collector definitions`	This error is raised when `user_agent` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`7`	`Required setting. user_agent is not of expected type: str`	This error is raised when `user_agent` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`8`	`Optional setting, flatten_data not of expected type: bool`	This error is raised when the optional value `flatten_data` defined in `config.json` is not of type `bool`.	Edit the value of `flatten_data` in `config.json` so it is of type `bool`. You can also remove `override_flatten_data` parameter from `config.json` so it takes the default value.
`InitVariablesError`	`9`	`Optional setting, requests_per_second not of expected type: int`	This error is raised when the optional value `requests_per_second` defined in `config.json` is not of type `int`.	Edit the value of `requests_per_second` in `config.json` so it is of type `int`. Or leave it empty so it takes the default value.
`InitVariablesError`	`10`	`Required setting. requested_page_size_in_items is not of expected type: int`	This error is raised when `requested_page_size_in_items` defined in `collector_definitions.yaml` is not of type `int`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`11`	`access_token_timeout is the required field for checking if the token is expired. Specify it in collector definitions`	This error is raised when `access_token_timeout` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`12`	`Required setting. access_token_timeout is not of expected type: int`	This error is raised when `access_token_timeout` defined in `collector_definitions.yaml` is not of type `int`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`13`	`default_historic_days is the required field in case historic_date_utc is not specified. Specify it in collector definitions`	This error is raised when `default_historic_days` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`14`	`Required setting. default_historic_days is not of expected type: int`	This error is raised when `default_historic_days` defined in `collector_definitions.yaml` is not of type `int`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`15`	`api_url_regex is the required field for validating the base url. Specify it in collector definitions`	This error is raised when `api_url_regex` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`16`	`Required setting. api_url_regex is not of expected type: str`	This error is raised when `api_url_regex` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`17`	`historic_date_time_format is the required field for validating datetime format. Specify it in collector definitions`	This error is raised when `api_url_regex` is not found in `collector_definitions.yaml`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`18`	`Required setting. historic_date_time_format is not of expected type: str`	This error is raised when `api_url_regex` defined in `collector_definitions.yaml` is not of type `str`.	This is an internal issue. Contact with Devo Support team.
`InitVariablesError`	`19`	`api_base_url not of expected type: str`	This error could be raised for two reasons: `api_base_url` defined in `collector_definitions.yaml` is not of type `str`. `override_api_base_url` defined in `config.json` is not of type `str`.	Solutions for both cases, respectively: This is an internal issue. Contact with Devo Support team. Edit the value of `override_api_base_url` in `config.json` so it is of type `str`. Or remove the parameter definition, so it takes the default value.
`InitVariablesError`	`20`	`api_base_url must match regex: <regex>`	This error could be raised for two reasons: `api_base_url` defined in `collector_definitions.yaml` does not match the required regex. `override_api_base_url` defined in `config.json` does not match the required regex.	Solutions for both cases, respectively: This is an internal issue. Contact with Devo Support team. Edit the value of `override_api_base_url` in `config.json` so it matches the indicated regex. Or remove the parameter definition so it takes the default value. If default value was used it is an internal issue, so contact with Devo Support team.
`InitVariablesError`	`21`	`Required setting, credentials not found in user configuration`	This error is raised when the required property `credentials` is not found in `config.json`.	Add `credentials` dictionary in `config.json`, including `client_id` and `client_secret` fields.
`InitVariablesError`	`22`	`Required setting, credentials not of expected type: dict`	This error is raised when `credentials` is defined in `config.json` but the format is not `dict`.	Edit the value of `credentials` in `config.json` so it is of type `dict`.
`InitVariablesError`	`23`	`Required setting, client_id not found in user configuration`	This error is raised when the required property `client_id` is not found in `config.json`, into `credentials` dictionary.	Add `client_id` property in `config.json`, into `credentials` dictionary.
`InitVariablesError`	`24`	`Required setting, client_id not of expected type: str`	This error is raised when `client_id` is defined in `config.json` but the format is not `str`.	Edit the value of `client_id` in `config.json`, into `credentials` dictionary, so it is of type `str`.
`InitVariablesError`	`25`	`Required setting, client_secret not found in user configuration`	This error is raised when the required property `client_secret` is not found in `config.json`, into `credentials` dictionary.	Add `client_secret` property in `config.json`, into `credentials` dictionary.
`InitVariablesError`	`26`	`Required setting, client_secret not of expected type: str`	This error is raised when `client_secret` is defined in `config.json` but the format is not `str`.	Edit the value of `client_secret` in `config.json`, into `credentials` dictionary, so it is of type `str`.
`InitVariablesError`	`27`	`Required setting, historic_date_utc not of expected type: str`	This error is raised when the optional value `historic_date_utc` defined in `config.json` is not of type `str`.	Edit the value of `historic_date_utc` in `config.json` so it is of type `str`. Or leave it empty so the collector starts pulling data N days ago at the current time.
`InitVariablesError`	`28`	`Time format for historic date must be <time_format>. e.g. 2022-02-15T14:32:33.043Z`	This error is raised when the optional value `historic_date_utc` defined in `config.json` does not match the indicated format.	Make the value of `historic_date_utc` in `config.json` match the indicated format. Or leave it empty so the collector starts pulling data N days ago at the current time.
`InitVariablesError`	`29`	`historic datetime cannot be greater than the present UTC time`	This error is raised when the optional value `historic_date_utc` defined in `config.json` is higher than the current time. Time is indicated in UTC time.	Make the value of `historic_date_utc` in `config.json` be a past value in UTC time. Or leave it empty so the collector starts pulling data N days ago at the current time.
`SetupError`	`100`	`Error occurred while requesting access token from the Wiz server. Error message: <error_message>`	This error is raised when a generic error occurs during the request to get the token to authenticate the collector in the API.	This is an internal issue. Contact with Devo Support team.
`SetupError`	`101`	`The credentials provided in the config file are incorrect. Please provide the correct credentials.` `Status code: 401` `Error type: <error_type>` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are not valid. (401 Unauthorized error)	Add the correct `client_id` and `client_secret` fields in `config.json`, at `credentials` property.
`SetupError`	`102`	`The credentials provided in the config file does not have necessary permissions to create access token.` `Status code: 403` `Error type: <error_type>` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but the credentials used to access the API have no permissions to create a token. (403 Forbidden error)	Add some credentials in `config.json` with enough privileges to create a token from the API.
`SetupError`	`103`	`The requested URL <URL> is not found. The URL may have been deprecated.` `Status code: 404` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but the authentication endpoint that is being requested to get a token is not found.	This is an internal issue. Contact with Devo Support team.
`SetupError`	`104`	`Unexpected error occurred while getting access token from the Wiz server` `Status code: <status_code>` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but an unexpected response has been returned from the API.	This is an internal issue. Contact with Devo Support team.
`SetupError`	`105`	`The credentials does not have valid permissions to fetch issues from the Wiz server`	This error is raised when the credentials provided in `config.json` are valid, but the credentials used have no permissions to access the API endpoint to get issues.	Check that the provided credentials have enough permission to retrieve issues from the API.
`SetupError`	`106`	`Failed to check if the provided credentials have valid permissions.` `Error message: <error_message>`	This error is raised when the credentials provided in `config.json` are valid, but there was an error in the process of checking if the provided credentials have permission to get issues.	This is an internal issue. Contact with Devo Support team.
`PullError`	`300`	`Error occurred while requesting issues from the Wiz server. Error message: <error_message>`	This error is raised when a generic error occurs during the request to get issues from the API.	This is an internal issue. Contact with Devo Support team.
`PullError`	`301`	`The token used to make this request is not valid anymore.` `Status code: 401` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is not valid anymore.	Check if the credentials need to be renewed. Add some valid credentials in `config.json`. If credentials are still valid, contact with Devo Support team.
`PullError`	`302`	`The access token does not have necessary permissions to fetch issues from Wiz.` `Status code: 403` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is valid, but it lost permissions to get issues from API.	Check what happened with credential permissions. Add some credentials in `config.json` with permissions to get issues. If credentials are valide and still have permissions, contact with Devo Support team.
`PullError`	`303`	`The requested URL <URL> is not found. The URL may have been depreciated` `Status code: 404` `Error message: <error_message>`	This error is raised when the token being used to make requests to the API is valid, but cannot find the endpoints to get issues.	This is an internal issue. Contact with Devo Support team.
`PullError`	`304`	`The server has returned <status_code> status code. The server may not be available for fetching`

issues. Try after sometime. Error message from server: <error_message>

issues. Try after sometime. Error message from server: <error_message>

This error is raised when the token being used to make requests to the API is valid, but there has been an error on Wiz's API.

The error is on Wiz’s side. Wiz can be contacted for more info.

It should work again when the incident at Wiz is solved.

PullError

305

Unexpected error occurred while getting issues from the Wiz server

Status code: <status_code>

Error message: <error_message>

This error is raised when the token being used to make requests to the API is valid, but there has been an unexpected return from the API.

This is an internal issue. Contact with Devo Support team.

PullError

306

After <retry_count> retries still getting the too many requests error.

This error is raised when the token being used to make requests to the API is valid, but

there has been an error on Wiz's API.

The error is on Wiz’s side. Wiz can be contacted for more info.

It should work again when the incident at Wiz is solved.

PullError

305

Unexpected error occurred while getting issues from the Wiz server

Status code: <status_code>

Error message: <error_message>

This error is raised when the token being used to make requests to the API is valid, but there has been an unexpected return from the API.

This is an internal issue. Contact with Devo Support team.

PullError

306

After <retry_count> retries still getting the too many requests error.

This error is raised when the token being used to make requests to the API is valid, but we are constantly receiving a 429 error response (too many requests)

Check throttle limitations on Wiz API and change the value of request_period_in_seconds and xxxxxxxx from config.json to adapt to it.

Page Comparison

Versions Compared

Old Version 16

New Version Current

Key

Data sources

Devo collector features

Flattening preprocessing

Devo collector features

Flattening preprocessing

How to enable the collection in the vendor

Editing the JSON configuration

Structure

Devo credentials

Editing the config.yaml file

Download the Docker image

Docker

Docker Compose

Collector service details

Setup output

Puller output

Structure

Devo credentials

Editing the config.yaml file

Download the Docker image

Docker

Docker Compose

Collector service details

Issue Service

Setup output

Puller output

Vulnerability Service

Setup output

Puller output

AuditLogs Service

Setup output

Puller output

CloudConfiguration Service

Setup output

Puller output

Custom Service

Setup output

Puller output

Initialization

Collector operations

Collector operations

Initialization

Events delivery and Devo ingestion

Events delivery and Devo ingestion

Sender services

Sender statistics

Sender services

Sender statistics

Change log

Change log

New Features

Improvements

New Features

Improvements