Wiz collector

[ 1 Overview ] [ 2 Data sources ] [ 3 Devo collector features ] [ 4 Flattening preprocessing ] [ 5 How to enable the collection in the vendor ] [ 6 Minimum configuration required for basic pulling ] [ 7 Accepted authentication methods ] [ 8 Run the collector ] [ 9 Collector service details ] [ 10 Collector operations ] [ 11 Change log ]

Overview

Wiz is a cloud infrastructure security tool that provides organizations with an in-depth contextual risk assessment. Wiz’s agentless solution builds inventory, and scans for varied risk factors such as vulnerabilities, excessive permissions, malware, exposed secrets, practical exposure, and more, and prioritizes the alerts for the security teams based on the likelihood to be exploited and potential business impact.

The Devo Wiz collector allows customers to retrieve Wiz cloud security issues into Devo to query, correlate, analyze, and visualize to enable Enterprise IT and Cybersecurity teams to take the most impactful decisions at the petabyte scale. The collector processes the Wiz API responses and sends them to the Devo platform, which then categorizes all data received on tables along rows and columns in your Devo domain.

Data sources

Data source	Description	API Endpoint	Collector service name	Devo table	Available from release

Data source	Description	API Endpoint	Collector service name	Devo table	Available from release
Issues	An issue in wiz is a vulnerability that is detected in the cloud infrastructure	`/graphql`	`issues`	`cspm.wiz.issues.default`	`v1.0.0`
Vulnerability	Vulnerabilities are weaknesses in computer systems that can be exploited by malicious attackers. Whether they are caused by bugs or design flaws, vulnerabilities can allow attackers to execute code in an environment or elevate privileges.	`/graphql`	`vulnerabilities`	`cspm.wiz.vulnerabilities.default`	`v1.5.0`
Audit Logs	The Audit Log records key events in Wiz, such as login, logout, and user update. The Audit Log is primarily used to investigate potentially suspicious activity or diagnose and troubleshoot errors.	`/graphql`	`auditLogs`	`cspm.wiz.audit.default`	`v1.5.0`
Cloud Configuration Findings	This returns the problems with configurations and the remediation solutions for the same.	`/graphql`	`cloudConfiguration`	`cspm.wiz.cloud_configuration.default`	`v1.5.0`
Custom Service	This provides an option to add custom graphql query in the config and ingest data.	`/graphql`	`custom_query`	`my.app.wiz.custom_query` (default) User can provide override tag in the config if the parser is deployed for their custom query or if they want a different table in my.app .	`v1.7.0`

Devo collector features

Feature	Details

Feature	Details
Allow parallel downloading (`multipod`)	`not allowed`
Running environments	`collector server` `on-premise`
Populated Devo events	`table`
Flattening preprocessing	`yes`

Flattening preprocessing

In order to improve the data exploitation and enrichment, this collector applies some flattening actions to the collected data before delivering it to Devo:

Data source	Collector service	Optional	Flattening details

Data source	Collector service	Optional	Flattening details
Issues	`issues`	`No`	The `control` key content is transferred to the first JSON level with the prefix `control_`. The `entity` key content is transferred to the first JSON level with the prefix `entity_`. The `entitySnapshot` key content is transferred to the first JSON level with the prefix `entitySnapshot_`.
Vulnerabilities	`vulnerabilities`	`Yes`	The `layer` key content is transferred to the first json level with the prefix `layer_`. The `vulnerable_asset` key content is transferred to the first json level with the prefix `asset_`.
Audit Logs	`auditLogs`	`Yes`	The `action_parameters` key content is transferred to the first json level with the prefix `action_`.
Cloud Configuration Findings	`cloudConfiguration`	`Yes`	The `resource` key content is transferred to the first json level with the prefix `resource_`.
Custom Service	`custom_query`	`Yes`	N/A

How to enable the collection in the vendor

Minimal requirements to follow this guide

In order to retrieve the data, the following details will be required from your Wiz instance.

Instance domain	Wiz domain of your cloud instance where the collector will make the requests.
Client ID	Wiz user ID.
Client secret	Wiz user passwords.

Minimum configuration required for basic pulling

Although this collector supports advanced configuration, the fields required to download data with basic configuration are defined below.

This minimum configuration refers exclusively to the specific parameters of this integration. There are more required parameters related to the generic behavior of the collector. Check setting sections for details.

Setting	Details

Setting	Details
`override_api_base_url`	By default, the base URLs `https://api.us1.app.wiz.io`. This parameter allows you to customize the base URL and is mandatory when the customer URL is different than the given default value.
`client_id`	User Client ID to authenticate to the service.
`client_secret`	User Secret Key to authenticate to the service.

Accepted authentication methods

The following are the accepted authentication methods for this collector.

Authentication method	Client ID	Client secret

Authentication method	Client ID	Client secret
Basic authentication	REQUIRED	REQUIRED

Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).

Collector service details

Issue Service

All events of this service are ingested into the table cspm.wiz.issues.default

Issue service is based on the following GraphQL command:

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
    action {
      id
      type
    }
  }
}

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Note that the PrePull action is executed only one time before the first run of the Pull action.

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Updating deduplication buffers content
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Terminated
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Data collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

The value @devo_pulling_id is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that Pull action in Devo’s search window.

Vulnerability Service

All events of this service are ingested into the table cspm.wiz.vulnerabilities.default