Page Comparison

...

The Devo Wiz collector allows customers to retrieve Wiz cloud security issues into Devo to query, correlate, analyze, and visualize to enable Enterprise IT and Cybersecurity teams to take the most impactful decisions at the petabyte scale. The collector processes the Wiz API responses and sends them to the Devo platform, which then categorizes all data received on tables along rows and columns in your Devo domain.

Data sources

Data source	Description	API Endpoint	Collector service name	Devo table	Available from release
Issues	An issue in wiz is a vulnerability that is detected in the cloud infrastructure	`/graphql`	`issues`	`cspm.wiz.issues.default`	`v1.0.0`
Vulnerability	Vulnerabilities are weaknesses in computer systems that can be exploited by malicious attackers. Whether they are caused by bugs or design flaws, vulnerabilities can allow attackers to execute code in an environment or elevate privileges.	`/graphql`	`vulnerabilities`	`cspm.wiz.vulnerabilities.default`	`v1.5.0`
Audit Logs	The Audit Log records key events in Wiz, such as login, logout, and user update. The Audit Log is primarily used to investigate potentially suspicious activity or diagnose and troubleshoot errors.	`/graphql`	`auditLogs`	`cspm.wiz.audit.default`	`v1.5.0`
Cloud Configuration Findings	This returns the problems with configurations and the remediation solutions for the same.	`/graphql`	`cloudConfiguration`	`cspm.wiz.cloud_configuration.default`	`v1.5.0`

Devo collector features

Feature	Details
Allow parallel downloading (`multipod`)	`Not allowed`
Running environments	`Collector Server`, `On Premise`
Populated Devo events	`Table`
Flattening preprocessing	`Yes`

...

Collector service details

Issue Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.issues.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
    action {
      id
      type
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Updating deduplication buffers content
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Terminated
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Data collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

Vulnerability Service

Issue Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.issues.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
    action {
      id
      type
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Updating deduplication buffers content
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Terminated
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Data collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

AuditLogs Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.issues.default

Expand

title	Used GraphQL command

Issue service is based on the following GraphQL command:

Code Block

query IssuesTable(
  $filterBy: IssueFilters
  $first: Int
  $after: String
  $orderBy: IssueOrder
) {
  issues(
    filterBy: $filterBy
    first: $first
    after: $after
    orderBy: $orderBy
  ) {
    nodes {
      ...IssueDetails
    }
    pageInfo {
      hasNextPage
      endCursor
    }
    totalCount
    informationalSeverityCount
    lowSeverityCount
    mediumSeverityCount
    highSeverityCount
    criticalSeverityCount
    uniqueEntityCount
  }
}

fragment IssueDetails on Issue {
  id
  control {
    id
    name
    query
    securitySubCategories {
      id
      title
      category {
        id
        name
        framework {
          id
          name
        }
      }
    }
  }
  createdAt
  updatedAt
  projects {
    id
    name
    slug
    businessUnit
    riskProfile {
      businessImpact
    }
  }
  status
  severity
  entity {
    id
    name
    type
  }
  entitySnapshot {
    id
    type
    nativeType
    name
    subscriptionId
    subscriptionExternalId
    subscriptionName
    resourceGroupId
    resourceGroupExternalId
    region
    cloudPlatform
    cloudProviderURL
    providerId
    status
    tags
    subscriptionTags
  }
  note
  serviceTicket {
    externalId
    name
    url
  }
  serviceTickets {
    externalId
    name
    url
    action {
      id
      type
    }
  }
}

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Updating deduplication buffers content
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Terminated
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Data collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

CloudConfiguration Service

Expand

title	Devo categorization and destination

All events of this service are ingested into the table cspm.wiz.issues.default

...

Expand

title	Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component	Description
Setup	The setup module is in charge of authenticating the service and managing the token expiration when needed.
Puller	The setup module is in charge of pulling the data in a organized way and delivering the events via SDK.

Setup output

A successful run has the following output messages for the setup module:

Code Block

INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Started
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> successfully generated new access token
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> The credentials provided in the configuration have required permissions to request issues from Wiz server
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Puller Setup Terminated
INFO InputProcess::WizDataPullerSetup(wiz_collector,wiz_data_puller#111,issues#predefined) -> Setup for module <WizDataPuller> has been successfully executed

Puller output

A successful initial run has the following output messages for the puller module:

Info
Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Started.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> User has specified 2022-01-01 00:00:00 as the datetime. Historical polling will consider this datetime for creating the default values.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> No saved state found, initializing with state: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
WARNING InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Saved state loaded: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(1970, 1, 1, 0, 0), 'buffer_ids_with_duplication_risk': []}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> PrePull Terminated
2INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Starting data collection every 60 seconds
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Started
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Fetching for issues from 2022-01-01T00:00:00
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Total number of issues in this poll: 45
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 5, 12, 19, 13, 20, 193191), 'buffer_ids_with_duplication_risk': ['09992ee4-1450-44fa-951c-d5fc4815473a']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 1; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 20.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO OutputProcess::SyslogSender(standard_senders,syslog_sender_0) -> syslog_sender_0 -> Created sender: {"client_name": "collector-4ac42f93cffaa59c-9dc9f67c9-cgm84", "url": "sidecar-service-default.integrations-factory-collectors:601", "object_id": "140446617222352"}
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 20 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 9, 0, 1, 927011), 'buffer_ids_with_duplication_risk': ['87e301c5-d3b7-4c2b-9495-9163772b3517', '7c95e45f-694e-4843-8aa7-d697a66fb14a', '5f3daede-c375-424f-9034-d9f423310b4a', '584ac078-87f2-45a5-b2eb-6e72e0594bd7', '5057cb24-ce5b-405d-bd5d-fd7b3ba70fc0', '22933fcb-ebb0-4a03-bb00-c1cba0b5abca', '1bed50e0-7825-41c9-a9de-8d32e0a35de8', '03a303c8-000c-4544-8f2c-65486a225e15']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Requesting Wiz API for issues
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> successfully retried issues from Wiz
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Removing the duplicate issues if present
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Flatten data is set to True. Flattening the data and adding 'devo_pulling_id' to events
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Delivering issues to the SDK
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> 5 issues delivered
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> State has been updated during pagination: {'historic_date_utc': datetime.datetime(2022, 1, 1, 0, 0), 'last_polled_timestamp': datetime.datetime(2022, 1, 1, 0, 0), 'ids_with_same_timestamp': [], 'buffer_timestamp_with_duplication_risk': datetime.datetime(2022, 6, 30, 13, 14, 40, 673424), 'buffer_ids_with_duplication_risk': ['4d819843-61ef-4e70-a2b6-5834a3f96403']}.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Updating deduplication buffers content
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Pull Terminated
INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Data collection completed. Elapsed time: 1.334 seconds. Waiting for 58.666 second(s)

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Code Block

INFO InputProcess::WizDataPuller(wiz_data_puller,00011,issues,predefined) -> Statistics for this pull cycle (@devo_pulling_id=1656602793.044179):Number of requests made: 3; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 45; Average of events per second: 33.797.

Info
The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.

Note that a Partial Statistics Report will be displayed when pagination is required in order to pull all available events. Look for the report without the Partial reference.

(Partial) Statistics for this pull cycle (@devo_pulling_id=1656602793.044179) so far: Number of requests made: 2; Number of events received: 45; Number of duplicated events filtered out: 0; Number of events generated and sent: 40.

...

Versions Compared

Old Version 17

New Version 18

Key

Data sources

Devo collector features

Collector service details

Issue Service

Setup output

Puller output

Vulnerability Service

Issue Service

Setup output

Puller output

AuditLogs Service

Setup output

Puller output

CloudConfiguration Service

Setup output

Puller output