Page Comparison

Table of Contents

minLevel	2
maxLevel	2
type	flat

...

Here you will find the steps to filter this type of event:

Action

Steps

1

Activate Security Command Center service

Info
When SCC is activated, the events will go directly through the Logging service to the default sink. The following steps are optional but recommended to filter SCC events on another Pub/Sub.

In order to receive this type of event, it is necessary to have the Security Command Center service activated.

Refer to the Security Command Center Quickstart video from the Google guide.

2

Setting up a new topic

Refer to the Configuration of the Loggingsection to know how to do it.

3

Setting up a Pub/Sub

Refer to the Configuration of the Loggingsection to know how to do it.

4

Setting up a sink

Refer to the Configuration of the Loggingsection to know how to do it.

...

Note
You have to use the `scc_findigs` service to pull this data source.

	Action	Steps
1	Configure Identity and Access Management (IAM) roles.	Refer to the official Google guide in which additional configurations are described.
2	Activate the Security Command Center API.
3	Setting up a Pub/Sub topic.
4	Creating a Notification configuration.

...

Authentication method

Project ID

Base64 credentials

File credentials

Available on

1

Service account with Base64.

Status

colour	Green
title	Required

Status

colour	Green
title	Required

Collector Server
On-Premise

2

Service account with the file credentials.

Status

colour	Green
title	Required

Status

colour	Green
title	Required

On-Premise

...

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).

<any_directory> └── devo-collectors/ └── <product_name>/ ├── certs/ │ ├── chain.crt │ ├── <your_domain>.key

Rw ui tabs macro

│ └── <your_domain>.crt

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

Code Block

Cloud collector

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

In the Collector Server GUI, access the domain in which you want this instance to be created
Click Add Collector and find the one you wish to add.
In the Version field, select the latest value.
In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).
In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.
In the Parameters section, establish the Collector Parameters as follows below:

Editing the JSON configuration

Code Block

{
  "global_overrides": {
    "debug": false
  },
  "inputs": {
    "gcp": {
      "id": "<short_unique_id>",
      "enabled": <input_status>,
      "requests_per_second": <requests_per_second>,
      "credentials": {
         ├── state/"source_id": "<source_id_value>",
         └── config/"project_id": "<project_id_value>",
        "filename": "<filename_value>",
    └── config.yaml

Note
Replace `<product_name>` with the proper value.

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Image Removed

Note
Replace `<product_name>` with the proper value.

Editing the config.yaml file

Code Block

globals:
  debug: false
  id: <collector_id>
  name: <collector_name>
  persistence:
    type: filesystem
    config:
   "file_content_base64": "<file_content_base64>"
      },
      "services": {
        "logging": {
     directory_name: state outputs:   devo"subscription_1name":     type: devo_platform
"<subscription_id>",
   config:       address"tag": <devo_address>"<custom_tag>",
      port: 443   "auto_categorization_method": "<auto_categorization_method>",
  type: SSL       chain"request_period_in_seconds": <chain_filename><request_period_in_seconds>,
      cert: <cert_filename>       key: <key_filename>
inputs:
  gcp:"enable_pubsub_undelivered_messages_logging": <enable_pubsub_undelivered_messages_logging>,
         id: <short"pubsub_unique_id>
    enabled: <input_status>
    requests_per_second: <requests_per_second>
    credentials:
      source_id: <source_id_value>
      project_id: <project_id_value>undelivered_messages_request_interval_in_seconds": <pubsub_undelivered_messages_request_interval_in_seconds>
      filename: <filename_value> },
     file_content_base64: <file_content_base64_value>
    services:
    "scc-findings": {
   logging:         "subscription_name": "<subscription_id>",
          "tag": "<custom_tag>
        auto_categorization_method: <auto_categorization_method>",
          "request_period_in_seconds": <request_period_in_seconds>
        enable_pubsub_undelivered_messages_logging: <enable_pubsub_undelivered_messages_logging>}
      }
  pubsub_undelivered_messages_request_interval_in_seconds: <pubsub_undelivered_messages_request_interval_in_seconds>
 }
     scc-findings:
        subscription_name: <subscription_id>
        tag: <custom_tag>
        request_period_in_seconds: <request_period_in_seconds>}
}

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Replace Please replace the placeholders with your required real world values following the description table below:

Parameter

Data

Type

type

Type

Value

Range

range / Format

Details

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

input_status

bool

Mandatory

false / true

Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.

requests_per_second

int

Optional

Minimum value: 1

Customize the maximum number of API requests per second. If not used, the default setting will be used: 100000 requests/sec.

info

This parameter should be removed if it is not used.

source_id_value

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to assign a custom name for identifying the environment of the infrastructure.

Info
This `source_id` will be added to all events ingested by this collector in the field `@devo_source_id` and helps you to filter based on this entity.

project_id_value

str

Mandatory

Minimum length: 1

The name of the GCP project. The Configuration of the Logging service section guide shows how to get this value.

Info
This `project_id` will be added to all events ingested by this collector in the field `@devo_project_id` and helps you to filter based on this entity.

filename_value

str

Optional

Minimum length: 1

The file name with the account credentials. The Configuration of the Logging service section guide shows how to get this value.

Info
This parameter is optional, but you must select one between `filename_value` and `file_content_base64_value`.

info

This parameter should be removed if it is not used.

file_content_base64_value

str

Optional

Minimum length: 1

The Service Account credentials in base64. The Configuration of the Logging service section guide shows how to get this value.

Info
This parameter is optional, but you must select one between `filename_value` and `file_content_base64_value`.

info

This parameter should be removed if it is not used.

subscription_id

str

Mandatory

Minimum length: 1

The ID of the Pub/sub subscription. The Configuration of the Logging service section guide shows how to get this value.

custom_tag

str

Optional

A Devo tag. See for more information see Devo tags

InfoRecommended

Recommended custom_tag for the scc-findings service:
cloud.gcp.scc.findings

There are three possible scenarios:

Send all events to custom tag: Add the desired value to the custom_tag
Send the events to the default tag in the Logging service: Do not use the custom_tag
Send SCC findings events to their corresponding tag: add the cloud.gcp.scc.findings value to the custom tag in scc-findings service.

Info

If a custom_tag is not defined, all events that are not in JSON format will be ingested to the tag cloud.gcp.unknown.none

Info

This parameter should be removed if it is not used.

auto_categorization_method

str

Optional

It can have one of these two values:

logname_field
resource_type_field

The default value of this parameter islogname_field. It is used only to set the auto-categorization method to resource_type_field.

Info
Visit the Devo categorization and destination section for details.

info

This parameter should be removed if it is not used.

request_period_in_seconds

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (60 seconds)

Info

This parameter should be removed if it is not used.

enable_pubsub_undelivered_messages_logging

bool

Optional

false / true

Use this param to enable/disable the undelivered messages logging feature, that will show how many undelivered messages remains in the GCP pubsub queue every N data pulls.
Default value: false

Note
Bear in mind that this feature uses the monitoring API, that might have a cost if we exceed a certain number of requests. More information here.

info

This parameter should be removed if it is not used.

pubsub_undelivered_messages_request_interval_in_seconds

int

Optional

Minimum value: 1

This param, defines (when enabled) every how many seconds will be trigered the undelivered messages log line.
Default value: 300

Note
Bear in mind that this feature uses the monitoring API, that might have a cost if we exceed a certain number of requests. More information here.

info

This parameter should be removed if it is not used.

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image

SHA-256 hash

collector-gcp_collector-docker-image-1.7.0.tgz

febf2e2aecdd8dc5ebf2561611fbb3bcdbbd8a995a1e922da2932b1625b7b5ea

Use the following command to add the Docker image to the system:

Code Block
gunzip -c <image_file>-<version>.tgz \| docker load

Note
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

Code Block

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

Code Blockversion: '3' services: collector-<product_name>: image: <image_name>:${IMAGE_VERSION:-latest} container_name: collector-<product_name> volumes: - ./certs:/devo-collector/certs

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

Code Block

<any_directory>
└── devo-collectors/
    └── <product_name>/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── state/
        - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials└── config/ 
     - ./state:/devo-collector/state     environment:       - CONFIG_FILE=${CONFIG_FILE:-└── config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

Code Block
IMAGE_VERSION=<version> docker-compose up -d

Note
Replace `<product_name>` with the proper value.

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Image Added

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper valuesvalue.

Rw tab

title	Cloud collector

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

In the Collector Server GUI, access the domain in which you want this instance to be created
Click Add Collector and find the one you wish to add.
In the Version field, select the latest value.
In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).
In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.
In the Parameters section, establish the Collector Parameters as follows below:

Editing the JSON configuration

Code Block{ "global_overrides": { "debug": false }, "inputs": { "gcp": {

Editing the config.yaml file

Code Block

globals:
  debug: false
  id: <collector_id>
  name: <collector_name>
  persistence:
    type: filesystem
    config:
       "id": "<short_unique_id>",
 directory_name: state
outputs:
  devo_1:
    "enabled"type: <input_status>,devo_platform
    config:
      "requests_per_second"address: <requests_per_second>,<devo_address>
      "credentials": {
 port: 443
      type: SSL
      "source_id"chain: "<source_id_value>",<chain_filename>
        "project_id"cert: "<project_id_value>",<cert_filename>
        "filename"key: "<filename_value>",
<key_filename>
inputs:
  gcp:
    "file_content_base64"id: "<file<short_contentunique_base64>"id>
    enabled: <input_status>
},    requests_per_second: <requests_per_second>
 "services": {  credentials:
      "logging"source_id: {<source_id_value>
          "subscription_name": "<subscription_id>",project_id: <project_id_value>
          "tag"filename: "<custom_tag>",<filename_value>
          "auto_categorization_method": "<auto_categorization_method>",file_content_base64: <file_content_base64_value>
    services:
      "logging:
        subscription_name: <subscription_id>
        tag: <custom_tag>
        auto_categorization_method: <auto_categorization_method>
        request_period_in_seconds": <request_period_in_seconds>,
          "enable_pubsub_undelivered_messages_logging": <enable_pubsub_undelivered_messages_logging>,
 
        "pubsub_undelivered_messages_request_interval_in_seconds": <pubsub_undelivered_messages_request_interval_in_seconds>
    
   },         "scc-findings": {
 
        "subscription_name": "<subscription_id>",
 
        "tag": "<custom_tag>",
 
        "request_period_in_seconds": <request_period_in_seconds>
        }
      }
    }
  }
}

Info

All defined service entities will be executed

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Please replace Replace the placeholders with real world your required values following the description table below:

Parameter

Data

type

Type

Value

range

Range / Format

Details

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

input_status

bool

Mandatory

false / true

Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.

requests_per_second

int

Optional

Minimum value: 1

Customize the maximum number of API requests per second. If not used, the default setting will be used: 100000 requests/sec.

Info
This parameter should be removed if it is not used.

source_id_value

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to assign a custom name for identifying the environment of the infrastructure.

Info
This `source_id` will be added to all events ingested by this collector in the field `@devo_source_id` and helps you to filter based on this entity.

project_id_value

str

Mandatory

Minimum length: 1

The name of the GCP project. The Configuration of the Logging service section guide shows how to get this value.

Info
This `project_id` will be added to all events ingested by this collector in the field `@devo_project_id` and helps you to filter based on this entity.

filename_value

str

Optional

Minimum length: 1

The file name with the account credentials. The Configuration of the Logging service section guide shows how to get this value.

Info
This parameter is optional, but you must select one between `filename_value` and `file_content_base64_value`.

Info
This parameter should be removed if it is not used.

file_content_base64_value

str

Optional

Minimum length: 1

The Service Account credentials in base64. The Configuration of the Logging service section guide shows how to get this value.

Info
This parameter is optional, but you must select one between `filename_value` and `file_content_base64_value`.

Info
This parameter should be removed if it is not used.

subscription_id

str

Mandatory

Minimum length: 1

The ID of the Pub/sub subscription. The Configuration of the Logging service section guide shows how to get this value.

custom_tag

str

Optional

A Devo tag. See for more information see Devo tags

Info
Recommended custom_tag for the `scc-findings` service: `cloud.gcp.scc.findings`

There are three possible scenarios:

Send all events to custom tag: Add the desired value to the custom_tag
Send the events to the default tag in the Logging service: Do not use the custom_tag
Send SCC findings events to their corresponding tag: add the cloud.gcp.scc.findings value to the custom tag in scc-findings service.

Info
If a custom_tag is not defined, all events that are not in `JSON` format will be ingested to the tag `cloud.gcp.unknown.none`

Info
This parameter should be removed if it is not used.

auto_categorization_method

str

Optional

It can have one of these two values:

logname_field
resource_type_field

The default value of this parameter islogname_field. It is used only to set the auto-categorization method to resource_type_field.

Info
Visit the Devo categorization and destination section for details.

Info
This parameter should be removed if it is not used.

request_period_in_seconds

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (60 seconds)

Info
This parameter should be removed if it is not used.

enable_pubsub_undelivered_messages_logging

bool

Optional

false / true

Use this param to enable/disable the undelivered messages logging feature, that will show how many undelivered messages remains in the GCP pubsub queue every N data pulls.
Default value: false

Note
Bear in mind that this feature uses the monitoring API, that might have a cost if we exceed a certain number of requests. More information here.

Info
This parameter should be removed if it is not used.

pubsub_undelivered

_messages_request_interval_in_seconds

int

Optional

Minimum value: 1

This param, defines (when enabled) every how many seconds will be trigered the undelivered messages log line.
Default value: 300

Note
Bear in mind that this feature uses the monitoring API, that might have a cost if we exceed a certain number of requests. More information here.

This parameter should be removed if it is not used

_messages_request_interval_in_seconds

int

Optional

Minimum value: 1

This param, defines (when enabled) every how many seconds will be trigered the undelivered messages log line.
Default value: 300

Note
Bear in mind that this feature uses the monitoring API, that might have a cost if we exceed a certain number of requests. More information here.

Info
This parameter should be removed if it is not used.

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image	SHA-256 hash
collector-gcp_collector-docker-image-1.7.0.tgz	`febf2e2aecdd8dc5ebf2561611fbb3bcdbbd8a995a1e922da2932b1625b7b5ea`

Use the following command to add the Docker image to the system:

Code Block
gunzip -c <image_file>-<version>.tgz \| docker load

Note
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

Code Block

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

Code Block

version: '3'
services:
  collector-<product_name>:
    image: <image_name>:${IMAGE_VERSION:-latest}
    container_name: collector-<product_name>
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

Code Block
IMAGE_VERSION=<version> docker-compose up -d

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Collector services detail

...

Versions Compared

Old Version 30

New Version Current

Key

Structure

Editing the JSON configuration

Devo credentials

Editing the config.yaml file

Download the Docker image

Docker

Docker Compose

Structure

Devo credentials

Editing the JSON configuration

Editing the config.yaml file

Download the Docker image

Docker

Docker Compose

Collector services detail