Page Comparison

...

Accepted authentication methods

Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

1. In the Collector Server GUI, access the domain in which you want this instance to be created

2. Click Add Collector and find the one you wish to add.

3. In the Version field, select the latest value.

4. In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).

5. In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.

6. In the Parameters section, establish the Collector Parameters as follows below:

{
  "global_overrides": {
    "debug": false
  },
  "inputs": {
    "google_workspace_logs_in_bigquery": {
       ├── state/"id": "<short_unique_id>",
      "enabled": true,
 └── config/      "credentials": {
        "service_account_info": "<service_account_info_value>"
  └── config.yaml 

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Image Removed

Editing the config.yaml file

globals:
  debug: false
  id: <collector_id_value>
  name: <collector_name_value>
  persistence:
    type: filesystem
    config:
      directory_name: state
outputs:
  devo_us_1:
    type: devo_platform
    config:
      address: <devo_address>
      port: 443
      type: SSL   },
      "dataset_id": "<dataset_id_value>",
      "environment": "<environment_value>",
      "services": {
        "activity_records": {
          "request_period_in_seconds": "<request_period_in_seconds_value>",
          "start_time_in_utc": "<start_time_in_utc_value>",
          "include_record_types": "<include_record_type_values>",
       chain: <chain_filename>  "exclude_record_types": "<exclude_record_type_values>",
   cert: <cert_filename>       key"override_tag": <key_filename>
inputs:"<override_tag_value>",
   google_workspace_logs_in_bigquery:     id: <short_unique_id>
    enabled: true
    credentials:
      service_account_info: <service_account_info_value>
    dataset_id: <dataset_id_value> "override_table_id": "<override_table_id_value>"
    environment: <environment_value>   }
 services:       activity_records:}
    }
   request_period_in_seconds: <request_period_in_seconds_value>
        start_time_in_utc: <start_time_in_utc_value>
        include_record_types: <include_record_type_values>
        exclude_record_types: <exclude_record_type_values>
        override_tag: <override_tag_value>
        override_table_id: <override_table_id_value>

Replace the placeholders with your required values following the description table below:

Parameter

Data type

Requirement

Value range / Format

Description

short_unique_id

str

Mandatory

Min length: 1, Max length: 5

Short, unique ID for input service, used in persistence addressing. Avoid duplicates to prevent collisions.

service_account_info

json

Mandatory

Min length: 1

The service account credential info. This is a JSON block downloadable from Google upon creating a service account credential. The service account must have read access to the BigQuery dataset that stores the activity records.

}
}

The following table outlines the parameters available for configuring the collector. Each parameter is categorized by its necessity (mandatory or optional), data type, acceptable values or formats, and a brief description.

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image

SHA-256 hash

collector-google_workspace_logs_in_bigquery-docker-image-1.0.0

37f402945ba2fbb8385d9a8c010e676802a0b55687b21eed5753e9ffc7dce9e3

Use the following command to add the Docker image to the system:

gunzip -c <image_file>-<version>.tgz | docker load

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

version: '3'
services:
  collector-<product_name>:
    image: <image_name>:${IMAGE_VERSION:-latest}
    container_name: collector-<product_name>
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

IMAGE_VERSION=<version> docker-compose up -d

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

1. In the Collector Server GUI, access the domain in which you want this instance to be created

2. Click Add Collector and find the one you wish to add.

3. In the Version field, select the latest value.

4. In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).

5. In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.

6. In the Parameters section, establish the Collector Parameters as follows below:

{
  "global_overrides": {
    "debug": false
  },
  "inputs": {
    "google_workspace_logs_in_bigquery": {
      "id": "<short_unique_id>",
      "enabled": true,
      "credentials": {
        "service_account_info": "<service_account_info_value>"
      },
      "dataset_id": "<dataset_id_value>",
      "environment": "<environment_value>",
      "services": {
        "activity_records": {
          "request_period_in_seconds": "<request_period_in_seconds_value>",
          "start_time_in_utc": "<start_time_in_utc_value>",
          "include_record_types": "<include_record_type_values>",
          "exclude_record_types": "<exclude_record_type_values>",
          "override_tag": "<override_tag_value>",
          "override_table_id": "<override_table_id_value>"
        }
      }
    }
  }
}

The following table outlines the parameters available for configuring the collector. Each parameter is categorized by its necessity (mandatory or optional), data type, acceptable values or formats, and a brief description.

Parameter

Data type

Requirement

Value range / Format

Description

short_unique_id

str

Mandatory

Min length: 1, Max length: 5

Short, unique ID for input service, used in persistence addressing. Avoid duplicates to prevent collisions.

service_account_info

json

Mandatory

Min length: 1

The service account credential info. This is a JSON block downloadable from Google upon creating a service account credential. The service account must have read access to the BigQuery dataset that stores the activity records.

environment_value

str

Optional

Min length: 1

Differentiates environments (e.g., dev, prod). Remove if unused.

dataset_id

str

Optional

Min length: 1

The BigQuery dataset ID that contains the activity data.

request_period_in_seconds_value

int

Optional

Min: 60

Custom period in seconds between data pulls, overriding default (300s).

start_time_in_utc_value

str

Optional

UTC datetime format: %Y-%m-%dT%H-%M-%SZ

Custom start date for data retrieval, for historical data download. Remove if unused.

include_record_types

list

Optional

List of strings. E.g. ["gmail", "login"]

If specified, the service will fetch only those record types defined in the list. The record types are present in the record_type field in the BigQuery table.

exclude_record_types

list

Optional

List of strings. E.g. ["gmail", "login"]

If specified, the service will exclude the record types defined in the list. The record types are present in the record_type field in the BigQuery table.

override_tag

str

Optional

Min length: 1

An optional tag to override the default tag for the Devo table.

override_table_id

str

Optional

Min length: 1

An optional override to support users whose activity data is stored in a table other than activity.

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

<any_directory>
└── devo-collectors/
    └── <product_name>/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── state/
        └── config/ 
            └── config.yaml 

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Image Added

Editing the config.yaml file

globals:
  debug: false
  id: <collector_id_value>
  name: <collector_name_value>
  persistence:
    type: filesystem
    config:
      directory_name: state
outputs:
  devo_us_1:
    type: devo_platform
    config:
      address: <devo_address>
      port: 443
      type: SSL
      chain: <chain_filename>
      cert: <cert_filename>
      key: <key_filename>
inputs:
  google_workspace_logs_in_bigquery:
    id: <short_unique_id>
    enabled: true
    credentials:
      service_account_info: <service_account_info_value>
    dataset_id: <dataset_id_value>
    environment: <environment_value>
    services:
      activity_records:
        request_period_in_seconds: <request_period_in_seconds_value>
        start_time_in_utc: <start_time_in_utc_value>
        include_record_types: <include_record_type_values>
        exclude_record_types: <exclude_record_type_values>
        override_tag: <override_tag_value>
        override_table_id: <override_table_id_value>

Replace the placeholders with your required values following the description table below:

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Use the following command to add the Docker image to the system:

gunzip -c <image_file>-<version>.tgz | docker load

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

version: '3'
services:
  collector-<product_name>:
    image: <image_name>:${IMAGE_VERSION:-latest}
    container_name: collector-<product_name>
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

IMAGE_VERSION=<version> docker-compose up -d

Collector services detail

...

This collector has different security layers that detect both an invalid configuration and abnormal operation. This table will help you detect and resolve the most common errors.

Collector operations

This section is intended to explain how to proceed with specific operations of this collector.

...