Content Comparison

...

Rw ui tabs macro

Rw tab

title	Cloud collector

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

In the Collector Server GUI, access the domain in which you want this instance to be created
Click Add Collector and find the one you wish to add.
In the Version field, select the latest value.
In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).
In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.
In the Parameters section, establish the Collector Parameters as follows below:

Editing the JSON configuration

Code Block

{
  "global_overrides": {
    "debug": false
  },
  "sendmarc": {
    "id": "<short_unique_id>",
    "enabled": true,
    "credentials": {
      "user_id": "<user_id>",
      "api_key": "<api_key>",
      "accounts_url": "<opt_accounts_url>"
    },
    "services": {
      "senders": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
        "include_sender_data": "<opt_include_sender_data>",
        "include_sender_ip_address_data": "<opt_include_sender_ip_address_data>",
        "override_devo_tag": "<opt_override_devo_tag>",
        "override_devo_sender_tag": "<opt_override_devo_sender_tag>",
        "override_devo_sender_ip_address_tag": "<opt_override_devo_sender_ip_address_tag>"
      },
      "ip_address": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
        "include_ip_address_source": "<opt_include_ip_address_source>"
        "include_ip_address_aggregate_records": "<opt_include_ip_address_aggregate_records>",
        "override_devo_tag": "<opt_override_devo_tag>", 
        "override_devo_ip_address_source_tag": "<opt_override_devo_ip_address_source_tag>",
        "override_devo_ip_address_aggregate_tag": "<opt_override_devo_ip_address_aggregate_tag>"
        },
      "sts_settings": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
        "override_devo_tag": "<opt_override_devo_tag>"
        },
      "spf_settings": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
        "override_devo_tag": "<opt_override_devo_tag>"
        },
      "dmarc_settings": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
        "override_devo_tag": "<opt_override_devo_tag>"
        },
       "domains_dkim": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
         "override_devo_tag": "<opt_override_devo_tag>",
         "include_public_key_data": "<opt_include_public_key_data>",
        "override_devo_public_key_tag": "<opt_override_devo_public_key_tag>"
        },
       "domains_bimi": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
         "override_devo_tag": "<opt_override_devo_tag>",
         "include_bimi_selector_data": "<opt_include_bimi_selector_data>",
         "override_devo_bimi_selector_tag": "<opt_override_devo_bimi_selector_tag>"
        },
       "dmarc_volume_timeline": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
         "override_devo_tag": "<opt_override_devo_tag>"
        },
       "dmarc_volume_total": {
        "request_period_in_seconds": "<opt_request_period_in_seconds>",
         "override_devo_tag": "<opt_override_devo_tag>",
         "group_values": "<group_values>",
        "include_total_group_data": "<opt_include_total_group_data>",
        "override_devo_total_group_tag": "<opt_override_devo_total_group_tag>"
        }
     }
  }
}

Parameter	Data type	Type	Value range / Format	Details
<short_unique_id>	int	Mandatory	Minimum Length 5	Use this param to give a unique id to this input service. This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.
<input_status>	bool	Mandatory	false / true	Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.
<user_id>	str	Mandatory	Minimum Length 1	user_id for Sendmarc API
<api_key>	str	Mandatory	Minimum Length 1	api_key for Sendmarc API
<opt_accounts_url>	str	Optional	Minimum Length 1	If provided it will override the default sendmarc `accounts_url`. ex: https://<sendmarc_url>/api/v1/accounts
<opt_override_devo_tag>	str	Optional	A devo tag	This parameter allows defining a custom devo tag.
<opt_request_period_in_seconds>	int	Optional	Minimum Length 1	Period in seconds used between each data pulling, this value will override the value. The default value is 86400 seconds
<opt_include_sender_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/senders/{sender}`
<opt_include_sender_ip_address_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/senders/{sender}/ip-addresses`
<opt_include_ip_address_source>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/ip-addresses/{source}`
<opt_include_total_group_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/volume/totals/grouped/{grouping}`
<opt_include_bimi_selector_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/bimi-selectors/{bimiSelector}`
<opt_include_public_key_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/dkim-public-keys/{dkimPublicKey}`
<opt_include_ip_address_aggregate_records>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/ip-addresses/{source}/aggregate-report-records`
<group_values>	array	Optional/Mandatory	Minimum Length 1	Array of available group_values. ex:- ["country"] . This field will be mandatory if `opt_include_total_group_data` is true
<opt_override_devo_sender_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `statistics/dmarc/senders/{sender}`
<opt_override_devo_sender_ip_address_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/senders/{sender}/ip-addresses`
<opt_override_devo_ip_address_source_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/ip-addresses/{source}`
<opt_override_devo_ip_address_aggregate_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/ip-addresses/{source}/aggregate-report-records`
<opt_override_devo_bimi_selector_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/bimi-selectors/{bimiSelector}`
<opt_override_devo_public_key_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/dkim-public-keys/{dkimPublicKey}`
<opt_override_devo_total_group_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/volume/totals/grouped/{grouping}`

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

Code Block

<any_directory>
└── devo-collectors/
    └── <product_name>/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── state/
        └── config/ 
            └── config.yaml

Note
Replace `<product_name>` with the proper value.

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Note
Replace `<product_name>` with the proper value.

Editing the config.yaml file

Code Block

globals:
  debug: false
  id: not_used
  name: sendmarc
  persistence:
    type: filesystem
    config:
      directory_name: state

outputs:
  devo_1:
    type: devo_platform
    config:
      address: collector-us.devo.io
      port: 443
      type: SSL
      chain: chain.crt
      cert: <devo_domain>.crt
      key: <devo_domain>.key
  console_1:
    type: console

inputs:
  sendmarc:
    id: <short_unique_id>
    enabled: true
    credentials:
      user_id: <user_id>
      api_key: <api_key>
      accounts_url: <account_url> #optional
    services:
      senders:
        request_period_in_seconds : <request_period_in_seconds> #optional
        include_sender_data: <include_sender_data> #optional
        include_sender_ip_address_data: <include_sender_ip_address_data> #optional
        override_devo_tag: <override_devo_tag> #optional
        override_devo_sender_tag: <override_devo_sender_tag> #optional
        override_devo_sender_ip_address_tag: <override_devo_sender_ip_address_tag> #optional
      ip_address:
        request_period_in_seconds: <request_period_in_seconds> #optional
        include_ip_address_source: <include_ip_address_source> #optional
        include_ip_address_aggregate_records: <include_ip_address_aggregate_records> #optional
        override_devo_tag: <override_devo_tag> #optional
        override_devo_ioverride_devo_bimi_selector_tagp_address_source_tag: <override_devo_ip_address_source_tag> #optional
        override_devo_ip_address_aggregate_tag: <override_devo_ip_address_aggregate_tag> #optional
      sts_settings:
        request_period_in_seconds: <request_period_in_seconds> #optional
        override_devo_tag: <override_devo_tag> #optional
      spf_settings:
        request_period_in_seconds: <request_period_in_seconds> #optional
        override_devo_tag: <override_devo_tag> #optional
      dmarc_settings:
        request_period_in_seconds: <request_period_in_seconds> #optional
        override_devo_tag: <override_devo_tag> #optional
      domains_dkim:
        request_period_in_seconds: <request_period_in_seconds> #optional
        override_devo_tag: <override_devo_tag> #optional
        include_public_key_data: <include_public_key_data> #optional
        override_devo_public_key_tag: <override_devo_public_key_tag> #optional
      domains_bimi:
        request_period_in_seconds: <request_period_in_seconds> #optional
        include_bimi_selector_data: <include_bimi_selector_data> #optional
        override_devo_tag: <override_devo_tag> #optional
        override_devo_bimi_selector_tag: <override_devo_bimi_selector_tag> #optional
      dmarc_volume_timeline:
        request_period_in_seconds: <request_period_in_seconds> #optional
        override_devo_tag: <override_devo_tag> #optional
      dmarc_volume_total:
        group_values: <group_values> # mandatory if include_total_group_data is true
        include_total_group_data: <include_total_group_data>
        request_period_in_seconds: <request_period_in_seconds> #optional
        override_devo_tag: <override_devo_tag> #optional
        override_devo_total_group_tag: <override_devo_total_group_tag>

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Replace the placeholders with your required values following the description table below:

Parameter	Data type	Type	Value range / Format	Details
<short_unique_id>	int	Mandatory	Minimum Length 5	Use this param to give a unique id to this input service. This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.
<input_status>	bool	Mandatory	false / true	Use this param to enable or disable the given input logic when running the collector. If the value is true, the input will be run. If the value is false, it will be ignored.
<user_id>	str	Mandatory	Minimum Length 1	user_id for Sendmarc API
<api_key>	str	Mandatory	Minimum Length 1	api_key for Sendmarc API
<accounts_url>	str	Optional	Minimum Length 1	If provided it will override the default sendmarc `accounts_url`. ex: https://<sendmarc_url>/api/v1/accounts
<override_devo_tag>	str	Optional	A devo tag	This parameter allows defining a custom devo tag.
<request_period_in_seconds>	int	Optional	Minimum Length 1	Period in seconds used between each data pulling, this value will override the value. The default value is 86400 seconds
<include_sender_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/senders/{sender}`
<include_sender_ip_address_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/senders/{sender}/ip-addresses`
<include_ip_address_source>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/ip-addresses/{source}`
<include_total_group_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/volume/totals/grouped/{grouping}`
<include_bimi_selector_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/bimi-selectors/{bimiSelector}`
<include_public_key_data>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/dkim-public-keys/{dkimPublicKey}`
<include_ip_address_aggregate_records>	boolean	Optional	false / true	if true collector will fetch data from endpoint `/statistics/dmarc/ip-addresses/{source}/aggregate-report-records`
<group_values>	array	Optional/Mandatory	Minimum Length 1	Array of available group_values. ex:- ["country"] . This field will be mandatory if `include_total_group_data` is true
<override_devo_sender_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `statistics/dmarc/senders/{sender}`
<override_devo_sender_ip_address_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/senders/{sender}/ip-addresses`
<override_devo_ip_address_source_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/ip-addresses/{source}`
<override_devo_ip_address_aggregate_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/ip-addresses/{source}/aggregate-report-records`
<override_devo_bimi_selector_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/bimi-selectors/{bimiSelector}`
<override_devo_public_key_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/dkim-public-keys/{dkimPublicKey}`
<override_devo_total_group_tag>	str	Optional	A devo tag	It will override the devo tag for the endpoint `/statistics/dmarc/volume/totals/grouped/{grouping}`

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image	SHA-256 hash
collector-sendmarc_if-docker-image-1.0.1	`a1c614642bc6f72db907f769762d3be00fc70c69b595b80836bf93c78291d4bce965671ee2832f8337467a5811cb76c8cf40a68f`

Use the following command to add the Docker image to the system:

Code Block
gunzip -c <image_file>-<version>.tgz \| docker load

Note
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

Code Block

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

Code Block

version: '3'
services:
 collector-<product_name>:
   image: <image_name>:${IMAGE_VERSION:-latest}
   container_name: collector-<product_name>
   restart: always
   volumes:
     - ./certs:/devo-collector/certs
     - ./config:/devo-collector/config
     - ./credentials:/devo-collector/credentials
     - ./state:/devo-collector/state
   environment:
     - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

Code Block
IMAGE_VERSION=<version> docker-compose up -d

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

...

Version	Old Version 4	New Version Current
Changes made by	MD Tausif	MD Tausif
Saved on	Jan 10, 2025	Jan 10, 2025

Content Comparison

Versions Compared

Key

Editing the JSON configuration

Structure

Devo credentials

Editing the config.yaml file

Download the Docker image

Docker

Docker Compose