Document toolboxDocument toolbox

Sendmarc collector

Owned by Borja Moro Moreno
Last updated: Nov 27, 2024 by MD Tausif
28 min read
[ 1 Devo collector features ] [ 2 Data sources ] [ 3 Accepted authentication methods ] [ 4 Minimum configuration required for basic pulling ] [ 5 Run the collector ] [ 6 Collector services detail ] [ 7 Collector operations ] [ 8 Change log ]

Devo collector features

Feature

Details

Feature

Details

Allow parallel downloading (multipod)

not allowed

Running environments

  • collector server

  • On premise

Populated Devo events

table

Flattening preprocessing

no

Data sources

Data source

Description

API endpoint

Collector service name

Devo table

Available from release

Data source

Description

API endpoint

Collector service name

Devo table

Available from release

Domains - BIMI

Get the bimi selectors for a domain

/bimi-selectors

domains_bimi

dmarc.sendmarc.bimi.domain

v1.0.1

Get a bimi selector for a domain

/bimi-selectors/{bimiSelector}

dmarc.sendmarc.bimi.selector

Domains - DKIM

Get the DKIM public keys for a domain

/dkim-public-keys

domains_dkim

dmarc.sendmarc.dkim.domain

Get a DKIM public key for a domain

/dkim-public-keys/{dkimPublicKey}

dmarc.sendmarc.dkim.public_key

Domains - DMARC

Get the DMARC settings for a domain

/dmarc-settings

dmarc_settings

dmarc.sendmarc.setting.dmarc

Domains - IP Addresses Details and Totals

Get Details and totals for a specific IPaddress (source)

/statistics/dmarc/ip-addresses/{source}

ip_address

dmarc.sendmarc.ip_address.source

Domains - IP Addresses Statistics

Get IP addresses statistics for domain

/statistics/dmarc/ip-addresses

dmarc.sendmarc.ip_address.domain

Domains - Senders

Get volume of totals on domain

/statistics/dmarc/volume/totals

dmarc_volume_total

dmarc.sendmarc.volume.total

Get totals according to a group

/statistics/dmarc/volume/totals/grouped/{grouping}

dmarc.sendmarc.volume.group_total

Get details and totals in the IP addresses used by a sender on a domain grouped by date

/statistics/dmarc/volume/timeline

dmarc_volume_timeline

dmarc.sendmarc.volume.timeline

Get aggregate report records for IP addresses

/statistics/dmarc/ip-addresses/{source}/aggregate-report-records

ip_address

dmarc.sendmarc.ip_address.aggregate_records_report

Get timeline statistics for senders on a domain

/statistics/dmarc/senders

senders

dmarc.sendmarc.sender.domain

Get timeline statistics for a sender on a domain

/statistics/dmarc/senders/{sender}

dmarc.sendmarc.sender.domain_detail

Get details and totals of IP addresses used by sender on domain

/statistics/dmarc/senders/{sender}/ip-addresses

dmarc.sendmarc.ip_address.sender

Domains - SPF

Get the SPF settings for a domain

/spf-settings

spf_settings

dmarc.sendmarc.setting.spf

Domain - STS

Get the TLS and STS settings for a domain

/sts-settings

sts_settings

dmarc.sendmarc.setting.sts

For more information on how the events are parsed, visit our page.

Accepted authentication methods

user_id

required

required

api_key

required

required

Minimum configuration required for basic pulling

Although this collector supports advanced configuration, the fields required to retrieve data with basic configuration are defined below.

This minimum configuration refers exclusively to those specific parameters of this integration. There are more required parameters related to the generic behavior of the collector. Check running the collector section for details.

user_id → The user_id for Sendmarc API

api_key → The api_key for Sendmarc API

Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).

Collector services detail

This section is intended to explain how to proceed with specific actions for services.

2024-10-22T18:40:34.001 INFO MainProcess::MainThread -> Started all objects from "MainProcess" process 2024-10-22T18:40:34.003 INFO InputProcess::MainThread -> Process started (pid=23008, ppid=22978, multiprocessing.start_method="fork") 2024-10-22T18:40:34.022 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/sendmarc;123451;senders;predefined;SendMarcBasePuller.json 2024-10-22T18:40:34.023 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.023 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/330ed6ca6825ea26199bf8f0161f15e6" 2024-10-22T18:40:34.023 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/4ec2dbb861d0d8af5fa1fa3cc3d224f2" 2024-10-22T18:40:34.023 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-10-22T18:40:34.023 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/330ed6ca6825ea26199bf8f0161f15e6" 2024-10-22T18:40:34.023 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/4ec2dbb861d0d8af5fa1fa3cc3d224f2" 2024-10-22T18:40:34.024 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) Starting the execution of init_variables() 2024-10-22T18:40:34.024 INFO InputProcess::MainThread -> Validating service metadata 2024-10-22T18:40:34.026 INFO InputProcess::MainThread -> Validating defined module definition 2024-10-22T18:40:34.031 INFO InputProcess::MainThread -> Validating common input config 2024-10-22T18:40:34.033 INFO InputProcess::MainThread -> Validating service input config 2024-10-22T18:40:34.035 INFO InputProcess::MainThread -> Running overriding rules 2024-10-22T18:40:34.035 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-10-22T18:40:34.035 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-10-22T18:40:34.036 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-10-22T18:40:34.036 INFO InputProcess::MainThread -> Running custom validation rules 2024-10-22T18:40:34.036 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) Finalizing the execution of init_variables() 2024-10-22T18:40:34.037 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/DevoSenderManager;standard_senders;devo_1.json.gz 2024-10-22T18:40:34.037 INFO InputProcess::MainThread -> InputThread(sendmarc,123451) - Starting thread (execution_period=60s) 2024-10-22T18:40:34.037 INFO InputProcess::MainThread -> ServiceThread(sendmarc,123451,senders,predefined) - Starting thread (execution_period=60s) 2024-10-22T18:40:34.038 INFO InputProcess::MainThread -> SendMarcPullerSetup(sendmarc,sendmarc#123451,senders#predefined) -> Starting thread 2024-10-22T18:40:34.038 INFO InputProcess::MainThread -> SendMarcBasePuller(sendmarc#123451,senders#predefined) - Starting thread 2024-10-22T18:40:34.038 WARNING InputProcess::SendMarcPullerSetup(sendmarc,sendmarc#123451,senders#predefined) -> The token/header/authentication has not been created yet 2024-10-22T18:40:34.038 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.038 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/5bdbabbd30e1988d6e2b0b74b6ec3289" 2024-10-22T18:40:34.038 WARNING InputProcess::SendMarcBasePuller(sendmarc#123451,senders#predefined) -> Waiting until setup will be executed 2024-10-22T18:40:34.041 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/DevoSender;standard_senders;devo_sender_0.json.gz 2024-10-22T18:40:34.043 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.043 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/b1ee30ea2c4b3be1eb50c8b2ca80d8d8" 2024-10-22T18:40:34.045 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/OutputStandardConsumer;standard_senders;0.json.gz 2024-10-22T18:40:34.046 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.046 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/de28663278a264d05d531fbc1db51a93" 2024-10-22T18:40:34.047 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/DevoSenderManager;lookup_senders;devo_1.json.gz 2024-10-22T18:40:34.048 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.048 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/a28c135f3fc69ece87d8fc2fdb029e8c" 2024-10-22T18:40:34.051 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/DevoSender;lookup_senders;devo_sender_0.json.gz 2024-10-22T18:40:34.052 INFO InputProcess::MainThread -> [GC] global: 30.2% -> 30.3%, process: RSS(42.00MiB -> 42.00MiB), VMS(424.98MiB -> 424.98MiB) 2024-10-22T18:40:34.053 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.053 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/df8895fef2a509cbd87fcc9850dc0c81" 2024-10-22T18:40:34.054 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/OutputLookupConsumer;lookup_senders;0.json.gz 2024-10-22T18:40:34.055 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.055 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/865a79c1b99ad39b22becc235c9732cb" 2024-10-22T18:40:34.057 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/DevoSenderManager;internal_senders;devo_1.json.gz 2024-10-22T18:40:34.057 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.057 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/e784c94a6fe28de5937e50674407d82e" 2024-10-22T18:40:34.059 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/DevoSender;internal_senders;devo_sender_0.json.gz 2024-10-22T18:40:34.059 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.059 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/4ff7b345dc444ac050cf75f93e5dcb3b" 2024-10-22T18:40:34.060 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-sendmarc/state/not_used/OutputInternalConsumer;internal_senders;0.json.gz 2024-10-22T18:40:34.060 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-10-22T18:40:34.060 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-sendmarc/state/10dd360c86621afd5a28a029a0dddcf6" 2024-10-22T18:40:34.061 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> Starting thread 2024-10-22T18:40:34.061 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(standard_senders,devo_1) -> Starting thread (every 300 seconds) 2024-10-22T18:40:34.061 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> Starting thread 2024-10-22T18:40:34.061 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-10-22T18:40:34.061 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-10-22T18:40:34.061 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> Starting thread 2024-10-22T18:40:34.062 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(lookup_senders,devo_1) -> Starting thread (every 300 seconds) 2024-10-22T18:40:34.062 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> Starting thread 2024-10-22T18:40:34.062 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-10-22T18:40:34.062 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-10-22T18:40:34.062 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> Starting thread 2024-10-22T18:40:34.063 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(internal_senders,devo_1) -> Starting thread (every 300 seconds) 2024-10-22T18:40:34.063 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:34.063 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-10-22T18:40:34.063 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> Starting thread 2024-10-22T18:40:34.063 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-10-22T18:40:34.064 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:34.064 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-10-22T18:40:34.064 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-10-22T18:40:34.064 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:34.064 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-10-22T18:40:34.076 INFO OutputProcess::MainThread -> [GC] global: 30.3% -> 30.3%, process: RSS(41.60MiB -> 41.72MiB), VMS(929.25MiB -> 929.25MiB) 2024-10-22T18:40:34.076 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:34.076 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-10-22T18:40:34.076 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:34.076 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-10-22T18:40:34.077 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:34.077 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-10-22T18:40:34.586 INFO OutputProcess::DevoSender(internal_senders,devo_sender_0) -> Created a sender: {"name": "DevoSender(internal_senders,devo_sender_0)", "url": "collector-eu.devo.io:443", "chain_path": "/home/md_tausif/gitlab/devo-collector-sendmarc/certs/chain.crt", "cert_path": "/home/md_tausif/gitlab/devo-collector-sendmarc/certs/int-if-integrations-india.crt", "key_path": "/home/md_tausif/gitlab/devo-collector-sendmarc/certs/int-if-integrations-india.key", "transport_layer_type": "SSL", "last_usage_timestamp": null, "socket_status": null}, hostname: "2023-apac-0046", session_id: "138689817995632" 2024-10-22T18:40:34.586 INFO OutputProcess::DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-10-22T18:40:36.701 INFO InputProcess::SendMarcPullerSetup(sendmarc,sendmarc#123451,senders#predefined) -> Setup for module <SendMarcBasePuller> has been successfully executed

Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

Component

Description

Component

Description

Setup

The setup module is in charge of authenticating the service and managing the token expiration when needed.

Puller

The setup module is in charge of pulling the data in an organized way and delivering the events via SDK.

Puller output

A successful initial run has the following output messages for the puller module:

Note that the PrePull action is executed only one time before the first run of the Pull action.

2024-10-22T14:30:29.692 INFO InputProcess::SendMarcBasePuller(sendmarc#123451,senders#predefined) -> Pull Started 2024-10-22T14:31:12.831 INFO InputProcess::SendMarcBasePuller(sendmarc#123451,senders#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1729587629690):Number of requests made: 1; Number of events received: 19; Number of duplicated events filtered out: 0; Number of events generated and sent: 19; Average of events per second: 0.440. 2024-10-22T14:31:12.831 INFO InputProcess::SendMarcBasePuller(sendmarc#123451,senders#predefined) -> Sent 19 events to Devo for account: <account> and domain: <domain>

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

2024-10-22T14:31:12.831 INFO InputProcess::SendMarcBasePuller(sendmarc#123451,senders#predefined) -> Statistics for t

Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component

Description

Component

Description

Setup

The setup module is in charge of authenticating the service and managing the token expiration when needed.

Puller

The setup module is in charge of pulling the data in an organized way and delivering the events via SDK.

Puller output

A successful initial run has the following output messages for the puller module:

Note that the PrePull action is executed only one time before the first run of the Pull action.

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component

Description

Component

Description

Setup

The setup module is in charge of authenticating the service and managing the token expiration when needed.

Puller

The setup module is in charge of pulling the data in an organized way and delivering the events via SDK.

Puller output

A successful initial run has the following output messages for the puller module:

Note that the PrePull action is executed only one time before the first run of the Pull action.

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

Verify data collection

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

Component

Description

Component

Description

Setup

The setup module is in charge of authenticating the service and managing the token expiration when needed.

Puller

The setup module is in charge of pulling the data in an organized way and delivering the events via SDK.

Puller output

A successful initial run has the following output messages for the puller module:

Note that the PrePull action is executed only one time before the first run of the Pull action.

Collector operations

Change log

Release

Released on

Release type

Recommendations

Release

Released on

Release type

Recommendations

v1.0.1

Nov 5, 2024

bug fixing

Recommended version

v1.0.0

Oct 24, 2024

new feature

-