Thinkst Canary collector

Service name

Implementation details

birds

This service obtains the full list of birds (devices) and sends all the retrieved events in Devo in a “snapshot” fashion. This is the reason we limit the maximum frequency of this service to at least one hour.

audit_trail_log

This service uses an endpoint that allows filtering by the start and end date (up to the day), but it does not offer a sorting mechanism. To circumvent this limitation and send all the events in an ordered fashion, we do the following:

1. Establish a range of days between the date_from parameter and the day before the current one to chonologically fetch the logs day by day.

   1. All the logs from one single day are sorted before these are sent to Devo.

   2. We keep track of the last “visited day” whose logs were fetched and sent.

2. We obtain the logs for the current day, filtering out those that were already obtained.

   1. We keep track of the last fetched and sent log to only send consequent ones.

   2. Once the day finishes, it will be treated as in the previous step.

incidents

The endpoint that this service uses, instead of allowing to filter by dates, uses a numeric ID that facilitates following the sequence of events ingested. The service starts from the beginning of the sequence until the last updated ID (included in every response). During the first pulling execution, however, we filter out those events that are previous to the datetime_from (timezone-aware) parameter.

tokens

This service obtains a paginated list of tokens and sends all the retrieved events in Devo in a “snapshot” fashion. This is the reason we limit the maximum frequency of this service to at least one hour.

Error Type

Error Id

Error Message

Cause

Solution

InitVariableError

10

Ping endpoint not found in module globals in collector_definitons.yaml

There is an internal error with the collector.

Please, contact Devo support.

InitVariableError

11

datetime_format not found in module globals for service: {service_name}

There is an internal error with the collector.

Please, contact Devo support.

InitVariableError

12

Start date from config: {start_date} is greater than current date for service: {service_name}

The value provided in date_from/datetime_from parameters is a future date.

Please provide a date/datetime that is before the current date. Note that the datetime_from parameter in the incidents service is timezone-aware.

InitVariableError

13

date_format not found in module globals for service: {self.service_name}

There is an internal error with the collector.

Please, contact Devo support.

PullError

308

Auth token expired. Confirm from Thinkst Canary that the token is still valid.

Auth token expired. Confirm from Thinkst Canary that the token is still valid.

Login to canary console and confirm if you are using the correct auth token.

PullError

311, 312, 313, 320, 321

<Error Message will be received from Thinkst>

Follow the instructions in the error message

internal_senders

In charge of delivering internal metrics to Devo such as logging traces or metrics.

standard_senders

In charge of delivering pulled events to Devo.

Number of available senders: 1

Displays the number of concurrent senders available for the given Sender Service.

sender manager internal queue size: 0

Displays the items available in the internal sender queue.

Standard - Total number of messages sent: 57, messages sent since "2023-01-10 16:09:16.116750+00:00": 0 (elapsed 0.000 seconds

Displays the number of events from the last time and following the given example, the following conclusions can be obtained:

• 44 events were sent to Devo since the collector started.

• The last checkpoint timestamp was 2023-01-10 16:09:16.116750+00:00.

• 21 events where sent to Devo between the last UTC checkpoint and now.

• Those 21 events required 0.007 seconds to be delivered.