Page Comparison

Table of Contents

maxLevel	2
type	flat

Overview

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs to individual companies. Each available AWS service generates information related to different aspects of its functionality. The available data types include service events, audit events, metrics, and logs.

You can use the AWS collector to retrieve data from the AWS APIs and send it to your Devo domain. Once the gathered information arrives at Devo, it will be processed and included in different tables in the associated Devo domain so users can analyze it.

To run this collector, there are some configurations detailed below that you need to consider:

...

Info

More information

Refer to the Vendor setup section to know more about these configurations.

Note
If you need to pull global service events created by CloudFront, IAM and AWS STS, US regions will need to be enabled within the collector. For more information, see Viewing CloudTrail events with the AWS CLI and Using update-trail.

Overview

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs to individual companies. Each available AWS service generates information related to different aspects of its functionality. The available data types include service events, audit events, metrics, and logs.

...

Using update-trail.

Devo collector features

Feature	Details
Allow parallel downloading (`multipod`)	Not `not allowed`
Running environments	`Collector collector server` `Onon-premise`
Populated Devo events	`Table` `table`
Flattening preprocessing	`No` `no`

Data sources

Data Sourcesource	Description	API Endpointendpoint	Collector service name	Devo table	Available from release
Service events	The different available services in AWS usually generate some information related to their internal behaviors, such as "a virtual machine has been started", "a new file has been created in an S3 bucket" or "an AWS lambda function has been invoked" and this kind of event can be triggered by no human interaction. The service events are managed by the `CloudWatch Events` service (`CWE`), recently AWS has created a new service called `Amazon EventBridge` that tends to replace the `CWE` service. The findings detected by `AWS Security Hub` are also managed by `CloudWatch Events (CWE)`.	`ReceiveMessage` ReceiveMessage - Amazon Simple Queue Service	Generic events: `service-events-all` Security Hub events: `sqs-cloudwatch-consumer`	Generic events: If `auto_event_type` parameter in config file is not set or set to `false`: `cloud.aws.cloudwatch.events` If `auto_event_type` parameter in config file is set to `true`: `cloud.aws.cloudwatch.{event_type}` Security Hub events: `cloud.aws.securityhub.findings`	`-`
Audit events	This kind of event is more specific because they are triggered by a human interaction no matter the different ways used: API, web interaction, or even the CLI console. The audit events are managed by the `CloudTrail` service. There are two ways to read Audit events: API: using CloudTrail API. This way is slower, but it can retrieve data back in time. S3+SQS: forwarding CloudTrail data to an S3 bucket and reading from there through a SQS queue. This way is much faster, but it only can retrieve elements since the creation of the S3+SQS pipeline.	Via API: `LookupEvents` LookupEvents - AWS CloudTrail Via S3+SQS: `ReceiveMessage` ReceiveMessage - Amazon Simple Queue Service	`audit-events-all`	If `auto_event_type` parameter in config file is not set or set to `false`: `cloud.aws.cloudtrail.events` If `auto_event_type` parameter in config file is set to `true`: `cloud.aws.cloudtrail.{event_type}`	`-`
Metrics	According to the standard definition, this kind of information is usually generated at the same moment is requested because it is usually a query about the status of a service (all things inside AWS are considered services). AWS makes something slightly different because what is doing is to generate metrics information every N time slots, such as 1 min, 5 min, 30 min, 1h, etc., even if no one makes a request (also is possible to have information every X seconds but this would require extra costs). The metrics are managed by the `CloudWatch Metrics` service (`CWM`).	`ListMetrics` ListMetrics - Amazon CloudWatch After listing the metrics, `GetMetricData` and `GetMetricStatistics` are also called. GetMetricData - Amazon CloudWatch GetMetricStatistics - Amazon CloudWatch	`metrics-all`	`cloud.aws.cloudwatch.metrics`	`-`
Logs	Logs could be defined as information with a non-fixed structure that is sent to one of the available “logging” services, these services are `CloudWatch Logs` and `S3`. There are some very customizable services, such as `AWS Lambda`, or even any developed application which is deployed inside an AWS virtual machine (`EC2`), that can generate custom log information, this kind of information is managed by the `CloudWatch Logs` service (`CWL`) and also by the `S3` service. There are also some other services that can generate logs with a fixed structure, such as `VPC Flow Logs` or `CloudFront Logs`. These kinds of services require one special way of collecting their data.	`DescribeLogStreams` DescribeLogStreams - Amazon CloudWatch Logs	Logs can be: Managed by Cloudwatch: This is a custom service that is activated using service `custom_service` and including the type `logs` into the `types` parameter in the config file. Not managed by Cloudwatch: Use `non-cloudwatch-logs` service and include the required type (`flowlogs` for VPC Flow Logs and/or `cloudfrontlogs` for CloudFront Logs) into the `types` parameter in the config file.	Managed by Cloudwatch: `cloud.aws.cloudwatch.logs` Not managed by Cloudwatch: VPC Flow Logs: If `auto_event_type` parameter in config file is set to `true`: `cloud.aws.vpc.unknown` If `auto_event_type` parameter in config file is set to `true`: `cloud.aws.vpc.{event_type}` CloudFront Logs: If `auto_event_type` parameter in config file is set to `true`: `cloud.aws.cloudfront.unknown` If `auto_event_type` parameter in config file is set to `true`: `cloud.aws.cloudfront.{event_type}` `-`	`-`
AWS GuardDuty	AWS GuardDuty is a managed threat detection service that continuously monitors for malicious activity and unauthorized behavior to protect your AWS accounts, workloads, and data stored in Amazon S3. Data Sources: GuardDuty ingests and processes data from AWS CloudTrail logs, VPC Flow Logs, and DNS logs Findings: When a potential threat is detected, GuardDuty generates a finding. These findings provide details about the activity, including the affected resources, type of threat, and suggested remediation actions. We are using API to get findings of guardduty service.	What is Amazon GuardDuty? - Amazon GuardDuty	`aws-guardduty`	`cloud.aws.cloudfront.findings.{account_id}.{region_id}.{fromat_version}`	`v1.10.0`
Cisco Umbrella [Non-AWS service]	Cisco Umbrella is a cloud-driven Secure Internet Gateway (SIG) that leverages insights gained through the analysis of various logs, including DNS logs, IP logs, and Proxy logs, to provide a first line of defense. DNS logs record all DNS queries that are made through the Cisco Umbrella DNS resolvers. These logs contain data about the DNS queries originating from your network, requested domain names and the IP address of the requester. IP logs capture all IP-based communications that occur through the network. These logs store details such as the source and destination IP addresses, ports and protocols used. Proxy logs are generated when users access web resources through the Cisco Umbrella intelligent proxy. They contain detailed information on the web traffic including the URL accessed, the method of access (GET, POST, etc.), the response status, etc	Via S3+SQS: `ReceiveMessage` ReceiveMessage - Amazon Simple Queue Service	`cisco-umbrella`	`sig.cisco.umbrella.dns` `sig.cisco.umbrella.ip` `sig.cisco.umbrella.proxy`	`v1.6.0`

...

There are some minimal requirements to set up this collector.:

AWS console access: Credentials are required to access the AWS console.
Owner or Administrator permissions within the AWS console, or the fill access to configure AWS services.

Some manual actions are necessary in order to get all the required information or services and allow Devo to gather information from AWS. The following sections describe how to get the required AWS credentials and how to proceed with the different required setups depending on the gathered information type.

...

Expand

title	Policy details

Some collector services require the creation of some IAM policies before creating the IAM user that will be used for the AWS collector. The following table contains the details about the policies that could be used by the AWS collector:

Source type

AWS Data Bus

Recommended policy name

Variant

Additional info

Service events

CloudWatch Events

devo-cloudwatch-events

All resources

Tip
It’s not required the creation of any new policy due to there are not needed any permissions

Audit events

CloudTrail API

devo-cloudtrail-api

All resources

Code Block
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "cloudtrail:LookupEvents", "Resource": "*" } ] }

-

Specific resource

Warning
There is no way for limiting the accessed resources

CloudTrail S3+SQS

devo-cloudtrail-s3

All resources

Code Block
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:GetObject", "Resource": "*" } ] }

-

Specific S3 bucket

Info
Note that the value for the property called Resource should be changed with the proper value

Note
It very important the `/*` string at the end of each bucket name

Code Block

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::devo-cloudtrail-storage-bucket1/*",
                "arn:aws:s3:::devo-cloudtrail-storage-bucket2/*"
            ]
        }
    ]
}

Metrics

CloudWatch Metrics

devo-cloudwatch-metrics

All resources

Code Block
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "cloudwatch:GetMetricData", "cloudwatch:ListMetrics" ], "Resource": "*" } ] }

Specific resource

Warning
There is no way for limiting the accessed resources

Logs

CloudWatch Logs

devo-cloudwatch-logs

All log groups

Info
Note that the value for property `Resource` should be adapted with the proper account id value.

Code Block

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:FilterLogEvents"
            ],
            "Resource": "arn:aws:logs:*:936082584952:log-group:*"
        }
    ]
}

Specific log groups

Info
Note that values inside the `Resources` property are only examples and they should be changed with the proper values.

Code Block

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:FilterLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:*:936082584952:log-group:/aws/events/devo-cloudwatch-test-1:*",
                "arn:aws:logs:*:936082584952:log-group:/aws/events/devo-cloudwatch-test-2:*"
            ]
        }
    ]
}

Logs to S3 + SQS

devo-vpcflow-logs

All resources

Code Block
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:GetObject", ]"Resource": "*" } ] }

Logs to S3 + SQS

devo-vpcflow-logs

All resourcesSpecific resource

Code Block
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::vpc-flowlogs-test1/*" } ] }

Cisco Umbrella

Logs to S3 + SQS

devo-cisco-umbrella

Specific resource

Code Block
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::vpc-flowlogs-test1cisco-logs-us-west-1/<prefix>/*" } ] }

Cisco Umbrella

Logs to S3 + SQSAWS GuardDuty

Guardduty API

devo-ciscoguardduty-umbrellaapi

Specific resource

Code Block

{
    "Version": "2012-10-17",
    "Statement": [
   
        {
            "Effect": "Allow",
            "Action": [
                "guardduty:GetFindings",
                "guardduty:ListFindings",
    {             "Sid": "VisualEditor0",guardduty:ListDetectors"
            "Effect": "Allow"],
            "Action": "s3:GetObject",           "Resource": "arn:aws:guardduty:<region>:<accountID>:detector/*"
"Resource": "arn:aws:s3guardduty:<region>::cisco-logs-us-west-1/<prefix>/<accountID>:detector/*"

        }
    ]
}

Info
All the previous policies are defined to be AWS region agnostic, which means, that they will be valid for any AWS region.

Expand

title	Using a user account and local policies

Depending on which source types are collected, one or more of the policies described above will be used. Once the required policies are created, each one must be associated with an IAM user. To create it, visit the AWS Console and log in with a user account with enough permissions to create and access AWS structures:

Go to IAM → Users.
Click Add users button.
Enter the required value in the filed User name.
Enable the checkbox Access key - Programmatic access.
Click on Next: Tags button.
Click on Next: Review button.
Click on Create user button.
The Access Key ID and Secret Key will show. Click Download.csv button and save it.

...

Rw ui tabs macro

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the collector:

Code Block

<any_directory>
└── devo-collectors/
    └── <product_name>/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── state/
        └── config/ 
            └── config.yaml

Note
Replace `<product_name>` with the proper value.

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/. Learn more about security credentials in Devo here.

Note
Replace `<product_name>` with the proper value.

Editing the config.yaml file

Code Block

globals:
  debug: false
  id: <collector_id>
  name: <collector_name>
  persistence:
    type: filesystem
    config:
      directory_name: state
outputs:
  devo_1:
    type: devo_platform
    config:
      address: <devo_address>
      port: 443
      type: SSL
      chain: <chain_filename>
      cert: <cert_filename>
      key: <key_filename>
inputs:
  aws:
    id: <short_unique_id>
    enabled: true
    credentials:
      access_key: <access_key_value>
      access_secret: <access_secret_value>
      base_assume_role: <base_assume_role_value>
      target_assume_role: <target_assume_role_value>
      assume_role_external_id: <assume_role_external_id_value>
    services:
      service-events-all:
        request_period_in_seconds: <request_period_in_seconds_value>
        cloudwatch_sqs_queue_name: <sqs_queue_name_value>
        auto_event_type: <auto_event_type_value>
        regions: <list_of_regions>
      sqs-cloudwatch-consumer:
        request_period_in_seconds: <request_period_in_seconds_value>
        cloudwatch_sqs_queue_name: <sqs_queue_name_value>
        auto_event_type: <auto_event_type_value>
        regions: <list_of_regions>
      audit-events-all:
        types: <list_of_types>
        request_period_in_seconds: <request_period_in_seconds_value>
        start_time: <start_time_value>
        auto_event_type: <auto_event_type_value>
        drop_event_names: <list_of_drop_event_names>
        audit_sqs_queue_name: <sqs_queue_name_value>
        s3_file_type_filter: <s3_file_type_filter_value>
        use_region_and_account_id_from_event: <use_region_and_account_id_from_event_value>
        regions: <list_of_regions>
      metrics-all:
        regions: <list_of_regions>
      non-cloudwatch-logs:
        types: <list_of_types>
        regions: <list_of_regions>
        start_time: <start_time_value>
        vpcflowlogs_sqs_queue_name: <sqs_queue_name_value>
        cloudfrontlogs_sqs_queue_name: <sqs_queue_name_value>
      cisco-umbrella:
        ciscoumbrella_sqs_queue_name: <sqs_queue_name_value>
        regions: <list_of_regions>
      custom_service:
        types: <list_of_types>
        log_groups: <log_group_value>
        start_time: <start_time_value>
        regions: <list_of_regions>
      aws-guardduty:
        request_period_in_seconds: <request_period_in_seconds_value>
        start_time: <start_time_value>
        regions: <list_of_regions>

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Replace the placeholders with your required values following the description table below:

Parameter

Data Type

Type

Value Range

Details

collector_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this collector.

collector_name

str

Mandatory

Minimum length: 1
Maximum length: 10

Use this param to give a valid name to this collector.

devo_address

str

Mandatory

collector-us.devo.io
collector-eu.devo.io

Use this param to identify the Devo Cloud where the events will be sent.

chain_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the chain.cert file downloaded from your Devo domain. Usually this file's name is: chain.crt

cert_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the file.cert downloaded from your Devo domain.

key_filename

str

Mandatory

Minimum length: 4
Maximum length: 20

Use this param to identify the file.key downloaded from your Devo domain.

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

access_key_value

str

See Accepted authentication methods section above.

Minimum length: 1

The access key ID value obtained from AWS when a user is created to access programmatically. It is used when authenticating with a user account and also to assume a self-account role.

access_secret_value

str

See Accepted authentication methods section above.

Minimum length: 1

The secret access key value obtained from AWS when a user is created to access programatically. It is used when authenticating with a user account and also to assume a self-account role.

base_assume_role_value

str

See Accepted authentication methods section above.

Minimum length: 1

The ARN of the role to be assumed in the base account. It can be used for self- or cross-account authentication methods.

target_assume_role_value

str

See Accepted authentication methods section above.

Minimum length: 1

The ARN of the role to be assumed in the customer’s account. It is used for cross-account authentication method.

assume_role_external_id_value

str

See Accepted authentication methods section above.

Minimum length: 1

This is an optional string implemented by the customer to add an extra security layer. It can only be used for cross-account authentication method.

request_period_in_seconds_value

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (300 seconds)

Info
This parameter should be removed if it is not used.

auto_event_type_value

bool

Optional

true/false

Used to enable the auto categorization of message tagging.

start_time_value

datetime

Mandatory for GuardDuty, Optional for the rest of services.

1970-01-01T00:00:00.000Z

Datetime Date time from which to start collecting data. It must match ISO-8601 format8601 format. Note that this is mandatory for GuardDuty and optional for the rest of services.

list_of_types

list (of strings)

Optional

Code Block
types: - type1 - type2 - type3

Enable/Disable modules only when several modules per service are defined. For example, to get audit events from the API, this field should be set to audits_api.

list_of_regions

list (of strings)

Mandatory, if defined in the collector’s definition.

Code Block
regions: - region1 - region2 - region3

Property name (regions) should be aligned with the one defined in the submodules_property property from the “Collector definitions”

list_of_drop_event_names

list (of strings)

Optional

Code Block
drop_event_names: - drop1 - drop2 - drop3

If the value in eventName field matches any of the values in this field, the event will be discarded.

i.e. if this parameter is populated with the next values ["Decrypt", "AssumeRole"], and the value of eventName field is Decrypt or AssumeRole, the event will be discarded.

sqs_queue_name_value

str

Mandatory

Minimum length: 1

Name of the SQS queue to read from.

s3_file_type_filter_value

str

Optional

Minimum length: 1

RegEx to retrieve proper file type from S3, in case there are more than one file types in the same SQS queue from which the service is pulling data.

This parameter can be used for those services getting data from a S3+SQS pipeline.

use_region_and_account_id_from_event_value

bool

Optional

true/false

If true the region and account_id are taken from the event; else if false, they are taken from the account used to do the data pulling.

Default: true

It can be used only in those services using a S3+SQS pipeline.

log_group_value

str

Mandatory

Minimum length: 1

The log group name must be set here as-is, including the different levels separated by slashes.
It can be set to'/' (forward slash) as well to get all the log group names.

Download the Docker image

The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:

Collector Docker image	SHA-256 hash
collector-aws-if-docker-image-1.10.0	`e5f62752b1fd91ca3347f7a639933bdd6db56d9f411b088352b737da0d1ee308`

Use the following command to add the Docker image to the system:

Code Block
gunzip -c <image_file>-<version>.tgz \| docker load

Note
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.

The Docker image can be deployed on the following services:

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/

Code Block

docker run 
--name collector-<product_name> 
--volume $PWD/certs:/devo-collector/certs 
--volume $PWD/config:/devo-collector/config 
--volume $PWD/state:/devo-collector/state 
--env CONFIG_FILE=config.yaml 
--rm 
--interactive 
--tty 
<image_name>:<version>

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/ directory.

Code Block

version: '3'
services:
  collector-<product_name>:
    image: <image_name>:${IMAGE_VERSION:-latest}
    container_name: collector-<product_name>
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./credentials:/devo-collector/credentials
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/ directory:

Code Block
IMAGE_VERSION=<version> docker-compose up -d

Note
Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

Rw tab

title	Cloud collector

We use a piece of software called Collector Server to host and manage all our available collectors.

To enable the collector for a customer:

In the Collector Server GUI, access the domain in which you want this instance to be created
Click Add Collector and find the one you wish to add.
In the Version field, select the latest value.
In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).
In the sending method select Direct Send. Direct Send configuration is optional for collectors that create Table events, but mandatory for those that create Lookups.
In the Parameters section, establish the Collector Parameters as follows below:

Editing the JSON configuration

Code Block

{
  "global_overrides": {
    "debug": false
  },
  "inputs": {
    "aws": {
      "id": "<short_unique_id>",
      "enabled": true,
      "credentials": {
        "access_key": "<access_key_value>",
        "access_secret": "<access_secret_value>",
        "base_assume_role": "<base_assume_role_value>",
        "target_assume_role": "<target_assume_role_value>",
        "assume_role_external_id": "<assume_role_external_id_value>"
      },
      "services": {
        "service-events-all": {
          "request_period_in_seconds": <request_period_in_seconds_value>,
          "cloudwatch_sqs_queue_name": "<sqs_queue_name_value>",
          "auto_event_type": <auto_event_type_value>,
          "regions": <list_of_regions>
        },
        "sqs-cloudwatch-consumer": {
          "request_period_in_seconds": <request_period_in_seconds_value>,
          "cloudwatch_sqs_queue_name": "<sqs_queue_name_value>",
          "auto_event_type": <auto_event_type_value>,
          "regions": <list_of_regions>
        },
        "audit-events-all": {
          "types": <list_of_types>,
          "request_period_in_seconds": <request_period_in_seconds_value>,
          "start_time": "<start_time_value>",
          "auto_event_type": <auto_event_type_value>,
          "drop_event_names": <list_of_drop_event_names>,
          "audit_sqs_queue_name": "<sqs_queue_name_value>",
          "s3_file_type_filter": "<s3_file_type_filter_value>",
          "use_region_and_account_id_from_event": "<use_region_and_account_id_from_event_value>",
          "regions": <list_of_regions>
        },
        "metrics-all":{
          "regions": <list_of_regions>
        },
        "non-cloudwatch-logs": {
          "types": <list_of_types>,
          "regions": <list_of_regions>,
          "start_time": "<start_time_value>",
          "vpcflowlogs_sqs_queue_name": "<sqs_queue_name_value>",
          "cloudfrontlogs_sqs_queue_name": "<sqs_queue_name_value>"
        },
        "cisco-umbrella": {
          "start_time": "<start_time_value>",
          "ciscoumbrella_sqs_queue_name": "<sqs_queue_name>",
          "regions": <list_of_regions>
        },
        "aws-guardduty": {
          "start_time": "<start_time_value>",
          "regions": [
            "ap-southeast-1"
          ]
        },
        "custom_service": {
          "types": "<list_of_types>",
          "log_group": "<log_group_value>",
          "start_time": "<start_time_value>",
          "regions": <list_of_regions>
        }
      }
    }
  }
}

Info
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.

Please replace the placeholders with real world values following the description table below:

Type Range

Parameter

Data

type

Type

Value

range

Details

short_unique_id

int

Mandatory

Minimum length: 1
Maximum length: 5

Use this param to give an unique id to this input service.

Note
This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision.

access_key_value

str

See Accepted authentication methods section above.

Minimum length: 1

The access key ID value obtained from AWS when a user is created to access programmatically. It is used when authenticating with a user account and also to assume a self-account role.

access_secret_value

str

See Accepted authentication methods section above.

Minimum length: 1

The secret access key value obtained from AWS when a user is created to access programatically. It is used when authenticating with a user account and also to assume a self-account role.

base_assume_role_value

str

See Accepted authentication methods section above.

Minimum length: 1

The ARN of the role to be assumed in the base account. It can be used for self- or cross-account authentication methods.

target_assume_role_value

str

See Accepted authentication methods section above.

Minimum length: 1

The ARN of the role to be assumed in the customer’s account. It is used for cross-account authentication method.

assume_role_external_id_value

str

See Accepted authentication methods section above.

Minimum length: 1

This is an optional string implemented by the customer to add an extra security layer. It can only be used for cross-account authentication method.

request_period_in_seconds_value

int

Optional

Minimum length: 1

Period in seconds used between each data pulling, this value will overwrite the default value (300 seconds)

This parameter should be removed if it is not used.

auto_event_type_value

bool

Optional

true/false

Used to enable the auto categorization of message tagging.

start_time_value

datetime

Mandatory for GuardDuty, Optional for the rest of services.

1970-01-01T00:00:00.000Z

Datetime Date time from which to start collecting data. It must match ISO-8601 format. Note that this is mandatory for GuardDuty and optional for the rest of services.

list_of_types

list (of strings)

Optional

Code Block
"types" : [ "type1", "type2", "type3" ]

Enable/Disable modules only when several modules per service are defined. For example, to get audit events from the API, this field should be set to audits_api.

list_of_regions

list (of strings)

Mandatory, if defined in the collector’s definition.

Code Block
"regions": [ "region1", "region2", "region3" ]

Property name (regions) should be aligned with the one defined in the submodules_property property from the “Collector definitions”

list_of_drop_event_names

list (of strings)

Optional

Code Block
"drop_event_names" : [ "drop1", "drop2", "drop3" ]

If the value in eventName field matches any of the values in this field, the event will be discarded.

i.e. if this parameter is populated with the next values ["Decrypt", "AssumeRole"], and the value of eventName field is Decrypt or AssumeRole, the event will be discarded.

sqs_queue_name_value

str

Mandatory

Minimum length: 1

Name of the SQS queue to read from.

s3_file_type_filter_value

str

Optional

Minimum length: 1

RegEx to retrieve proper file type from S3, in case there are more than one file types in the same SQS queue from which the service is pulling data.

This parameter can be used for those services getting data from a S3+SQS pipeline.

use_region_and_account_id_from_event_value

bool

Optional

true/false

If true the region and account_id are taken from the event; else if false, they are taken from the account used to do the data pulling.

Default: true

It can be used only in those services using a S3+SQS pipeline.

log_group_value

str

Mandatory

Minimum length: 1

The log group name must be set here as-is, including the different levels separated by slashes.
It can be set to'/' (forward slash) as well to get all the log group names.

...

Expand

title	Troubleshooting

This collector has different security layers that detect both an invalid configuration and abnormal operation. This table will help you detect and resolve the most common errors.

ErrorTypeError type	Error IdID	Error Messagemessage	Cause	Solution
`AwsModuleDefinitionError`	1	`"{module_properties_key_path}" mandatory property is missing or empty`	`module_properties` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	2	`"{module_properties_key_path}" property must be a dictionary`	`module_properties` is not a dictionary type data structure.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	3	`"{module_properties_key_path}.tag_base" mandatory property is missing or empty`	`tag_base` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	4	`"{module_properties_key_path}.tag_base" property must be a string`	`tag_base` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	5	`module_properties_key_path}.tag_base" property must have {event_type}, {account_id}, {region_id} and {format_version} placeholders`	`tag_base` does not literally contain all those placeholders, and they are required.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	6	`"{module_properties_key_path}.tag_base" property is containing some unexpected placeholders, the allowed ones are: ["event_type", "account_id", "region_id", "format_version", "environment", "service_name"]`	`tag_base` has an unauthorized placeholder or is not correctly built.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	7	`"{module_properties_key_path}.event_type_default" mandatory property is missing or empty`	`event_type_default` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	8	`"{module_properties_key_path}.event_type_default" property must be a string`	`event_type_default` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	26	`"{module_properties_key_path}.event_type_source_field_name" mandatory property is missing or empty`	`event_type_source_field_name` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	27	`"{module_properties_key_path}.event_type_source_field_name" property must be a boolean`	`event_type_source_field_name` is not of type boolean.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	26	`"{module_properties_key_path}.event_type_extracting_regex" mandatory property is missing or empty`	`event_type_extracting_regex` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	27	`"{module_properties_key_path}.event_type_extracting_regex" property must be a boolean`	`event_type_extracting_regex` is not of type boolean.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	5	`"{module_properties_key_path}.event_type_extracting_regex" property is not a valid regular expression`	`event_type_extracting_regex` is not a valid Regular Expression.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	7	`"{module_properties_key_path}.enable_auto_event_type" mandatory property is missing or empty`	`enable_auto_event_type` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	8	`"{module_properties_key_path}.enable_auto_event_type" property must be a string`	`enable_auto_event_type` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	9	`"{module_properties_key_path}.enable_auto_event_type_config_key" mandatory property is missing or empty`	`enable_auto_event_type_config_key` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	10	`"{module_properties_key_path}.enable_auto_event_type_config_key" property must be a string`	`enable_auto_event_type_config_key` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	31	`"{module_properties_key_path}.event_type_processor_mapping" property should be a dictionary.`	`event_type_processor_mapping` is not of type dictionary.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	32	`"{module_properties_key_path}.event_type_processor_mapping" exists but it is empty.`	`event_type_processor_mapping` cannot be empty, and it is.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	33	`"{module_properties_key_path}.event_type_processor_mapping.{processor_name}.processor_class" mandatory property is missing or empty`	`processor_class` does not exist or is empty.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	34	`"{module_properties_key_path}.event_type_processor_mapping.{processor_name}.processor_class" should be a string`	`processor_class` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	35	`"{module_properties_key_path}.event_type_processor_mapping.{processor_name}.tagging" mandatory property is missing or empty`	`tagging` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	36	`"{module_properties_key_path}.event_type_processor_mapping.{processor_name}.tagging" should be a string`	`tagging` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	10	`"{module_properties_key_path}.sqs_queue_custom_name_key" mandatory property is missing or empty`	`sqs_queue_custom_name_key` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	11	`"{module_properties_key_path}.sqs_queue_custom_name_key" property must be a string`	`sqs_queue_custom_name_key` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	12	`"{module_properties_key_path}.sqs_queue_required_default_name" property must be a boolean`	`sqs_queue_required_default_name` is not of type boolean.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	13	`"{module_properties_key_path}.sqs_queue_default_name" mandatory property is missing or empty`	`sqs_queue_default_name` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	14	`"{module_properties_key_path}.sqs_queue_default_name" property must be a string`	`sqs_queue_default_name` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	15	`"{module_properties_key_path}.sqs_queue_default_name" property must have {input_id} placeholder`	`sqs_queue_default_name` does not have the required `{input_id}` placeholder.	This is an internal issue. Please, contact with Devo Support team.
`AwsInputConfigurationError`	1	`"{input_config_key_path}" mandatory property is missing or empty`	The `inputs` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsInputConfigurationError`	2	`"{input_config_key_path}" property must be a dictionary`	The `inputs` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}" mandatory property is missing or empty`	The `services` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	2	`"{service_config_key_path}" property must be a dictionary`	The `services` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	3	`"{service_config_key_path}.tag" property must be a string`	`tag` is not of type string.	Change the `tag` parameter to be a string.
`AwsServiceConfigurationError`	4	`"{service_config_key_path}.{sqs_queue_custom_name_key}" mandatory property is missing or empty`	The parameter indicated in the error message is not present in the configuration file.	Add the indicated parameter to the configuration file.
`AwsServiceConfigurationError`	5	`"{service_config_key_path}.{sqs_queue_custom_name_key}" property must be a string`	The parameter indicated in the error message is not of type string.	Make the indicated parameter be a string.
`AwsServiceConfigurationError`	6	`"{service_config_key_path}.{sqs_queue_custom_name_key}" property must be a string`	The parameter indicated in the error message is not of type string.	Make the indicated parameter be a string.
`AwsServiceConfigurationError`	7	`"{service_config_key_path}.{enable_auto_event_type_config_key}" property must be a string`	The parameter indicated in the error message is not of type string.	Make the indicated parameter be a string.

Common for all the services using the S3+SQS pipeline

ErrorTypeError type	Error IdID	Error Messagemessage	Cause	Solution
`AwsModuleDefinitionError`	1	`"{module_properties_key_path}" mandatory property is missing or empty`	`module_properties` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	2	`"{module_properties_key_path}" property must be a dictionary`	`module_properties` is not a dictionary type data structure.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	10	`"{module_properties_path}.start_time_regex" mandatory property is missing or empty`	`start_time_regex` is not present at collector definition file.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	11	`"{module_properties_path}.start_time_regex" property must be a string`	`start_time_regex` is of a type other than string.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	12	`"{module_properties_path}.start_time_regex" property is not a valid regular expression`	`start_time_regex` is not a valid Regular Expression.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	21	`"{sqs_s3_processor_properties_key_path}" mandatory property is missing or empty`	The parameter indicated in the error message is not present in collector definition file.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	22	`"{sqs_s3_processor_properties_key_path}" property must be a dictionary`	The parameter indicated in the error message is not of type dictionary.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	26	`"{sqs_s3_processor_properties_key_path}.class_name" mandatory property is missing or empty`	`class_name` is empty or is not present in the collector definition file.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	27	`"{sqs_s3_processor_properties_key_path}.class_name" property must be a string`	`class_name` is not of type string.	This is an internal issue. Please, contact with Devo Support tea
`AwsModuleDefinitionError`	10	`"{module_properties_key_path}.sqs_queue_custom_name_key" mandatory property is missing or empty`	`sqs_queue_custom_name_key` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	11	`"{module_properties_key_path}.sqs_queue_custom_name_key" property must be a string`	`sqs_queue_custom_name_key` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	12	`"{module_properties_key_path}.sqs_queue_required_default_name" property must be a boolean`	`sqs_queue_required_default_name` is not of type boolean.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	13	`"{module_properties_key_path}.sqs_queue_default_name" mandatory property is missing or empty`	`sqs_queue_default_name` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	14	`"{module_properties_key_path}.sqs_queue_default_name" property must be a string`	`sqs_queue_default_name` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	15	`"{module_properties_key_path}.sqs_queue_default_name" property must have {input_id} placeholder`	`sqs_queue_default_name` does not have the required `{input_id}` placeholder.	This is an internal issue. Please, contact with Devo Support team.
`AwsInputConfigurationError`	1	`"{input_config_key_path}" mandatory property is missing or empty`	The `inputs` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsInputConfigurationError`	2	`"{input_config_key_path}" property must be a dictionary`	The `inputs` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}" mandatory property is missing or empty`	The `services` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	2	`"{service_config_key_path}" property must be a dictionary`	The `services` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	3	`"{service_config_key_path}.tag" property must be a string`	`tag` is not of type string.	Change the `tag` parameter to be a string.
`AwsServiceConfigurationError`	4	`"{service_config_key_path}.{sqs_queue_custom_name_key}" mandatory property is missing or empty`	The parameter indicated in the error message is not present in the configuration file.	Add the indicated parameter to the configuration file.
`AwsServiceConfigurationError`	5	`"{service_config_key_path}.{sqs_queue_custom_name_key}" property must be a string`	The parameter indicated in the error message is not of type string.	Make the indicated parameter be a string.
`AwsServiceConfigurationError`	6	`"{service_config_key_path}.{sqs_queue_custom_name_key}" property must be a string`	The parameter indicated in the error message is not of type string.	Make the indicated parameter be a string.
`AwsServiceConfigurationError`	7	`"{service_config_key_path}.{enable_auto_event_type_config_key}" property must be a string`	The parameter indicated in the error message is not of type string.	Make the indicated parameter be a string.
`AwsServiceConfigurationError`	7	`"{service_config_key_path}.start_time" property must be a string`	`start_time` is not of type string.	Change the `start_time` parameter to be a string.
`AwsServiceConfigurationError`	8	`The property "{service_config_key_path}.start_time" from configuration is having a wrong format, expected pattern: "{start_time_regex}"`	`start_time` parameter does not match the Regular Expression.	Change the `start_time` parameter to march the Regular Expression.
`AwsQueueException`	0	`Queue "{sqs_queue_name}" used by service "{service_name}" in "{submodule_config}" region is not available: reason: {reason}`	The queue indicated in the error message is not available.	Check the next things: Name of the queue. The queue exists in the indicated region. Read carefully the reason the error message is returning.

Audit (via API)

ErrorTypeError type	Error IdID	Error Messagemessage	Cause	Solution
`AwsModuleDefinitionError`	1	`"{module_properties_key_path}" mandatory property is missing or empty`	`module_properties` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	2	`"{module_properties_key_path}" property must be a dictionary`	`module_properties` is not a dictionary type data structure.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	3	`"{module_properties_key_path}.tag_base" mandatory property is missing or empty`	`tag_base` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	4	`"{module_properties_key_path}.tag_base" property must be a string`	`tag_base` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	5	`module_properties_key_path}.tag_base" property must have {event_type}, {account_id}, {region_id} and {format_version} placeholders`	`tag_base` does not literally contain all those placeholders, and they are required.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	6	`"{module_properties_key_path}.tag_base" property is containing some unexpected placeholders, the allowed ones are: ["event_type", "account_id", "region_id", "format_version", "environment", "service_name"]`	`tag_base` has an unauthorized placeholder or is not correctly built.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	7	`"{module_properties_key_path}.event_type_default" mandatory property is missing or empty`	`event_type_default` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	8	`"{module_properties_key_path}.event_type_default" property must be a string`	`event_type_default` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	9	`"{module_properties_key_path}.enable_auto_event_type" mandatory property is missing or empty`	`enable_auto_event_type` is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	10	`"{module_properties_key_path}.enable_auto_event_type" property must be a string`	`enable_auto_event_type` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	11	`"{module_properties_key_path}.enable_auto_event_type_config_key" mandatory property is missing or empty`	`enable_auto_event_type_config_key` is empty or is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	12	`"{module_properties_key_path}.enable_auto_event_type_config_key" property must be a string`	`enable_auto_event_type_config_key` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	13	`"{module_properties_key_path}.start_time_regex" mandatory property is missing or empty`	`start_time_regex` is empty or is not present in the collector definition file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	14	`"{module_properties_key_path}.start_time_regex" property must be a string`	`start_time_regex` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	15	`"{module_properties_key_path}.start_time_regex" property is not a valid regular expression`	`start_time_regex` is using a invalid Regular Expression.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	16	`"{module_properties_key_path}.gap_until_now_in_minutes" mandatory property is missing or empty`	`gap_until_now_in_minutes` is empty or not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	17	`"{module_properties_key_path}.gap_until_now_in_minutes" property must be a string`	`gap_until_now_in_minutes` is not of type integer.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	18	`"{module_properties_key_path}.gap_until_now_in_minutes" property can not be a negative value`	`gap_until_now_in_minutes` is having a negative value, which is not allowed.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	19	`"{module_properties_key_path}.time_slot_in_hours" mandatory property is missing or empty`	`time_slot_in_hours` is empty or is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	20	`"{module_properties_key_path}.time_slot_in_hours" property must be an integer`	`time_slot_in_hours` is not of type integer.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	21	`"{module_properties_key_path}.time_slot_in_hours" property can not be 0 or a negative value`	`time_slot_in_hours` is zero or has a negative value, which is not allowed.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	22	`"{module_properties_key_path}.sources" mandatory property is missing or empty`	`sources` is empty or is not present in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	23	`"{module_properties_key_path}.sources" property exists but with wrong format, only "str" or "list" values are allowed`	`sources` is not of type string or list.	This is an internal issue. Please, contact with Devo Support team.
`AwsInputConfigurationError`	1	`"{input_config_key_path}" mandatory property is missing or empty`	The `inputs` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsInputConfigurationError`	2	`"{input_config_key_path}" property must be a dictionary`	The `inputs` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}" mandatory property is missing or empty`	The `services` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	2	`"{service_config_key_path}" property must be a dictionary`	The `services` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	3	`"{service_config_key_path}.tag" property must be a string`	`tag` is not of type string.	Change the `tag` parameter to be a string.
`AwsServiceConfigurationError`	4	`"{service_config_key_path}.sources" property exists but with wrong format, only "str" or "list" values are allowed`	`sources` is not of type string or list.	Change the `sources` parameter to be a string or a list.
`AwsServiceConfigurationError`	5	`"{service_config_key_path}.gap_until_now_in_minutes" property must be an integer`	`gap_until_now_in_minutes` is not of type integer.	Change the `gap_until_now_in_minutes` parameter to be an integer.
`AwsServiceConfigurationError`	6	`"{service_config_key_path}.start_time" property must be a string`	`start_time` is not of type string.	Change the `start_time` parameter to be a string.
`AwsServiceConfigurationError`	7	`The property "{service_config_key_path}.start_time" from configuration is having a wrong format, expected pattern: "{start_time_regex}"`	`start_time` is using an invalid Regular Expression.	Change the `start_time` parameter to match the Regular Expression indicated in the error message.
`AwsServiceConfigurationError`	8	`"{service_config_key_path}.drop_event_names" property must be a list`	`drop_event_names` is not of type list.	Change the `drop_event_names` parameter to be a list.
`AwsServiceConfigurationError`	9	`"{service_config_key_path}.{enable_auto_event_type_config_key}" property must be a string`	The parameter indicated by the error message is not of type string.	Change the parameter indicated by the error message to be a string.
`AwsServiceConfigurationError`	10	`"{service_config_key_path}.time_slot_in_hours" property must be integer`	`time_slot_in_hours` is not of type integer.	Change the `time_slot_in_hours` parameter to be an integer.

Custom Logs

ErrorTypeError type	Error IdID	Error Messagemessage	Cause	Solution
`AwsInputConfigurationError`	0	`Mandatory property "requests_per_second" is missing (located at: aws.request_per_second)`	`requests_per_second` is not present at configuration file.	Add `requests_per_second` to the configuration file.
`AwsModuleDefinitionError`	1	`"{module_properties_key_path}" mandatory property is missing or empty`	`module_properties` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	2	`"{module_properties_key_path}" property must be a dictionary`	`module_properties` is not a dictionary type data structure.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	3	`"{module_properties_key_path}.tag_base" mandatory property is missing or empty`	`tag_base` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	4	`"{module_properties_key_path}.tag_base" property must be a string`	`tag_base` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsServiceDefinitionException`	5	`module_properties_key_path}.tag_base" property must have {event_type}, {account_id}, {region_id} and {format_version} placeholders`	`tag_base` does not literally contain all those placeholders, and they are required.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	6	`"{module_properties_key_path}.tag_base" property is containing some unexpected placeholders, the allowed ones are: ["event_type", "account_id", "region_id", "format_version", "environment", "service_name"]`	`tag_base` has an unauthorized placeholder or is not correctly built.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	7	`"{module_properties_key_path}.start_time_regex" mandatory property is missing or empty`	`start_time_regex` is not present or is empty in the collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	8	`"{module_properties_key_path}.start_time_regex" property must be a string`	`start_time_regex` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsModuleDefinitionError`	9	`"{module_properties_key_path}.start_time_regex" property is not a valid regular expression`	`start_time_regex` is not a valid Regular Expression.	This is an internal issue. Please, contact with Devo Support team.
`AwsInputConfigurationError`	1	`"{input_config_key_path}" mandatory property is missing or empty`	The `inputs` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsInputConfigurationError`	2	`"{input_config_key_path}" property must be a dictionary`	The `inputs` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}" mandatory property is missing or empty`	The `services` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	2	`"{service_config_key_path}" property must be a dictionary`	The `services` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	3	`"{service_config_key_path}.tag" property must be a string`	`tag` is not of type string.	Change the `tag` parameter to be a string.
`AwsServiceConfigurationError`	43	`"{service_config_key_path}.use_first_optimized_retrieval" property must be a boolean`	`use_first_optimized_retrieval` is not of type boolean.	Change the `use_first_optimized_retrieval` parameter to be a boolean.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}.log_group" mandatory property is missing or empty`	`log_group` is empty or is not present in the configuration file.	Add the `log_group` parameter to the configuration file.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}.log_group" property must be a string`	`log_group` is not of type string.	Change the `log_group` parameter to be a string.
`AwsServiceConfigurationError`	2	`"{service_config_key_path}.start_time" property must be a string`	`start_time` is not of type string.	Change the `start_time` parameter to be a string.
`AwsServiceConfigurationError`	1	`The property "{service_config_key_path}.start_time" from configuration is having a wrong format, expected pattern: "{start_time_regex}"`	`start_time` is not matching the pattern indicated in the error message.	Make `start_time` match the pattern indicated in the error message.

Metrics

ErrorTypeError type	Error IdID	Error Messagemessage	Cause	Solution
`AwsServiceDefinitionException`	1	`"{module_properties_key_path}" mandatory property is missing or empty`	`module_properties` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsServiceDefinitionException`	2	`"{module_properties_key_path}" property must be a dictionary`	`module_properties` is not a dictionary type data structure.	This is an internal issue. Please, contact with Devo Support team.
`AwsServiceDefinitionException`	5	`"{module_properties_key_path}.tag_base" mandatory property is missing or empty`	`tag_base` is not present in collector definitions file.	This is an internal issue. Please, contact with Devo Support team.
`AwsServiceDefinitionException`	6	`"{module_properties_key_path}.tag_base" property must be a string`	`tag_base` is not of type string.	This is an internal issue. Please, contact with Devo Support team.
`AwsServiceDefinitionException`	1	`"{module_properties_key_path}.metric_namespace" mandatory property is missing or empty`	`metric_namespace` is empty or is not present in collector definitions.	This is an internal issue. Please, contact with Devo Support team.
`AwsServiceDefinitionException`	1	`"{module_properties_key_path}.metric_namespace" property must be a list`	`metric_namespace` is not of type list.	This is an internal issue. Please, contact with Devo Support team.
`AwsInputConfigurationError`	1	`"{input_config_key_path}" mandatory property is missing or empty`	The `inputs` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsInputConfigurationError`	1	`"{input_config_key_path}" property must be a dictionary`	The `inputs` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `inputs`.
`AwsServiceConfigurationError`	1	`"{service_config_key_path}" mandatory property is missing or empty`	The `services` data structure is missing or empty in the configuration file.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	8	`"{service_config_key_path}" property must be a dictionary`	The `services` data structure is not of type dictionary.	Go to the “Running the collector on…“ section in the documentation and check the data structure required for the `services`.
`AwsServiceConfigurationError`	8	`"{service_config_key_path}.tag" property must be a string`	`tag` is not of type string.	Change the `tag` parameter to be a string.
`AwsServiceConfigurationError`	0	`Mandatory property "metric_namespaces" is missing`	`metric_namespaces` is not present in configuration file.	Add `metric_namespaces` to the configuration file.
`AwsServiceConfigurationError`	0	`Mandatory property "metric_namespaces" property must be a list`	`metric_namespaces` is not of type list.	Change the `metric_namespaces` parameter to be a list.
`AwsServiceConfigurationError`	1	`When a service uses "metrics" type, the property "request_period_in_seconds" must have one of the following values: 1, 5, 10, 30, 60, or any multiple of 60`	`request_period_in_seconds` is using a value that is not allowed.	Change the `request_period_in_seconds` parameter to match one of this values: 1, 5, 10, 30, 60, or any multiple of 60.
`AwsServiceConfigurationError`	4	`"start_time" property must be a string`	`start_time` is not of type string.	Change the `start_time` parameter to be a string.
`AwsServiceConfigurationError`	4	`The property "start_time" from configuration is having a wrong format, expected: YYYY-mm-ddTHH:MM:SSZ`	`start_time` is using an incorrect format.	Change the `start_time` to match the format indicated in the error message.

...

Version	Old Version 39	New Version 40
Changes made by	Juan Tomás Alonso Nieto	Juan Tomás Alonso Nieto
Saved on	Jun 04, 2024	Jun 13, 2024

Page Comparison

Versions Compared

Key

Overview

Overview

Devo collector features

Data sources

Structure

Devo credentials

Editing the config.yaml file

Download the Docker image

Docker

Docker Compose

Editing the JSON configuration

Common for all the services using the S3+SQS pipeline

Audit (via API)

Custom Logs

Metrics