Content Comparison

Table of Contents

maxLevel	2
type	flat

...

Amazon Web Services (AWS) provides on-demand cloud computing platforms and APIs to individuals and companies. Each available AWS service generates information related to different aspects of its functionality. The available data types include service events, audit events, metrics, and logs.

You can use the AWS collector to retrieve data from the AWS APIs and send it to your Devo domain. Once the gathered information arrives at Devo, it will be processed and included in different tables in the associated Devo domain so users can analyze it.

Data source description

From the monitoring point of view, AWS generates the following types of information:

...

Devo collector service name	Complete service name	CloudWatch filter used	CloudTrail source filter used	Metrics namespace used	Description	Service events (type: `events`)	Audit events (type: `audits`)	Metrics (type: `metrics`)	Logs (type: `logs`)
`service-events-all`	All service events	`{"account":["<account_id>"]}`	N/A	N/A	This service will collect all service events information available in the CloudWatch service, no matter the source defined in the event.	✓	X	X	X
`audit-events-all`	All audit events	N/A	`all_sources`	N/A	This service will collect all audit events information available in the CloudTrail service, no matter the source defined in the event.	X	✓	X	X
`metrics-all`	All metrics	N/A	N/A	`all_metrics_namespaces`	This service will collect all metric information from CloudWatch service. Metrics from all the available metric namespaces will be retrieved.	X	X	✓	X
`<cwl_custom>`	CloudWatch Logs	N/A	N/A	N/A	This service will collect the different “Log Streams” that are part of a “Log Group” from the CloudWatch Logs service. Since it is common to have more than one “Log Group” defined, this will require creating one `<cwl_custom>` entry per “Log Group”.	X	X	X	✓
`non-cloudwatch-logs`	Non-CloudWatch Logs	N/A	N/A	N/A	This service will collect data from the following services VPC Flow Logs and CloudFront Logs.	X	X	X	✓
`sqs-cloudwatch-consumer`	Service events generated by CloudWatch Events service	Check more info here.	N/A	N/A	This service will collect all Security Hub findings that have been sent to CloudWatch, no matter the source defined in the finding.	✓	X	X	X

Info
In the `service-events-all` collector service, the `<account_id>` string is automatically replaced with the real value.

...

Rw ui tabs macro

Rw tab

title	Cloud collector

We use a piece of software called Collector Server to host and manage all our available collectors. If you want us to host this collector for you, get in touch with us and we will guide you through the configuration.

Rw tab

title	On-premise collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the AWScollector:

Code Block

<any_directory>
└── devo-collectors/
    └── aws/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        └── config/ 
            └── config-aws.yaml

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <any directory>/devo-collectors/aws/certs. Learn more about security credentials in Devo here.

Editing the config-aws.yaml file
In the config-aws.yaml file, replace the <short_unique_identifier>, <access_key_value>, <access_secret_value>, <region_*> and <log_group_value_*> values and enter the ones that you got in the previous steps. In the <short_unique_identifier>, <region_*> and <log_group_value_*> placeholders> placeholders, enter the value that you choose.
Code Block
globals: debug: false id: not_used name: aws persistence: type: filesystem config: directory_name: state outputs: devo_1: type: devo_platform config: address: collector-eu.devo.io port: 443 type: SSL chain: chain.crt cert: <your_domain>.crt key: <your_domain>.key inputs: aws: id: <short_unique_identifier> enabled: true requests_per_second: 5 autoconfig: enabled: true refresh_interval_in_seconds: 600 credentials: access_key: <access_key_value> access_secret: <access_secret_value> services: service-events-all: #tag: my.app.aws_service_events #sqs_queue_name: <custom_queue_name>.fifo regions: - <region_a> - <region_b> - <region_c> audit-events-all: #tag: my.app.aws_audit_events regions: - <region_a> - <region_b> - <region_c> metrics-all: #tag: my.app.aws_metrics regions: - <region_a> - <region_b> - <region_c> non-cloudwatch-logs: #tag: my.app.aws_non_cwl #cloudfrontlogs_sqs_queue_name: <custom_queue_name> #vpcflowlogs_sqs_queue_name: <custom_queue_name> regions: - <region_a> - <region_b> - <region_c> cwl_1: #tag: my.app.aws_cwl types: - logs log_group: <log_group_a> regions: - <region_a> - <region_b> - <region_c> cwl_2: #tag: my.app.aws_cwl types: - logs log_group: <log_group_b> regions: - <region_a> - <region_b> - <region_c>
Info
By default, there are several SQS queue name patterns that the collector will try to use for retrieving information from AWS. Depending on the info type, the following patterns exist:
Service events: devo-<service_name>-<short_unique_identifier>.fifo. The property sqs_queue_name can be used to choose a custom name.
VPC Flow Logs: devo-ncwl-vpcfl-<short_unique_identifier>. The property vpcflowlogs_sqs_queue_name can be used to choose a custom name.
CloudFront: devo-ncwl-cfl-<short_unique_identifier>. The property cloudfrontlogs_sqs_queue_name can be used to choose a custom name.
Download the Docker image
The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:
Collector Docker image
SHA-256 hash
collector-aws-docker-image-1.4.1.tgz
21e735b6338537632396171bd09829508949947fb672f14543ce97a475bc72b3
Use the following command to add the Docker image to the system:
Code Block
gunzip -c collector-aws-docker-image-<version>.tgz | docker load
Info
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace "<version>" with a proper value.
The Docker image can be deployed on the following services:
Docker
Docker Compose
Docker
Execute the following command on the root directory <any_directory>/devo-collectors/aws/
Code Block
docker run \ --name collector-aws \ --volume $PWD/certs:/devo-collector/certs \ --volume $PWD/config:/devo-collector/config \ --volume $PWD/state:/devo-collector/state \ --env CONFIG_FILE=config-aws.yaml \ --rm -it docker.devo.internal/collector/aws:<version>
Note
Replace <version> with a proper value.
Docker Compose
The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/aws/ directory.
Code Block
version: '3' services: collector-aws: build: context: . dockerfile: Dockerfile image: docker.devo.internal/collector/aws:${IMAGE_VERSION:-latest} container_name: collector-aws volumes: - ./certs:/devo-collector/certs - ./config:/devo-collector/config - ./state:/devo-collector/state environment: - CONFIG_FILE=${CONFIG_FILE:-config-aws.yaml}

...

Version	Old Version 12	New Version 13
Changes made by	Former user	Former user
Saved on	Aug 25, 2022	Aug 25, 2022

Versions Compared

Key

Data source description

Structure

Devo credentials

Editing the config-aws.yaml file

Download the Docker image

Docker

Docker Compose