/
Generic deployment guidelines

Generic deployment guidelines

Depending on the desired deployment scenarios, there are a handful of different alternatives that can be approached for a particular customer installation. The Devo Endpoint Agent installation process is flexible so as to allow a number of deployment models by using a specific inventory file. In this document, the most common scenarios are depicted, as well as the list of additions to the inventory that are required to fulfill those. Thus, each subsection can be considered as an installation step that offers different alternatives for completion, and not a step-by-step installation procedure.

If there is any special need not included in this guideline, please contact Devo for more detailed information.

The inventory

The deployment of the Endpoint Agent is done via Ansible playbooks. The playbooks instrument all needed steps to deploy the platform according to your needs, defining what and where you want to deploy every component. To do so, the Ansible playbooks will use a YAML inventory file to understand what tasks are needed to accomplish this. The YAML inventory file is the base of the deployment and it is very important to have clear what type of deployment you want to achieve and to create a valid inventory file before starting the deployment process.

When preparing a new deployment of the Endpoint Agent, it is important to have in mind some questions:


  • If you are looking for PoC or non-productive environment deployment, the standalone All-in-One deployment is provided, where the ansible playbooks will create two dockers with the ancillary services for EA Manager to run properly.

  • If you are looking for a small production environment and you do not need HA, you can use the standalone deployment, but use your own ancillary services (MySQL and REDIS) to give the system more robustness and security.

  • If you are looking for a bigger production deployment enabling HA, you should consider a distributed deployment using several managers and your own DB clusters to ensure robustness, security and availability. Agents should communicate with the EA Manager via a Load Balancer to enable the use of more than one EA Manager.

  • The endpoints connect to the manager via /etc/hosts change

  • The endpoints connect to the manager with a known FQDN

  • The endpoints connect to the manager using a public IP but no FQDN

  • What data do i want to collect? Take a look at the out-of-the-box packs available here.


Endpoint Agent provides several YAML templates to base your inventory file in, depending on the case that you are trying to deploy. An Ansible YAML inventory file is composed of three main sections:

  • all → vars : This section defines all the variables that are going to be used in the tasks during the playbook run.

  • all → hosts: This section defines the information related to the hosts where the solution is going to be deployed. It can include one or more hosts, and it is composed of blocks named after the hostname of the destination host.

  • all → children: This section defines the topology that is going to be deployed. You can define hosts groups with a specific name and include there the hosts defined in the all->hosts section . The playbook will then deploy specific roles to those groups when its run. The pre-defined hosts groups are the following:

    • (Mandatory) devoeamanagerserverone: First EA Manager server

    • (Optional) devoeamanagerreplicas: Secondary (or more) manager servers. This role needs to include a variable deam_am_i_replica: true in this section.

    • (Optional) deamintsrvs: EA Manager required ancillary services that should be enabled only in one host. If this role is included, creates a docker for MySQL and Redis . If you will be using your own databases, this host group should not be included in the inventory.

    • (Mandatory) deaagentpackager: EA repository that can be enabled in one or more hosts. In each host included in this group, playbook packages osquery installers with certificates and secrets and generates the auto-installation scripts for the agents that will be deployed in the endpoints. It also installs and configures a NGINX service to easily download said packages.

    • (Mandatory) selfsignedcertificates / providedcertificates: Only one of these hosts groups can be enabled at the same time and are only required in devoeamanagerserverone:

      • selfsignedcertificates: Instructs the EA deployer to generate self-signed certificates that will be used to secure the communication between the endpoints and the manager. By default, the self-signed certificates created by the EA deployer will only trust the FQDN defined in deam_fqdn. In the case you need your certificate to trust different names, you can use the variable SAN_extensions and define a list of Subject Alternative Names (IPs and DNSs) to be included in the certificate.

      • providedcertificates: Instruct the EA deployer to use existing certificates instead of creating new ones. You should use this group if you want to use your own certificates for EA Manager.

Standalone deployment template

The below snippet is a valid inventory file for a standalone deployment.

all: vars: deam_fqdnname: << FQDN >> dea_ap_builder_update_etc_host: << yes/no >> dea_ap_overwrite_deam_fqdnname: << PUBLIC_HOST_IP >> set_deam_fqdnname_as_hostname: false deam_admin_email: << ADMIN_EMAIL >> deam_admin_username: << ADMIN_USER >> deam_admin_passwd: << ADMIN_PASS >> deam_client_org_name: << ORGANIZATION_NAME >> dea_ap_repo_user: << REPO_USER >> dea_ap_repo_passwd: << REPO_PASS >> deam_redis_address: << REDIS_ADDRESS:PORT >> deam_redis_password: << REDIS_PASSWORD >> deam_mysql_address: << MYSQL_ADDRESS:PORT >> deam_mysql_database: << MYSQL_DATABASE_NAME >> deam_mysql_username: << MYSQL_USER >> deam_mysql_password: << MYSQL_PASSWORD >> deam_mysql_root_password: << MYSQL_PASSOWRD_ROOT_USER >> _ _deam_relay_entrypoint: << ENTRYPOINT_URL >> pre_checks_tls_certs_query_bearer: << API v2 Bearer Token >> pre_checks_tls_certs_query_api_url: << Query API entrypoint>> deam_packs_enabled: - configuration.yaml - events.yaml - fetchfiles.yaml - performance.yaml - status.yaml hosts: << HOST_NAME >>: ansible_host: << HOST_IP >> ansible_user: << HOST_USER >> ansible_ssh_pass: << HOST_PASS >> # Only required if you connect with password ansible_ssh_common_args: '-o StrictHostKeyChecking=no' ansible_python_interpreter: << PYTHON_INTERPRETER >> children: devoeamanagerserverone: hosts: << HOST_NAME >>: deamintsrvs: hosts: << HOST_NAME >>: deaagentpackager: hosts: << HOST_NAME >>: selfsigenedcertificates: # Alternative providedcertificates hosts: << HOST_NAME >>: vars: #If no Subject Alternate Names are required, completely remove the SAN_extensions block. SAN_extensions: - IP: << HOST_PUBLIC_IP >> - DNS: << HOST_FQDN >> - DNS: << HOST_FQDN_2 >>

deam_fqdnname: FQDN of the EA Manager. Agents will use this FQDN to communicate to the EA Manager.

dea_ap_builder_update_etc_host: Indicates if the installation package of the endpoints will update /etc/hosts to communicate with the manager.

  • If the FQDN set in deam_fqdnname is public and the agents can reach it, set it as “no”.

  • If the FQDN set in deam_fqdnname is not reachable by the agents and the communication with the EA Manager is via internet, set it as “no”. Configure the parameter dea_ap_overwrite_deam_fqdnname with the public IP that the agents will use to communicate.

  • If the FQDN set in deam_fqdnname is not reachable by the agents and the communication with the EA Manager is via private network, set it as “yes”. When an agent is deployed, the /etc/hosts will be modified to assign the IP configured in ansible_host to the FQDN value.

dea_ap_overwrite_deam_fqdnname: If the FQDN is not reachable by the agent, but the manager needs to be accessed via the internet, put here the public IP that the agent will use to connect. When using public FQDNs or local connection using /etc/hosts this parameter is not necessary and can be removed completely from the inventory.

set_deam_fqdnname_as_hostname: This parameter can be kept always as false. It is a legacy parameter that will be deprecated in the future.

deam_admin_email: E-mail of the admin user. The email is used to log in to the UI interface of the EA Manager.

deam_admin_username: Username for the administrator user.

deam_admin_passwd: Password used to log in to the UI interface of the EA Manager.

deam_client_org_name: Organization Name. It is displayed in the UI.

dea_ap_repo_user: User for the agent download repository.

dea_ap_repo_passwd: Password for the agent download repository.

deam_redis_address: REDIS address and port:

  • If the deployment is self creating dockers for REDIS and MySQL, the value should be configured as localhost:6379.

  • If the deployment is using your own DB services, the value should be configured with the address of the REDIS database and the port of access.

deam_redis_password: REDIS password (when in use).

deam_mysql_address: MySQL address and port:

  • If the deployment is self creating dockers for REDIS and MySQL, the value should be configured as localhost:3306.

  • If the deployment is using your own DB services, the value should be configured with the address of the MySQL database and the port of access.

deam_mysql_database: Name of the database to be used by EA Manager.

deam_mysql_username: Username to access the database.

deam_mysql_password: Password to access the database.

deam_mysql_root_password: Password to access the database as root user (MySQL admin user by default). Only used and necessary if the deployment is self creating dockers for REDIS and MySQL

deam_relay_entrypoint: Includes the entry point value of the Devo collector for your deployment:

  • If your domain is deployed in the US Cloud (us.devo.com): tcp://us.elb.relay.logtrust.net:443.

  • If your domain is deployed in the EU Cloud (eu.devo.com): tcp://eu.elb.relay.logtrust.net:443.

  • If your domain is deployed in a different cloud, contact your Devo representative to know your entry point.

pre_checks_tls_certs_query_bearer: (Optional) Obtain a valid Query API v2 token that grant access to siem.logtrust.collector.counter to enable EA deployer to be able to validate data sent during deployment. How to obtain an API Token.

pre_checks_tls_certs_query_api_url: (Optional) Configure the query API endpoint corresponding to your domain.

deam_packs_enabled: List of YAML files including the definition of the packs that will be deployed into the EA Manager.

hosts: The hosts section of the inventory is composed by a group of hosts named with the hostname of the host. In the case of standalone deployment it will include only one. Replace << HOST_NAME >> with the hostname of the server where the installation is happening. Ansible will connect via SSH to every host where it needs to take action using the data configured in this section.

ansible_host: IP of the EA Manager server that ansible is going to use to connect to the host.

ansible_user: SSH User to connect to this host.

ansible_ssh_pass: Password for the SSH user. It is not mandatory to use a password when using the playbooks. If you want to use SSH keys, just comment out this parameter and add your ssh key at Linux session level or include it in this section using the parameter ansible_ssh_private_key_file .

ansible_python_interpreter: Specify what python interpreter will be used depending on your Linux distribution and version:

  • Python 2: /usr/bin/python

  • Python 3: /usr/bin/python3

Distributed deployment template

The following template is a deployment of Endpoint Agent with two EA Managers using existing databases. This deployment also requires a Load Balancer to be placed between the endpoints and the two EA Managers.

all: vars: deam_fqdnname: << FQDN >> dea_ap_builder_update_etc_host: << yes/no >> set_deam_fqdnname_as_hostname: false deam_admin_email: << ADMIN_EMAIL >> deam_admin_username: << ADMIN_USER >> deam_admin_passwd: << ADMIN_PASS >> deam_client_org_name: << ORGANIZATION_NAME >> dea_ap_repo_user: << REPO_USER >> dea_ap_repo_passwd: << REPO_PASS >> deam_redis_address: << REDIS_ADDRESS:PORT >> deam_redis_password: << REDIS_PASSWORD >> deam_mysql_address: << MYSQL_ADDRESS:PORT >> deam_mysql_database: << MYSQL_DATABASE_NAME >> deam_mysql_username: << MYSQL_USER >> deam_mysql_password: << MYSQL_PASSWORD >> deam_relay_entrypoint: << ENTRYPOINT_URL >> pre_checks_tls_certs_query_bearer: << API v2 Bearer Token >> pre_checks_tls_certs_query_api_url: << Query API entrypoint>> deam_packs_enabled: - configuration.yaml - events.yaml - fetchfiles.yaml - performance.yaml - status.yaml hosts: << HOST_NAME_SERVER_1 >>: ansible_host: << HOST_IP >> ansible_user: << HOST_USER >> ansible_ssh_pass: << HOST_PASS >> # Only required if you connect with password ansible_ssh_common_args: '-o StrictHostKeyChecking=no' ansible_python_interpreter: << PYTHON_INTERPRETER >> << HOST_NAME_SERVER_2 >>: # Set ansible_host your public ip used to conncet from ansbile and devo-ea agents ansible_host: << HOST_IP >> ansible_user: << HOST_USER >> ansible_ssh_pass: << HOST_PASS >> # Only required if you connect with password ansible_ssh_common_args: '-o StrictHostKeyChecking=no' # python3 required for Ubuntu 18 ansible-playbooks ansible_python_interpreter: << PYTHON_INTERPRETER >> children: devoeamanagerserverone: hosts: << HOST_NAME_SERVER_1 >>: devoeamanagerreplicas: hosts: << HOST_NAME_SERVER_2 >>: vars: deam_am_i_replica: true deamintsrvs: hosts: << HOST_NAME_SERVER_1 >>: deaagentpackager: hosts: << HOST_NAME_SERVER_1 >>: << HOST_NAME_SERVER_2 >>: selfsigenedcertificates: # Alternative providedcertificates hosts: << HOST_NAME_SERVER_1 >>:

deam_fqdnname: FQDN of the Load Balancer. Agents will use this FQDN to communicate with the EA Manager.

dea_ap_builder_update_etc_host: Indicates if the installation package of the endpoints will update /etc/hosts to communicate with the Load Balancer.

  • If the FQDN set in deam_fqdnname is public and the agents can reach it, set it as “no”.

  • If the FQDN set in deam_fqdnname is not reachable by the agents and the communication with the EA Manager is via internet, set it as “no”. Configure the parameter dea_ap_overwrite_deam_fqdnname with the public IP that the agents will use to communicate.

  • If the FQDN set in deam_fqdnname is not reachable by the agents and the communication with the EA Manager is via private network, set it as “yes”. When an agent is deployed, the /etc/hosts will be modified to assign the IP configured in ansible_host to the FQDN value.

dea_ap_overwrite_deam_fqdnname: If the FQDN is not reachable by the agent, but the manager needs to be accessed via the internet, put here the public IP that the agent will use to connect. When using public FQDNs or local connection using /etc/hosts this parameter is not necessary and can be removed completely from the inventory.

set_deam_fqdnname_as_hostname: This parameter can be always kept as false. It is a legacy parameter that will be deprecated in the future.

deam_admin_email:Email of the admin user. The email is used to log in to the UI interface of the EA Manager.

deam_admin_username: Username for the administrator user.

deam_admin_passwd: Password used to log in to the UI interface of the EA Manager.

deam_client_org_name: Organization Name. It is displayed in the UI.

dea_ap_repo_user: User for the agent download repository.

dea_ap_repo_passwd: Password for the agent download repository.

deam_redis_address: REDIS address and port.

deam_redis_password: REDIS password.

deam_mysql_address: MySQL address and port.

deam_mysql_database: Name of the database to be used by the EA Manager. The database should have been created in advance. Check the “Using existing databases” section.

deam_mysql_username: Username to access the database.

deam_mysql_password: Password to access the database.

deam_relay_entrypoint: Includes the entry point value of the Devo collector for your deployment:

  • If your domain is deployed in the US Cloud (us.devo.com): tcp://us.elb.relay.logtrust.net:443.

  • If your domain is deployed in the EU Cloud (eu.devo.com): tcp://eu.elb.relay.logtrust.net:443.

  • If your domain is deployed in a different cloud, contact your Devo representative to know your entry point.

pre_checks_tls_certs_query_bearer: (Optional) Obtain a valid Query API v2 token that grant access to siem.logtrust.collector.counter to enable the EA deployer to be able to validate data sent during deployment. How to obtain an API Token.

Include it here if you want the Devo EA deployer to validate that it can send data to it before starting the deployment.

pre_checks_tls_certs_query_api_url: (Optional) Configure the query API endpoint corresponding to your domain.

deam_packs_enabled: List of YAML files including the definition of the packs that will be deployed into the EA Manager.

hosts: The hosts section of the inventory is composed by a group of hosts named with the hostname of the host. Replace << HOST_NAME >> with the hostname of the server where the installation is happening. Ansible will connect via SSH to every host where it needs to take action using the data configured in this section.

For each host configure:

ansible_host: IP of the EA Manager server that ansible is going to use to connect to the host.

ansible_user: SSH User to connect to this host.

ansible_ssh_pass: Password for the SSH user. It is not mandatory to use a password when using the playbooks. If you want to use SSH keys, just comment out this parameter and add your ssh key at Linux session level or include it in this section using the parameter ansible_ssh_private_key_file.

ansible_python_interpreter: Specify what python interpreter will be used depending on your Linux distribution and version:

  • Python 2: /usr/bin/python

  • Python 3: /usr/bin/python3

Deployment credentials

The Devo EA Deployer uses SSH to connect to the different servers where it needs to install different components. When configuring the hosts in the inventory file, it is required to configure the SSH user and authentication method that the deployer needs to use in each task.

Using SSH user and password

For a user and password-based authentication method, configuration for the host should look like the one in the following snippet:

all: ... hosts:     devo-ea-manager:       ansible_host: 192.168.57.5       ansible_user: ssh_user       ansible_ssh_pass: ssh_pass       ansible_ssh_common_args: '-o StrictHostKeyChecking=no' ...

Using SSH private key

Alternatively, in order to use a private key, use the following code:

all: ... hosts:     devo-ea-manager:       ansible_host: 192.168.57.5       ansible_user: ssh_user ansible_ssh_private_key_file: << PATH TO YOUR SSH KEY HERE >> ...

Adding the path to the private key to the inventory file is not mandatory. It can be added to the environment using ssh-add 'key file' instead of adding it to the inventory file. Note that you should make sure that the ssh-agent is running to be able to add your identity. You can start the agent with eval $(ssh-agent).

Initial packs configuration (new from EA v1.1.0)

From v1.1.0 onwards, the EA deployer enables customers to decide which query packs will be deployed with the solution, as opposed to all of them being added automatically during the installation process. As a result, it will be up to the installer responsible to decide which packs will be added to the baseline configuration to fulfill the customer’s particular needs and use cases. The complete list of available packs, along with a description of their functionality and set of supported use cases, can be found here in the pre-integrated query packs section.

The process to enable packs is handled with two parameters in the inventory file:

  • deam_packs_enabled: Sets what packs will be deployed when running both devo-endpoint-agent and deam-packs playbooks.

In the following example, only the configuration (represented by configuration.yaml) and events (events.yaml) packs will be deployed:

The file names included in this list come from the files placed in $HOME/devo-ea-deployer/playbooks/roles/deam-packs/files/optional-devo-packs.

(Only From 1.2.1) The packs (and queries included in the pack) not included in the deam_packs_enabled will be automatically removed from the actual EA Manager if they were previously enabled.

  • (Deprecated in V1.2.1) deam_packs_purge_cached_files: Use this option when you do not want to deploy a pack anymore. Once the packs are deployed for the first time, they are cached under (/var/devo-ea-manager/packs/devo-packs). If you want to avoid some of the packages being redeployed, make sure to include them in the purge list before triggering the deploy operation. Otherwise, the deployer will use the cached version and deploy again the packs even if they are not included in deam_packs_enabled.

In the following example, the installation user is deploying only the configuration.yaml pack and is purging the events.yaml pack that was previously deployed from the temporary folder:

Python Interpreter

When deploying in CentOS 7 or Red Hat 7 OS, make sure that the host is configured to use python 2 instead of python 3 in the inventory file.

Hosts in Centos7 or RedHat 7:

all: ...   hosts:     devo-ea-manager: ...       ansible_python_interpreter: /usr/bin/python

Other OS:

all: ...   hosts:     devo-ea-manager: ...       ansible_python_interpreter: /usr/bin/python3

Firewalld and ufw management

(Only from 1.2.1)

By default, the EA deployer will take care of the configuration of ufw and firewalld to create the required exceptions in the default Linux firewalls if firewall services are up and running at deployment time.

vagrant@ubuntu2004:~/devo-ea-deployer$ sudo ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), deny (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 3306/tcp ALLOW IN Anywhere 6379/tcp ALLOW IN Anywhere 8080/tcp ALLOW IN Anywhere 8081/tcp ALLOW IN Anywhere 22 (v6) ALLOW IN Anywhere (v6) 3306/tcp (v6) ALLOW IN Anywhere (v6) 6379/tcp (v6) ALLOW IN Anywhere (v6) 8080/tcp (v6) ALLOW IN Anywhere (v6) 8081/tcp (v6) ALLOW IN Anywhere (v6)
public (active) target: default icmp-block-inversion: no interfaces: eth0 eth1 eth2 sources: services: cockpit dhcpv6-client ssh ports: 8080/tcp 8081/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules: trusted (active) target: ACCEPT icmp-block-inversion: no interfaces: br-912c94593591 sources: services: ports: 3306/tcp 6379/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:

When using firewalld (in Red Hat based linux) the firewall zones that EA Deployer will use are configurable in the inventory using the following parameters:

deam_fleet_firewall_zone. The zone name where the Devo EA manager listening port (by default, port 8080) will be attached, in the event that firewalld service is up and running. No network interface is added to this zone. Default value public.

deam_intsrv_firewall_zone. The zone name where the new docker network interface (For MySQL and REDIS) will be attached, in the event that firewalld service is up and running. The network interface that docker creates is added to this zone. Default value trusted.

dea_ap_firewall_zone. The zone name where the Devo EA packager repository listening port (by default, port 8081) will be attached, in the event that firewalld service is up and running. No network interface is added to this zone. Default value public.

If you do not want the EA deployer to make any configuration in the firewall, you can add the following variable in the inventory file:

all: vars: firewall_configuration_disabled: true

When the variable firewall_configuration_disabled is true, no action is taken in the firewall configuration. You need to do firewall configuration manually to allow the service to operate correctly.

Endpoint Agent <> EA Manager communication

Using FQDN

In case the system administrator does not want to rely on explicitly setting the IP address of the EAM servers in the configuration, hence easing the introduction of changes in the future, or if the EA Manager is going to be available on an open Internet environment, it is recommended to set the deployer to use an FQDN (Fully Qualified Domain Name) for endpoints <> manager connection. To do so, add a code section such as in the following example to the inventory file -only add the fields that are not already included:

all vars: deam_fqdnname: itops-devo-ea-public-pre-1902513970.eu-west-1.elb.amazonaws.com set_deam_fqdnname_as_hostname: false dea_ap_builder_update_etc_host: "no"
  • deam_fqdnname: FQDN of the Devo EA Manager or the Load Balancer FQDN in the case of multiple EA Managers. Endpoints will use this FQDN to connect to the Devo EA Manager.

  • set_deam_fqdnname_as_hostname: Set to false to avoid changing the EA Manager hostname to the value set in the above field.

  • dea_ap_builder_update_etc_host: Set to "no" to instruct the endpoint installers to not modify the endpoint hosts file.

Using hosts File

A simpler approach to enable the communication between agents and the manager is through the modification of the agents' /etc/hosts file.

The deployment script is pre-defined to enable this mechanism by default, so there is no specific configuration change needed to use it.

EA Manager <> Devo communication

Direct connectivity

This is the default scenario, where the EA Manager has direct connectivity to Devo. In this case, there is no need for any specific configuration as the installer should work as is.

Using a Devo Relay

In some cases, it is necessary to centralize all ingested traffic for a particular deployment, such as to enforce an extra level of security or isolation of the EA Manager.

Assuming that the in-house Relay’s IP is 192.168.43.147, we just need to configure deam_relay_entrypoint: tcp://192.168.43.147:13000. We chose port number 13000 port because the Relay service is configured to listen to that port by default.
Example snippet of inventory file based on that configuration:

all: vars: ... deam_relay_entrypoint: tcp://192.168.43.147:13000 deam_devo_key: "" deam_devo_cert: "" deam_devo_chain: ""

Other special deployment cases

The Devo EA installation process requires access to the Internet in order to be successful. If the server does not have Internet access and requires a proxy to be able to communicate externally, use the following configurations:

These configurations are outside of the EA Manager product. The following instructions are provided as recommendations to the customer and may have high variability depending on the system and specific scenario.

Depending on the system being deployed, append the following two lines:

export http_proxy=http://<PROXY_IP>:<PROXY_PORT> export https_proxy=http://<PROXY_IP>:<PROXY_PORT>

In the corresponding file, depending on the Linux distribution the EA Manager will be installed on:

  • If using root user in Ubuntu distribution: /etc/bash.bashrc

  • If using root user in Centos distribution: /etc/bashrc

  • If using a sudoer in Ubuntu or Centos distribution: ~/.bashrc

Create dir ~/.ssh and file ~/.ssh/environment if they do not exist and append the following lines to the file ~/.ssh/environment:

http_proxy=http://<PROXY_IP>:<PROXY_PORT> https_proxy=http://<PROXY_IP>:<PROXY_PORT>

Append the following line to the file /etc/sudoers:

Defaults env_keep = "http_proxy https_proxy"

Append the following line to the file /etc/ssh/sshd_config:

PermitUserEnvironment=yes

Reload the ssh service: sudo systemctl reload sshd

If you are using dockers in your deployment, you need to configure the docker daemon to use the proxy. Follow the steps in here.

In some cases, the Devo EA Manager may need to be deployed into a server with very limited outbound Internet connectivity. The following list is a compilation of outbound domains that would need to be specifically opened in the firewall so the Devo EA Deployer can be executed properly:

Domain/URL

Purpose

*.devo.com

*.logtrust.net

https://d2ur64jmn3k7yt.cloudfront.net

Devo connectivity, software packages and extension binaries

https://pkg.osquery.io

https://osquery-packages.s3.*.amazonaws.com

Used to download the upstream osquery software required to build the agent packages

*.fedoraproject.org

http://mirrors.kernel.org/fedora

EPEL repositories, For Red Hat and CentOS only

Take into account that the mirror list is dynamic and the package manager will automatically choose the better one, usually based on location and availability so you may need to enable other mirrors based on your distribution.

https://subscription.rhsm.redhat.com

https://cdn.redhat.com

For Red Hat repositories

Take into account that the mirror list is dynamic and the package manager will automatically choose the better one. usually based on location and availability so you may need to enable other mirrors based on your distribution.

*.centos.org

For CentOS repositories

Take into account that the mirror list is dynamic and the package manager will automatically choose the better one, usually based on location and availability so you may need to enable other mirrors based on your distribution.

http://deb.debian.org

http://security.debian.org

http://http.us.debian.org

https://launchpad.net

https://keyserver.ubuntu.com

http://ppa.launchpad.net

http://archive.ubuntu.com

http://security.ubuntu.com

For Ubuntu/Debian repositories

https://nginx.org

For NGINX repositories

https://pypi.org

https://pypi.python.org

https://files.pythonhosted.org

The Python packages

https://galaxy.ansible.com

Directory for Ansible galaxy

https://github.com

https://codeload.github.com

https://github-releases.githubusercontent.com

Ansible roles

There are other domains/URLs whose connection must be granted if the dockerized version of MySQL and Redis is configured (see Using Deployer DB self-deployment chapter below):

High availability configuration

Some deployment scenarios will require the deployment of more than one EA Manager to handle all traffic generated from the endpoints and to ensure high availability of the data processing and forwarding is observed. In this case, a specific configuration during the installation process is required to instruct the Devo EA Deployer to instantiate those roles.

*NOTE: Devo EA Deployer will take care of deploying several EA Managers, but in order to provide a single entry point to the endpoints, it is required that you set up a Load Balancer separately.

  • Load Balancer(s) should have 2 HTTPS listeners, in port 8080 and in port 8081, as there will be two different services. TCP listeners are not supported by the Devo EA Manager.

  • Load Balancer(s) should present the same certificate to the endpoints that the EA Manager is presenting. When using self-signed certificated created automatically by the deployer, the certificate can be found in /etc/devo-ea-manager/certs after the installation process is completed.

To add a secondary Devo EA Manager, add a new host in the all.hosts section and instruct that the EA Manager role should be installed on it in the children section. See the following code snippet as an example:

all: ... hosts: devo-ea-manager: ansible_host: 10.0.0.10 ansible_user: install_user ansible_python_interpreter: /usr/bin/python2 devo-ea-manager-2: ansible_host: 10.0.0.11 ansible_user: install_user ansible_python_interpreter: /usr/bin/python2 children: devoeamanagerserverone: hosts: devo-ea-manager: devoeamanagerreplicas: hosts: devo-ea-manager-2: vars: duem_am_i_replica: true ...

In the above example, two hosts are declared (devo-ea-manager and devo-ea-manager-2) in the hosts section. In the children section, the deployer is instructed to install the the EA Manager in devo-ea-manager and a replica in devo-ea-manager-2.

To enable the secondary manager to also provide the Agents download service in port 8081, it is needed to instruct explicitly the Deployer to enable that option in the inventory file. See the following code snippet as an example:

all ... children: ... deaagentpackager: hosts: devo-ea-manager: devo-ea-manager-2: vars: devo_ea_manager_port: 443 ...

Databases installation

The EAM relies on MySQL and Redis for data persistence. There are several scenarios to consider depending on the type of installation required for a particular customer.

The next section summarizes the most common alternatives for the deployment of the databases, as well as the procedure to implement them:

The Deployer can install a dockerized version of MySQL and Redis on the same server as the EA Manager or on a different server.

The parameters that need to be configured for both databases are the following:

deam_redis_address: 10.0.0.24:6379 deam_redis_password: password deam_mysql_address: 10.0.0.60:3306 deam_mysql_database: databasename deam_mysql_username: user deam_mysql_password: password deam_mysql_root_password: root_password

Mandatory parameters:

  • deam_redis_address : IP and port of the server that will host Redis

  • deam_mysql_address : IP and port of the server that will host MySQL

Optional parameters (if the following parameters are not included in the inventory file, the default value is used):

  • deam_redis_password : Password of the Redis instance, if it exists. In case Redis uses no password, set the value as ““. Default value: ““

  • deam_mysql_database : Name of the database to be used. Default value: devoea

  • deam_mysql_username : Name of the user to access database. Default value: devoea

  • deam_mysql_password : Password of the user to access database. Default value: insecure

  • deam_mysql_root_password : Password of the root (admin) user of the database service. Default value: toor

If the database dockers are going to be installed in the same server as the EA Manager, the role deamintsrvs must be configured to be deployed to the devo-ea-manager host.

all: ... children: ...    deamintsrvs:        hosts:          devo-ea-manager:

This snippet indicates that the role deamintsrvs (MySQL and Redis) will be deployed in the host devo-ea-manager which is defined as well in the inventory file. Check the example configuration files to get some ideas on how your inventory file should look like.

If the databases are going to be installed on server other than the EA Manager, the role deamintsrvs should be configured to be deployed to a different host:

all: ... hosts: devo-ea-manager: ... deam-int-services:      ansible_host: 10.0.10.10       ansible_user: ubuntu       ansible_ssh_private_key_file: /path/to/private.key.pem       ansible_python_interpreter: /usr/bin/python3 children: ... deamintsrvs: hosts: deam-int-services:

In the example above, a new host is defined called deam-int-services and the deployer is instructed to install the database services on it. The host information is included inside the all.hosts tag, so the deployer will know where and how to connect to the new server.

In distributed deployments, it is important to make sure all necessary configurations to enable visibility between services is done beforehand, such as enabling traffic through the designated ports in any intermediary firewalls.

Devo Endpoint Agent Deployer allows the option to use existing deployments of MySQL and Redis. This solution should be observed to enable full HA scenarios.

When using an existing MySQL database, you must manually create the database structures and the credentials that the EA Manager will use to interact with it.

  • Access the MySQL instance with root credentials (or create database privilege granted): mysql -uuser -ppasword

  • Create the database to be used by EA Manager (the database name can be changed, deamDB is just an example): CREATE DATABASE deamDB;

  • Grant ALL PRIVILEGES over the database to the user that DEAM will use (both username and password can be changed, the following is just an example):GRANT ALL PRIVILEGES ON deamDB.* TO 'deamUser'@'%' IDENTIFIED BY 'deamPassword';

No specific configurations in Redis are required other than having full accessibility to the database and valid credentials (password) to access it.

The parameters that control the DB deployment in Devo EA Deployer are the following:

deam_redis_address: 10.0.0.24:6379 deam_redis_password: password deam_mysql_address: 10.0.0.60:3306 deam_mysql_database: databasename deam_mysql_username: user deam_mysql_password: password

Mandatory parameters:

  • deam_redis_address : VIP and port of Redis instance/cluster.

  • deam_mysql_address : VIP and port of MySQL instance/cluster.

  • deam_redis_password : Password of the Redis instance, if it exists. In case Redis doesn’t use a password, input ““. (NOTE: REDIS password cannot be set when creating a docker for the first time. It needs to be set by the user, and then included here to connect to the database).

  • deam_mysql_database : Name of the database to be used.

  • deam_mysql_username : Name of the user to access database.

  • deam_mysql_password : Password of the user to access database.

When an existing database is used, make sure you remove the deamintsrvs from the installation plan in the inventory file. This will instruct the deployer to not deploy any database automatically. If a snippet like the following exists in the inventory file inside all.children tag, remove it:

deamintsrvs: hosts:    devo-ea-manager:

Devo EA Manager users and passwords

After a successful deployment of Devo EA Manager, it will create two services that require authentication. User and password can be configured to include your own passwords. If these parameters are not included in the inventory file, default values are used.

From EA 1.3 onwards, login the platform is done using deam_admin_email instead of deam_admin_username

  • Devo EA Manager Web Interface (port 8080): To control the user and password for this service, the following parameters can be used in the inventory file:

#deam_admin_email: _Devo EA manager_ username for initial admin user setup. Default value: `admin` deam_admin_email: no-reply@localhost.local #deam_admin_username: _Devo EA manager_ username for initial admin user setup. Default value: `admin` deam_admin_username: admin #deam_admin_passwd: _Devo EA manager_ password for initial admin user setup. Default value: `Th3Adm1n!` deam_admin_passwd: Th3Adm1n!
  • Devo Endpoint Agent download (port 8081): To control the user and password for this service, the following parameters can be used in the inventory file:

#dea_ap_repo_user: Username required by _Devo EA packager_ repository for basic authentication. Default value `dea-agent` dea_ap_repo_user: dea-agent #dea_ap_repo_passwd: Password required by _Devo EA packager_ repository for basic authentication. Default value `Th3Ag3nt!` dea_ap_repo_passwd: Th3Ag3nt!

Certificates

The Devo EA Deployer is configured to generate self-signed certificates and configure them by default but you can configure EA Deployer to use your own certificates.

It is recommended to use ECDSA certificates instead of RSA as TLS decryption in the EA Manager side is smoother and increases the performance significantly. The EA Deployer will generate ECDSA certificates by default when using self-generated certificates.

The Devo EA Deployer is configured to generate self-signed certificates and configure them by default.

You can provide your own key/certificate when you deploy EA Manager using Devo EA Deployer.

To do this, replace selfsigenedcertificates  section with providedcertificates  in the inventory file (at same yaml level). Only one of these sections can be present in the inventory file.

  • Inventory file with self-signed certificates generation:

selfsigenedcertificates: # Alternative providedcertificates hosts: devo-ea-manager:
  • Inventory file with provided certificates:

providedcertificates: hosts: devo-ea-manager:

Save your devo-ea-manager.key and devo-ea-manager.crt files in $HOME/devo-ea-deployer/provided-deam-certs path. Provided certs must match with entrypoint used by DEA agents to connect with EA Manager.

It is possible to also provide custom certificates to the NGINX server which is in charge of providing a repository for the Endpoint Agents generated at deployment time.

In order to do so, add the following variables to your inventory file:

all: vars: ... dea_ap_repo_fqdn_name: << REPO_FQDN_NAME >> dea_ap_repo_provided_certs: true

Where << REPO_FQDN_NAME >> is the FQDN of the NGINX server and the FQDN that is trusted by the custom certificates. Expected local path for source certificates is $HOME/devo-ea-deployer/provided-deam-certs/REPO_FQDN_NAME.{crt,key}

Appendix A - Example inventory files

The following code shows a deployment inventory file that enables the installation of two Devo EA Managers, with customized usernames and passwords, and leveraging on an existing MySQL and Redis deployment (e.g., customer-provided). Endpoints connectivity is enabled via FQDN:

all: vars: #deam_fqdnname: Put the Load Balancer FQDN here. This FQDN will be used by the endpoints to reach the EA Manager deam_fqdnname: loadbalancer_fqdn.com set_deam_fqdnname_as_hostname: false dea_ap_builder_update_etc_host: "no" #deam_redis_address: Input here the VIP and port for the Redis cluster deam_redis_address: 10.0.0.24:6379 #deam_redis_password: Redis service password. If there is no password, input "" deam_redis_password: password #deam_mysql_address: Input here the VIP and port for the MySQL cluster deam_mysql_address: 10.0.0.60:3306 #deam_mysql_database: Input here the database name deam_mysql_database: databasename #deam_mysql_username: MySQL user name. User must have enough permissions to create schema, tables, views, etc. deam_mysql_username: user #deam_mysql_password: MySQL password deam_mysql_password: password deam_relay_entrypoint: tcp://us.elb.relay.logtrust.net:443 #deam_admin_email: _Devo EA manager_ username for initial admin user setup. Default value: `admin` deam_admin_email: no-reply@localhost.local #deam_admin_username: _Devo EA manager_ username for initial admin user setup. Default value: `admin` deam_admin_username: admin #deam_admin_passwd: _Devo EA manager_ password for initial admin user setup. Default value: `Th3Adm1n!` deam_admin_passwd: Th3Adm1n! #dea_ap_repo_user: Username required by _Devo EA packager_ repository for basic authentication. Default value `dea-agent` dea_ap_repo_user: dea-agent #dea_ap_repo_passwd: Password required by _Devo EA packager_ repository for basic authentication. Default value `Th3Ag3nt!` dea_ap_repo_passwd: Th3Ag3nt! hosts: #ansible_host: Input IP for Devo EA Manager 1. Can be private, the IP from where the deployer is going to SSH the server for installation tasks. #ansible_user: SSH user to perform installation tasks #Depending on the type of SSH user used for the instalation, uncomment the proper lines in each host and fill a value: #ansible_ssh_pass: Password for the SSH user #ansible_ssh_private_key_file: Private key path to the used PEM file if not using password # Only required if you connect with password #ansible_ssh_common_args: '-o StrictHostKeyChecking=no' devo-ea-manager: ansible_host: 10.0.0.10 ansible_user: install_user #ansible_ssh_pass: #ansible_ssh_common_args: '-o StrictHostKeyChecking=no' #ansible_ssh_private_key_file: ansible_python_interpreter: /usr/bin/python2 devo-ea-manager-2: ansible_host: 10.0.0.11 ansible_user: install_user #ansible_ssh_pass: #ansible_ssh_common_args: '-o StrictHostKeyChecking=no' #ansible_ssh_private_key_file: ansible_python_interpreter: /usr/bin/python2 children: devoeamanagerserverone: hosts: devo-ea-manager: devoeamanagerreplicas: hosts: devo-ea-manager-2: vars: deam_am_i_replica: true deaagentpackager: hosts: devo-ea-manager: devo-ea-manager-2: vars: devo_ea_manager_port: 443 selfsigenedcertificates: # Alternative providedcertificates hosts: devo-ea-manager:

The following code shows a deployment inventory file that enables an AIO (All In One) deployment. Everything is automatically deployed in the same server, including 2 dockers running the database services. Endpoints connectivity is enabled via hosts file.

all:   vars:     deam_fqdnname: devo-ea-manager     deam_redis_address: localhost:6379     deam_mysql_address: localhost:3306 #deam_admin_email: _Devo EA manager_ username for initial admin user setup. Default value: `admin` deam_admin_email: no-reply@localhost.local #deam_admin_username: _Devo EA manager_ username for initial admin user setup. Default value: `admin` deam_admin_username: admin #deam_admin_passwd: _Devo EA manager_ password for initial admin user setup. Default value: `Th3Adm1n!` deam_admin_passwd: Th3Adm1n!     deam_relay_entrypoint: tcp://us.elb.relay.logtrust.net:443   hosts:     devo-ea-manager:       # Set ansible_host your public ip used to conncet from ansbile and devo-ea agents       ansible_host: 192.168.57.5       ansible_user: user       ansible_ssh_pass: pass       # Only required if you connect with password       ansible_ssh_common_args: '-o StrictHostKeyChecking=no'       # python3 required for Ubuntu 18 ansible-playbooks       ansible_python_interpreter: /usr/bin/python3   children:     devoeamanagerserverone:       hosts:         devo-ea-manager:     deamintsrvs:       hosts:         devo-ea-manager:     deaagentpackager:       hosts:         devo-ea-manager:     selfsigenedcertificates: # Alternative providedcertificates       hosts:         devo-ea-manager:     devoeaagents:       hosts:

Appendix B - Security Recommendations

EA Manager deployment will not modify any configuration present on the system as it is the responsibility of a system administrator to have the system hardened following their company’s policies.

However, since the EA Manager will be an element present there are several recommended good practices that can be followed at your discretion:

  • Ensure following directives are included in sshd_config:

X11Forwarding no IgnoreRhosts yes PermitEmptyPasswords no MaxAuthTries 3 PermitRootLogin no Protocol 2 PermitTunnel no PasswordAuthentication no
  • Restrict access to connections coming from certain ips using certain users, even including password authentication, but only from trusted origins, for example:

PasswordAuthentication no AllowUsers user1@ip1 user1@ip2 user2@ip3 ... Match User XXX Address XXX.XXX.XXX.XXX PasswordAuthentication yes PermitTunnel yes
  • Create unique users associated to a certain person and avoid usage of generic users with shared keys.

 

Related content