Endpoint agent 1.3.1 upgrade procedure

Check out our release notes to see the changes in this new version.

EA Manager v1.3.0 → v1.3.1 upgrade procedure

1. Back up your existing devo-ea-deployer folder. If you used one of the existing example inventory files (such as inventories/deployment_inventory.yaml) make sure to have a copy at hand.

2. Back up your existing /etc/devo-ea-manager and /etc/devo-ea folders in all DEAM instances.

3. Make sure that agent configuration properties in your inventory file are consistent with your actual configuration file. If you have made changes to the osquery configuration using the EAM Web UI instead of the inventory file there could be discrepancies. Download your current configuration with the following command :

   sudo env ROOT_CA_CERTIFICATE=/etc/devo-ea-manager/certs/devo-ea-manager.crt /usr/local/bin/deam_fleetctl.sh https://<DEA FQDN>:8080 <admin_email> '<your_password>' get config

   The agent configuration properties are saved in the spec -> agent_options -> config section of the output from previous command.

4. Save your current deployment folder as a backup:

   sudo mv $HOME/devo-ea-deployer $HOME/devo-ea-deployer-backup

5. If you have provided your MySQL instance (required on HA environments) you should make a backup and save it in a secure place in order to allow fallback procedure. If you are using the docker version of MySQL automatically generated by the previous version of EA deployer, skip this step as Ansible takes care of backing up your database.

   Making a backup from MySQL data depends on your environment, but a good starting point is to use mysqldump command. For example:

   mysqldump -u root -p devoea --result-file=migrations-backup.sql

6. Move to the $HOME folder :

7. Download the new package into the current folder.

8. Untar the downloaded package :

9. Copy the Devo domain certs to the new deployment folder:

10. Move to the new deployer folder:

11. If you haven’t done it before, activate the ansible-2.9 virtual environment.

12. Run migration inventory tool. After the migration script is executed, File Fetcher configurations and Windows Event Channel configurations should be kept, although other configuration flags will be overwritten with the new configuration baseline for EA 1.3. If there is any custom configuration that you want to keep specifically for your environment, add it manually after the script is run.

13. Run migration playbook:

14. The dea-migrations playbook generates a number of backups. Download them and keep them safe as they are required during the rollback procedure. In order to list all the backups generated execute: ansible all -m command -a "find /var/devo-ea-deployer/migrations" -e "ansible_become=true" -i inventories/deployment_inventory.yaml . The end result should look like this:

15. Run the upgrade playbook to EA 1.3.1:

Endpoints upgrade

1. The new EA package is available in https://DEAM_IP:8081.

2. Extract the files in the .zip / .tgz file.

3. Run the installation script:

   1. Windows: Run install.ps1 in a Powershell session with elevated privileges.

   2. Linux/Mac: Run install.sh as root user, for example run with sudo.

Post-installation steps

1. Check that the EA Manager interface is accessible.

2. Check that all the packs and queries are available:

   1. The default packs are installed in ENABLED state. If you want to disable any of them, simply deactivate them via the administration web UI.

   2. Any other additional packs should continue being available.

3. Check that the data collected from the endpoints flows into Devo after the installation.

EA Manager rollback procedure

In the unlikely event of the EA Manager node ending up in a faulty status after an upgrade process, apply the following rollback procedure:

1. Ensure that you have all the backup files created by hand and created by dea-migrations ansible playbook.

2. Move to deployment folder:

3. Stop all manager and agent repository services:

    

    

4. Stop internal services (mysql and redis) deployed in no-HA mode (dockerized mode):

5. Remove all configuration from managers:

    

    

6. Restore database backup only if you have provided your MySQL instance. Ignore this step if MySQL database was deployed by devo-ea-deployer in dockerized mode.

   If you have provided your MySQL instance (required on HA environments) you should restore the backup previously created.

   Restoring the backup from MySQL data depends on your environment, but a good starting point is to use mysql command. For example:

7. Restore manager and agent-repository backup files.

   Set appropriate value for date pattern 2021-08-12 in the example:

    

8. Restore MySQL docker volume only if your environment is no-HA mode (dockerized mode)

   Set appropriate value for date pattern 2021-08-12 in the example (check the date in step 13 of the deployment phase):

    

9. Start internal services only if your environment is no-HA (dockerized mode)

10. Restart all manager and agent repository services.

     

     

11. Check that service is working and agents are properly enrolled.

12. Uninstall OSQuery in each agent target host and reinstall agents if you upgraded agents to new version too.