Snowflake Collector Migration Guide

Welcome to the migration guide for Snowflake Collector version 2.0.0! This guide will walk you through the process of updating your configuration from the old version (1.4.1) to the new version (2.0.0). The new version introduces significant improvements and changes to the configuration style to enhance performance, usability, and security.

Overview of Changes

The new configuration format introduces several key changes:

The persistence has changed. We used to store every Id of every log we sent to Devo. This is no longer the case. We only store the last timestamp ran.
historic_date_utc is no longer a required parameter to start the collector.

Preparing for Migration

Before starting the migration process, we recommend the following steps:

Backup Your Current Configuration: Always ensure you have a backup of your existing configuration files to prevent any data loss.
Review the New Configuration Documentation: Familiarize yourself with the new configuration options available in version 2.0.0.

Migration Steps

Step 1: Update Configuration

An example of the old and new configuration is shown below:

v1.4.1

{
  "global_overrides": {
    "debug": false
  },
  "inputs": {
    "snowflake": {
      "id": "<input_id>",
      "enabled": true,
      "warehouse": "<warehouse>",
      "pager": "<pager>",
      "credentials": {
        "username": "<username>",
        "password": "<password>",
        "account_identifier": "<account_identifier>"
      },
      "services": {
        "access_history": {
          "historic_date_utc": "<historic_date_utc>",
          "request_period_in_seconds": "<request_period_in_seconds>",
          "role": "<role>"
        },
        "login_history": {
          "historic_date_utc": "<historic_date_utc>",
          "request_period_in_seconds": "<request_period_in_seconds>",
          "role": "<role>"
        },
        "sessions": {
          "historic_date_utc": "<historic_date_utc>",
          "request_period_in_seconds": "<request_period_in_seconds>",
          "role": "<role>"
        },
        "custom_sql": {
          "types": ["custom_db"],
          "request_period_in_seconds": "<request_period_in_seconds>",
          "sql_query": "<sql_query>",
          "database": "<database>",
          "schema": "<schema>",
          "role": "<role>"
        }
      }
    }
  }
}

v2.0.0

{
  "global_overrides": {
    "debug": false
  },
  "inputs": {
    "snowflake": {
      "id": "<input_id>",
      "enabled": true,
      "warehouse": "<warehouse>",
      "credentials": {
        "username": "<username>",
        "password": "<password>",
        "account_identifier": "<account_identifier>"
      },
      "services": {
        "access_history": {
          "historic_date_utc": "<historic_date_utc>",
          "request_period_in_seconds": "<request_period_in_seconds>",
          "role": "<role>"
        },
        "login_history": {
          "historic_date_utc": "<historic_date_utc>",
          "request_period_in_seconds": "<request_period_in_seconds>",
          "role": "<role>"
        },
        "sessions": {
          "historic_date_utc": "<historic_date_utc>",
          "request_period_in_seconds": "<request_period_in_seconds>",
          "role": "<role>"
        },
        "custom_sql": {
          "types": ["custom_db"],
          "request_period_in_seconds": "<request_period_in_seconds>",
          "sql_query": "<sql_query>",
          "database": "<database>",
          "schema": "<schema>",
          "role": "<role>"
        }
      }
    }
  }
}

Troubleshooting

Common Issue 1: Collector complains about state file being wrong.
Fix: Update the short id with a new 5 digit number and if using historic_date_utc update that to new time you want to start from.

For a smooth transition You should stop the running collector. And see the timestamp of the last log from Devo. Then use that as the new historic_date_utc Because if you copy your old configuration to your new configuration it will start over to that time listed.

For more detailed troubleshooting, refer to the official documentation or contact support.

Conclusion

Following these steps should help you smoothly transition to the new version of Software Script. If you encounter any issues, don't hesitate to reach out for support.