Page Comparison

Overview

This guide will walk you through the process of updating your Microsoft Azure collector configuration from version 1.7x.1 x to version 12.80.0. This new version introduces significant improvements and changes to the configuration style to enhance performance, usability, and security.

Main updates

The new configuration format introduces several key changes:

• Multiple inputs: The configuration now supports multiple inputs to better represent the different data sources and access mechanisms (azure and azure_event_hub)

• Credential config parameters renamed: The credentials configuration field names now follow names that are consistent with Microsoft Azure documentation: tenant_id, client_id, client_secret.

• Azure Blob Storage checkpoint support: The configuration now accepts Azure Blob Storage related parameters in the queue-specific configuration: blob_storage_connection_string, blob_storage_container_name, blob_storage_account_name.

• Moved VM Metrics to dedicated service: The VM metrics input has been moved to a dedicated service. The customer service configuration is no longer valid.

• Moved Event Hub to dedicated service: The Event Hub input has been moved to a dedicated service. The customer service configuration is no longer valid.

Preparing for migration

Before starting the migration process, we recommend the following steps:

1. Backup your current configuration: Always ensure you have a backup of your existing configuration files to prevent any data loss.

2. Review the new configuration documentation: Familiarize yourself with the new configuration options available in version 12.80.0.

Migration steps

Step 1: Update credential configuration parameter field names

The credential configuration field names have been updated:

...

An example of the old and new configuration is shown below:

# Old version
(1.7.1)
credentials:
  app_id: <app_id_value>
  active_directory_id: <active_directory_id_value>
  subscription_id: <subscription_id_value>
  secret: <secret_value>

↓

# New version (1.8.0)
credentials:
  client_id: <client_id_value>
  tenant_id: <tenant_id_value>
  subscription_id: <subscription_id_value>
  client_secret: <client_secret_value>

Step 2: Update VM Metrics configuration

The VM Metrics service has been moved to the azure input and a dedicated vm_metrics service.

...

If you wish to continue from the old configuration, you must input the time of the latest in Devo in the start_time_in_utc field to indicate the time from which the puller will start collecting data.

Step 3: Update Event Hub configuration

The Event Hub service(s) have been moved to the azure_event_hub input and a dedicated event_hub service.

...

By default, the collector will begin pulling from the latest event in the queue if there is not already a pre-existing checkpoint. To ensure your migrated collectors fetch from the last event previously sent to Devo, identify the datetime of the last event in Devo for the relevant queue and input it into the override_starting_position field in the format "%Y-%m-%dT%H:%M:%SZ". When the collector begins pulling from the queue, the collector will begin fetching from the indicated datetime for the first checkpoint.

Step 4: Example before and after configuration

Putting it all together, see below for an example of the old and new configuration:

# Old version (1.7.1)
inputs:
  azure:
    id: 10001
    enabled: true
    credentials:
      app_id: app_id_acme
      active_directory_id: active_directory_id_acme
      subscription_id: subscription_id_acme
      secret: secret_acme
    environment: test_environment
    requests_limits:
      - period: 1d
        number_of_requests: -1
    services:
      my_service_1:
        request_period_in_seconds: 300
        types:
          - eh_services
        queues:
          queue_a:
            event_hub_name: the-event-hub-name
            consumer_group: the-consumer-group
            connection_str: the-connection-string
            events_use_autocategory: true
            compatibility_version: 1.2.1
            duplicated_messages_mechanism: global
            use_global_counter_per_queue: true
      my_service_2:
        request_period_in_seconds: 300
        types:
          - vm_metrics

↓

# New version (1.8.0)
inputs:
  azure:
    id: 100001
    enabled: true
    credentials:
      subscription_id: subscription_id_acme
      client_id: app_id_acme
      client_secret: secret_acme
      tenant_id: active_directory_id_acme
    environment: test-env
    services:
      vm_metrics:
        request_period_in_seconds: 300
  azure_event_hubs:
    id: 100001
    enabled: true
    credentials:
      subscription_id: subscription_id_acme
      client_id: app_id_acme
      client_secret: secret_acme
      tenant_id: active_directory_id_acme
    environment: test-env
    services:
      event_hubs:
        queues:
          queue_a:
            event_hub_name: the-event-hub-name
            event_hub_connection_string: the-connection-string
            consumer_group: the-consumer-group
            events_use_autocategory: true
            compatibility_version: 1.2.0
            duplicated_messages_mechanism: global
            override_starting_position: "2022-01-01T00:00:00Z" # Replace with the datetime of the last event in Devo. Otherwise, collector pulls from latest event for the first checkpoint.

Troubleshooting

• Common Issue 1: Description and solution.

• Common Issue 2: Description and solution.

...