Table of Contents | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Overview
This guide will walk you through the process of updating your Microsoft Azure collector configuration from version 1.7x.1 x to version 12.80.0. This new version introduces significant improvements and changes to the configuration style to enhance performance, usability, and security.
Main updates
The new configuration format introduces several key changes:
Multiple inputs: The configuration now supports multiple inputs to better represent the different data sources and access mechanisms (
azure
andazure_event_hub
)Credential config parameters renamed: The credentials configuration field names now follow names that are consistent with Microsoft Azure documentation:
tenant_id
,client_id
,client_secret
.Azure Blob Storage checkpoint support: The configuration now accepts Azure Blob Storage related parameters in the queue-specific configuration:
blob_storage_connection_string
,blob_storage_container_name
,blob_storage_account_name
.Moved VM Metrics to dedicated service: The VM metrics input has been moved to a dedicated service. The customer service configuration is no longer valid.
Moved Event Hub to dedicated service: The Event Hub input has been moved to a dedicated service. The customer service configuration is no longer valid.
Preparing for migration
Before starting the migration process, we recommend the following steps:
Backup your current configuration: Always ensure you have a backup of your existing configuration files to prevent any data loss.
Review the new configuration documentation: Familiarize yourself with the new configuration options available in version 12.80.0.
Migration steps
Step 1: Update credential configuration parameter field names
The credential configuration field names have been updated:
...
An example of the old and new configuration is shown below:
Code Block |
---|
# Old version
(1.7.1)
credentials:
app_id: <app_id_value>
active_directory_id: <active_directory_id_value>
subscription_id: <subscription_id_value>
secret: <secret_value> |
↓
Code Block |
---|
# New version (1.8.0)
credentials:
client_id: <client_id_value>
tenant_id: <tenant_id_value>
subscription_id: <subscription_id_value>
client_secret: <client_secret_value> |
Step 2: Update VM Metrics configuration
The VM Metrics service has been moved to the azure
input and a dedicated vm_metrics
service.
...
If you wish to continue from the old configuration, you must input the time of the latest in Devo in the start_time_in_utc
field to indicate the time from which the puller will start collecting data.
Step 3: Update Event Hub configuration
The Event Hub service(s) have been moved to the azure_event_hub
input and a dedicated event_hub
service.
...
By default, the collector will begin pulling from the latest event in the queue if there is not already a pre-existing checkpoint. To ensure your migrated collectors fetch from the last event previously sent to Devo, identify the datetime of the last event in Devo for the relevant queue and input it into the override_starting_position
field in the format "%Y-%m-%dT%H:%M:%SZ". When the collector begins pulling from the queue, the collector will begin fetching from the indicated datetime for the first checkpoint.
Step 4: Example before and after configuration
Putting it all together, see below for an example of the old and new configuration:
Code Block |
---|
# Old version (1.7.1)
inputs:
azure:
id: 10001
enabled: true
credentials:
app_id: app_id_acme
active_directory_id: active_directory_id_acme
subscription_id: subscription_id_acme
secret: secret_acme
environment: test_environment
requests_limits:
- period: 1d
number_of_requests: -1
services:
my_service_1:
request_period_in_seconds: 300
types:
- eh_services
queues:
queue_a:
event_hub_name: the-event-hub-name
consumer_group: the-consumer-group
connection_str: the-connection-string
events_use_autocategory: true
compatibility_version: 1.2.1
duplicated_messages_mechanism: global
use_global_counter_per_queue: true
my_service_2:
request_period_in_seconds: 300
types:
- vm_metrics |
↓
Code Block |
---|
# New version (1.8.0)
inputs:
azure:
id: 100001
enabled: true
credentials:
subscription_id: subscription_id_acme
client_id: app_id_acme
client_secret: secret_acme
tenant_id: active_directory_id_acme
environment: test-env
services:
vm_metrics:
request_period_in_seconds: 300
azure_event_hubs:
id: 100001
enabled: true
credentials:
subscription_id: subscription_id_acme
client_id: app_id_acme
client_secret: secret_acme
tenant_id: active_directory_id_acme
environment: test-env
services:
event_hubs:
queues:
queue_a:
event_hub_name: the-event-hub-name
event_hub_connection_string: the-connection-string
consumer_group: the-consumer-group
events_use_autocategory: true
compatibility_version: 1.2.0
duplicated_messages_mechanism: global
override_starting_position: "2022-01-01T00:00:00Z" # Replace with the datetime of the last event in Devo. Otherwise, collector pulls from latest event for the first checkpoint. |
Troubleshooting
Common Issue 1: Description and solution.
Common Issue 2: Description and solution.
...