Microsoft Azure collector

To log in to the Azure subscription, the collector uses a Service Principal object. You need to get the subscription ID, Active Directory ID, Application ID (service principal identification), and the client secret (service principal "password"). To get them, follow these steps:

1. Log in to your Azure account and search for Azure Active Directory.

Open

2. Now, click App registrations in the left menu and click the app (or Service Principal) that you are going to use.

Open

3. In the Overview area, find the Application (client) ID and the Directory (tenant) ID.

4. Now click Certificates & Secrets on the menu and create a new client secret by clicking the New client secret button.

5. Get the subscription ID by searching for Subscriptions on the home page.

6. Find the correct subscription and note down the subscription ID.

1. After creating the App registration (or Service Principal), go to the desired Resource Group (or subscription if you want to retrieve metrics from all the available virtual machines).

2. Select Access control (IAM) in the left menu and click Add.

3. Select at least the Reader role and choose the previously created App registration.

4. Confirm the changes.

If you want to use Azure Blob Storage for checkpointing purposes, you need to create a storage account to store the checkpoints. If you do not wish to use Azure Blob storage (i.e. you will use Devo local persistence), you can skip the Blob Storage configuration steps.

Connection string

1. From the left portal menu, select Storage accounts to display a list of your storage accounts. If the portal menu isn't visible, select the menu button to toggle it on.

2. On the Storage accounts page, select Create.

3. After the storage account is created, select it from the list of storage accounts, click on Access keys in the left menu, and copy the connection string.

Role assignment

Alternatively, users can grant the necessary permissions to the registered application to access the Event Hub without using the RootManageSharedAccessKey. Roles can be assigned in a variety of ways (e.g. inherited from the subscription group), but the following steps will show how to assign the necessary roles directly to the Storage Account.

Repeat steps 1-2 from the Connection String section to create the Storage Account.

1. In the Storage Account, click Access control (IAM) in the left menu, click + Add, and click Add Access Role Assignment.

2. Search for either the Storage Blob Data Contributor or Storage Blob Data Owner role and select it and then click Next.

3. Click + Select members and search for the previously created App registration, select it, click Next.

4. Click Review + Assign.

Connection string

Users can either obtain a connection string or use Role Assignments to allow the collector to access the Event Hub.

1. In your Azure account, search for the Event Hubs service and click on it. 

2. Create an Event Hub resource per region (repeat the steps below for each region):

• Click Add.

• Fill the mandatory fields keeping in mind that the Event Hub must be in the same region as the resources that you are going to monitor (and only need one per region). The Throughput Units option refers to the ingress/egress limit in MB/s (each unit is 1 MB/s or 1000 events/second ingress, 2 MB/s, or 4096 events/second egress). You should adjust it according to the data volume (this can be modified later).

• The previous steps create an EventHub namespace; now go to Event Hubs, search the created one and click on it.

• Now click on the + Event Hub button and create a new resource. You only need to fill the Name and Partition Count fields (the Partition Count field will divide the data into different partitions to make it easier to read large volumes of data). Write down the EventHub name to be used later in the configuration file.

• Once the Event Hub is created in the namespace, click it and select Consumer Group in the left menu. Note that a dedicated Consumer Group for Devo needs to be created if the existing consumer groups are already in use.

• Here you will see the Event Hub consumer groups. This will be used by the collector (or other applications) for reading data from the Event Hub. Write down the Consumer group name that you will use later in the configuration file.

• Now, in the Event Hub Namespace, click on Shared access policies, search the default policy named RootManageSharedAccessKey and click it.

• Copy and write down the primary (or secondary) connection string to be used later in the configuration file.

Role assignment

Alternatively, users can grant the necessary permissions to the registered application to access the Event Hub without using the RootManageSharedAccessKey. Roles can be assigned in a variety of ways (e.g. inherited from the subscription group), but the following steps will show how to assign the necessary roles directly to the Event Hub Namespace.

Repeat all steps except the last one from the previous section to create the Event Hub.

1. In the Event Hub Namespace, click Access control (IAM) in the left menu, click + Add, and click Add Access Role Assignment.

2. Search for either the Azure Event Hubs Data Receiver or Azure Event Hubs Data Owner role and select it and then click Next.

3. Click + Select members and search for the previously created App registration, select it, click Next.

4. Click Review + Assign.

1. Now, search the Monitor service and click on it.

2. Click the Diagnostic Settings option in the left area.

3. A list of the deployed resources will be shown. Search for the resources that you want to monitor, select them, and click Add diagnostic setting.

4. Type a name for the rule and check the required category details (logs will be sent to the cloud.azure.eh.events table, and metrics will be sent to the cloud.azure.eh.metrics table).

5. Check Stream to an Event Hub, and select the corresponding Event hub namespace, Event hub name, and Event hub policy name.

6. Click Save to finish the process.