Virtual Machine Metrics in Azure collector

[ 1 Purpose ] [ 2 Example tables ] [ 3 Authorize It ] [ 4 Run It ] [ 5 Secure It ] [ 6 Monitor It ]

Purpose

An analyst wants to detect abusive resource consumption in Azure Virtual Machines. Using the VM Metrics Azure collector to send CPU and disk usage to Devo, the analyst will find machines with too much resource usage. As a result, the analyst will remove the malicious mining programs, preventing them from degrading service and stealing compute.

In addition to a collector for Virtual Machine Metrics, set up a collector to use the Azure monitor service to get virtual machine logs.

Metrics in Azure:

Metrics in Devo:

Example tables

Table	Description

Table	Description
cloud.azure.vm.metrics_simple	Virtual machine performance data in Azure

Authorize It

Microsoft credentials are confusing. Before beginning, be aware of the different credential fields.

This process will enable collection of metrics for existing and future virtual machines in an Azure subscription. If there are no virtual machines in the subscription, create one before beginning the authorization process.

In Azure, search for Entra ID.
Click App registrations in the left menu and click the app (or Service Principal) that you are going to use.
Register the application
In the Overview area, find the Application (client) ID and the Directory (tenant) ID.
Click Certificates & Secrets on the menu and create a new client secret by clicking the New client secret button.
Add the secret.

Data loss warning: At the secret expiration time, the collector will stop working until the secret is replaced.

Copy the secret value.

Save the client secret value. It will only be shown once.

The value contains a tilde ~ character.

Open Subscriptions.
Select the correct subscription and note the subscription ID.
Select Access control (IAM) in the left menu and click Add and Add role assignment.
Select the Monitoring Reader role.
Click “Select members” and add the VM Metrics application previously registered.
Confirm the changes with Review + Assign.

Run It

In the Cloud Collector App, create an Azure Collector instance using this parameters template, replacing the values enclosed in < >.

{
  "inputs": {
    "azure": {
      "credentials": {
        "client_id": "<CLIENT ID>",
        "client_secret": "<SECRET>",
        "subscription_id": "<SUBSCRIPTION>",
        "tenant_id": "<TENANT>"
      },
      "enabled": true,
      "id": "<UNIQUE ID>",
      "services": {
        "vm_metrics": {}
      }
    }
  }
}

The secret must be a secret value containing a tilde.

Secure It

Cryptominer

Detect excessive CPU usage which may be caused by mining software that is stealing CPU.

from cloud.azure.vm.metrics_simple where eq(metricName,"Percentage CPU"), isnotnull(average)
group every 1h by resourceId select avg(average) as cpu_percent
where cpu_percent>90

Load balancer

A malicious user has gained access to Azure Load Balancer. Unbalancing network load would be a devious way to degrade service and increase costs. Check if the network load has become unbalanced.

from cloud.azure.vm.metrics_simple 

//Traffic going in and out.
where endswith(metricName,"Flows")

//Each collector corresponds to an Azure subscription.
//Group by collector under the assumption that each subscription should be load balanced.
group every 1h by split(hostname,"-",1) as collector,metricName,resourceId select avg(average) as average
group every 1h by collector,metricName select stddev(average) as deviation_across_resources
//If the deviation_across_resources has increased, load balancing has been disrupted.

Monitor It

Create an inactivity alert to detect interruptions of transfer of VM metrics using the query

from cloud.azure.vm.metrics_simple
where toktains(hostchain,"collector-") 
select split(hostchain,"-",1) as collector_id

Set the inactivity alert to keep track of the collector_id and resourceId. Metrics may be interrupted if a VM is turned off or removed.

Devo latest