Deployment overview

Requirements

To complete this process you will need:

• A valid user account in Devo.

• A valid user account in Amazon Web Services (AWS) with enough privileges to use EC2, ECS, VPC, etc.

• A valid user account in Zscaler ZIA with admin privileges.

How data is sent to Devo

1. Once the data feeds have been configured, Zscaler automatically sends the logs to the Nanolog Streaming Service (NSS) Servers instantiated in EC2.

2. The NSS instances send all the events through TCP (one TCP connection for each NSS Feed) to an EC2/ECS instance of the Devo Relay.

3. The Devo Relay tags and transfers the new events to Devo, where they will be stored.

Open

How a Zscaler NSS feed works

The Nanolog Streaming Service (NSS) uses a virtual machine (VM) to stream traffic logs in real-time from the Zscaler Nanolog to your security information and event management (SIEM) system, enabling real-time alerting, correlation with the logs of your other devices, and long-term local log archival. Zscaler offers the following NSS subscriptions:

• NSS for web - Streams web and mobile traffic logs.

• NSS for firewall - Streams logs from the Zscaler next-generation firewall.

Open

As shown above, the web and mobile traffic logs and the firewall logs are stored in the Nanolog. When an organization deploys one NSS for web and mobile logs and another NSS for firewall logs, each NSS opens a secure tunnel to the Nanolog in the Zscaler cloud. The Nanolog then streams copies of the logs to each NSS in a highly compressed format to reduce bandwidth footprint; the original logs are retained on the Nanolog.

When an NSS receives the logs from the Nanolog, it unscrambles them, applies the configured filters to exclude unwanted logs, converts the filtered logs to the configured output format so they can be parsed by your SIEM, and then streams the logs to your SIEM over a raw TCP connection. Additionally, if your organization has Advanced Cloud Sandbox, you can open a Sandbox Detail Report based on the MD5 parameter that you retrieve from your logs in the SIEM.

The NSS requires minimal administration. After you deploy it, the NSS automatically polls the Zscaler service for updates and installs them.

There are two reliable log delivery mechanisms in NSS:

• NSS to Devo Relay - NSS buffers logs in the VM memory to increase its resiliency to transient network issues between the Devo Relay and NSS. If the connection drops, NSS will replay logs from the buffer, according to the Duplicate Logs setting.

• Nanolog to NSS - If the connectivity between our cloud and NSS is interrupted, NSS will miss logs that arrived to the Nanolog cluster during the interruption, and they won’t be delivered to the Devo Relay. Once the connection is restored, the NSS one-hour recovery allows the Nanolog to replay logs up to one hour back.

NSS buffering and log duplication

The NSS can buffer logs for at least one hour. If a Devo Relay goes offline for maintenance or if the connection between the NSS and the Devo Relay is disrupted, the NSS buffers the logs and sends them once the connection is re-established. There can be a difference between the time the Devo Relay becomes unavailable and the time that the NSS detects that the Devo Relay is unavailable. When you configure an NSS feed, you can use the Duplicate Logs option to control the period of time from before the NSS detected the Devo Relay for which it will resend logs.

For example, the Devo Relay went offline at 6:29:00 p.m., the NSS detected the lost connection at 6:30:00 p.m., and the connection was restored at 6:40:00 p.m.

• If Duplicate Logs was set to five minutes, the NSS will resend the logs from 6:25:00 onwards, after the connection is restored. It will send five minutes of duplicate logs.

• If Duplicate Logs was disabled, the NSS will resend the logs from 6:30:01 onwards, after the connection is restored.
  Each log record has a unique ID stored in the ‘recorded’ field that can be used to discover duplicate logs.