NXLog

NXLog is a tool for collecting, processing, and forwarding log events.

This article is meant to provide guidance and examples for how to configure NXLog to forward events to Devo and it assumes a general understanding of how NXLog configuration files are structured. That said, there are often several ways of accomplishing the same thing so these instructions should be considered as guidance and not as law. We refer you to the NXLog reference manual for further information about any of the NXLog modules, directives, or procedures mentioned in the article.

This article describes how to set up parts of the configuration file to:

1 Read events from a log file
2 Apply a fixed tag to all events read from a syslog file
3 Apply a dynamic tag to events read from a file
4 Send events securely and directly to Devo
5 Send events to a Devo Relay

Read events from a log file

To read events from a log file, you can use the im_file input module. Read more about im_file here.

This module requires at least one directive, file, to specify the location and name of the file to read. Without including any other directives, the default behavior of this module will:

Monitor up to 10 files that match the file specification
Keep input files open even when the latest event has been sent
Check for modifications to files every 2 seconds
Check for new files every 1 second
Save the position of the last sent event and start from there the next time it's run

Here's an example of an input module that will read all *.txt files in a specified folder on Windows:

<Input messages>
    Module  im_file
    File    "C:\\ProgramData\\product\\logs\\*.txt"
</Input>

Here's an example of an input module that will read from a file on Linux:

<Input messages>
    Module  im_file
    File    "/var/log/messages/filename*.log"
</Input>

Apply a fixed tag to all events read from a syslog file

When a program outputs its logs to a file in the syslog protocol format, we can use an Exec directive in the input module calling the parse_syslog() procedure. This allows us to assign the Devo tag by setting the $SourceName field; the field that corresponds to the tag part of the syslog message.

Because we are calling syslog procedures and fields, we need to include the xm_syslog extension module. Read more about xm_syslog here.

<Extension syslog>
	Module	xm_syslog
</Extension>

...

<Input cisco_anyconn>
    Module  im_file
    File    "/var/log/messages/filename*.log"
	Exec	parse_syslog()
	Exec	$SourceName = "vpn.cisco.asa.anyconnect";
</Input>

We can also assign the tag in the output module. In that case, we include one Exec directive calling the to_syslog_bsd() procedure and another Exec to assign the $SourceName field.

Apply a dynamic tag to events read from a file

Sometimes, the correct Devo tag to apply to a single event will depend upon content found within the raw event. This is the case with Meraki Network Security logs so we'll use them as an example.

Meraki logging uses syslog protocol format so we will call the extension module xm_syslog. We'll use the im_file module and set the File directive to read all files in the specified directory that match the meraki*.log name pattern. Then we include one Exec directive calling the parse_syslog() procedure. The second Exec directive uses a regular expression to reflect the pattern of the event headers and identify the defining value as a capturing group. Now when we assign the $SourceName field, we specify the concatenation of "network.meraki." and the value inside the capturing group.

Send events securely and directly to Devo

To forward events directly and securely using SSL to Devo, use the om_ssl output module and we define a Route module that sends an input to an output. The minimum required directives for om_ssl are Host , Port, CAFile, CertFile, and CertKeyFile. These provide the destination and secure accesss to your Devo domain. Read more about om_ssl here.

To download the certificate files, log into your Devo domain and go to Administration → Credentials, then open the X.509 Certificates tab.

We also need to include an Exec directive calling the to_syslog_bsd() procedure - it uses the event's fields to build a BSD syslog formatted log message. Finally, we also include an OutputType directive calling the Syslog_TLS procedure that prepends the payload size to the message.

Here's a simple example that includes the input we described earlier, output to a Devo cloud, and a Route module that sends the input to the output.

In the example above, the Devo tag is applied in the input module. It is also possible to do so in an output module. Whether you do it one way or the other depends on your overall strategy for managing the NXLog activity. For example, you may choose to keep your outputs as "generic" as possible so that they can be used for several different inputs. Alternatively, you can create a single output to Devo containing multiple rules using Exec directives that apply tags to events coming from multiple inputs.

Send events to a Devo Relay

To forward events to a Devo relay that sits within your network, use the om_tcp output module. If the output module is only meant to forward events already tagged in their input module, then it's only necessary to indicate the Host as the IP address of the Devo Relay and the Port as 13000. Port 13000 is one of the 4 predefined relay rules and simply forwards them to Devo.

However, if you will use the relay to apply the Devo tag(s), you will need to indicate the port on which you have set up the relay rule for processing the events.

This configuration can also be used as a guide for routing events to a central log server that resides in the same network.

7.13