Monitoring files using rsyslog

The instructions for setting up connections to a Devo Relay or Cloud require the creation of the 00-devo.conf and 49-devo.conf files and cover the sending of all system log events to the Devo endpoint.

For an example of the 00-devo.conf and 49-devo.conf files for sending events to a Devo relay, see Simple sending using rsyslog article. If you are planning to send the monitored files directly to the Devo Cloud, check out the 00-devo.conf and 49-devo.conf files in Secure sending using rsyslog.

This article explains how to use an additional /etc/rsyslog.d/*.conf file to send events from other logs on the host to the same Devo endpoint. You may create multiple .conf files depending on the variety of log files you need to send. Each file can include up to 100 different input files to monitor.

1 Create the configuration file
2 Ensure the required permissions are enabled
3 Restart rsyslog

Create the configuration file

For each log file that you want to monitor on the host, you will include a dedicated set of input parameters in the configuration file. These parameters are part of the imfile module loaded in 00-devo.conf. The sample below contains placeholder values to send logs to a Devo Relay:

template(
    name = "my-file-template"
    type = "string"
    string = "<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%"
)

input(
    type="imfile"
    File="/path/to/file.log"
    Tag="my.devo.tag:"
    Facility="user"
    Severity="info"
    PersistStateInterval="1"
    reopenOnTruncate="on"
    freshStartTail="on"

    ruleset="my-file-logging"
)

ruleset(name="my-file-logging") {
    action(
        type="omfwd"
        template="my-file-template"
        queue.type="LinkedList"
        queue.filename="fileq1"
        queue.saveonshutdown="on"
        action.resumeRetryCount="-1"
        Target="<your-devo-relay-ip>"
        Port="<your-devo-relay-ip>"
        Protocol="tcp"
    )
}

To send the logs directly to the Devo Cloud, you just need to add the specific TLS parameters:

/etc/rsyslog.d/45-filemonitor.conf file monitoring example

template(
    name = "my-file-template"
    type = "string"
    string = "<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%"
)

input(
    type="imfile"
    File="/path/to/file.log"
    Tag="my.devo.tag:"
    Facility="user"
    Severity="info"
    PersistStateInterval="1"
    reopenOnTruncate="on"
    freshStartTail="on"

    ruleset="my-file-logging"
)

ruleset(name="my-file-logging") {
    action(
        type="omfwd"
        template="my-file-template"
        queue.type="LinkedList"
        queue.filename="fileq1"
        queue.saveonshutdown="on"
        action.resumeRetryCount="-1"
        Target="xx.elb.relay.logtrust.net"
        Port="443"
        Protocol="tcp"
        StreamDriver="gtls"
        StreamDriverMode="1"
        StreamDriverAuthMode="x509/name"
        StreamDriverPermittedPeers="xx.elb.relay.logtrust.net"
    )
}

These are the parameters we recommend using to configure each input file.

`File`	The path and filename of the file to monitor.
`Tag`	The tag to assign to messages that come from the monitored file.
`Facility, Severity`	The syslog severity and facility to assign to messages from the input file.
`PersistStateInterval`	The number of events after which the processing status should be updated. This should be set to `1`.
`reopenOnTruncate`	This is an experimental feature that tells rsyslog to reopen the input file when it was truncated.
`freshStartTail`	This is used to tell rsyslog to seek to the end/tail of input files (discard old logs) at its first start (freshStartTail) and process only new log messages. This should be set to on.

Notice that, in this case, the action is inside a ruleset block. This means that it will be executed only by the input that references that ruleset (in this case, our file reading).

Ensure the required permissions are enabled

The user running rsyslog, usually syslog, must have permissions over all directories and files that you want to monitor. Here's an example of how you might change the owner:

chown :syslog /var/log/apache2 /var/log/apache2/*.log

If the input files are part of a logrotate policy and logrotate create mode is being used, ensure that syslog will still have permissions over the new file. For example, in Apache:

/etc/logrotate.d/apache2 configuration file extract

Restart rsyslog

Close the configuration file, then restart rsyslog.

Once restarted, go to Data Search in the Devo web application and look for the table that contains the events with the tag you just set up.

Devo v7.12.0