The tags beginning with proxy.squid identify log events generated by the Squid caching proxy.
Devo recommends using rsyslog as a means to forward Squid log events to a Devo endpoint. This enables you to correctly tag the events and forward them to either a Devo Relay or directly to the Devo Cloud.
Tag structure
The full tag must have at least four levels. The first two are fixed as proxy.squid. The third level identifies the event's log type and must be one of access-clf, access-combine, access-lt, access-squid, access-squid-mime, or cache. The fourth level of the tag is required and should be the hostname of the Squid proxy server that generated the event.
Technology | Brand | Log format/type | Subtype |
|---|---|---|---|
| proxy | squid |
| required - Squid server hostname |
Therefore, the valid tags include:
- proxy.squid.access-clf.serverHostname
- proxy.squid.access-combined.serverHostname
- proxy.squid.access-lt.serverHostname
- proxy.squid.access-squid.serverHostname
- proxy.squid.access-squid-mime.serverHostname
- proxy.squid.cache.serverHostname
For more information, read more about Devo tags.
Squid access.log configuration
Squid offers several default formats for log events that you can generate (or not) using the logformat and access_log directives. For more information about default formats, see the Squid documentation.
To these formats, Devo proposes a more detailed log format called access-lt that is expressed like this:
logformat access-lt %{%F:%T%z}tl %>a:%>p %ui %un %<A "%rm %ru HTTP/%rv" "%{Referer}>h" "%{User-Agent}>h" "%{Cookie}>h" %>Hs %>st %tr %<st %mt %Ss:%Sh
Sending to the Devo endpoint
We suggest you confirm the location of the Squid log files as dictated in the squid.conf file. Then you can use rsyslog to forward the log events to a Devo endpoint, either a Devo relay or directly to the Devo Cloud.
Below are examples of how the access_log directive is used in this file to indicate the location for each log file.
cache_log /var/log/squid3/cache.log cache_store_log /var/log/squid3/store.log #Choose one or more formats for the access log access_log /var/log/squid3/accessSquid.log squid access_log /var/log/squid3/accessMime.log squidmime access_log /var/log/squid3/accessClf.log common access_log /var/log/squid3/accessCombined.log combined access_log /var/log/squid3/access-lt.log access-lt
Below is an example of how to monitor these log files using a rsyslog configuration file:
$template squid,"<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%" # File access $InputFileName /var/log/squid3/cache.log $InputFileTag proxy.squid.cache.myProxyName: $InputFileStateFile stat-file1-SquidCache $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-lt.log $InputFileTag proxy.squid.accesslt.myProxyName: $InputFileStateFile stat-file1-SquidAccessLt $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-clf.log $InputFileTag proxy.squid.accessclf.myProxyName: $InputFileStateFile stat-file1-SquidAccessClf $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-combined.log $InputFileTag proxy.squid.accesscombined.myProxyName: $InputFileStateFile stat-file1-SquidAccessCombined $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-squid.log $InputFileTag proxy.squid.accesssquid.myProxyName: $InputFileStateFile stat-file1-SquidAccessSquid $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/accessMime.log $InputFileTag proxy.squid.access-squid-mime.myProxyName: $InputFileStateFile stat-file1-SquidAccessSquidMime $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # SSL config for sending to Devo Cloud #$DefaultNetstreamDriver gtls # use gtls netstream driver #$DefaultNetstreamDriverCAFile /etc/rsyslog.d/ca.crt #$DefaultNetstreamDriverCertFile /etc/rsyslog.d/user.crt #$DefaultNetstreamDriverKeyFile /etc/rsyslog.d/user.key #$ActionSendStreamDriverMode 1 # require TLS for the connection #$ActionSendStreamDriverAuthMode x509/name #$ActionSendStreamDriverPermittedPeer collector if $syslogtag contains 'proxy.squid.' and $syslogfacility-text == 'local7' then @@DEVO-RELAY:PORT;squid :syslogtag, contains, "proxy.squid." ~
- Sending to a Devo Relay: Replace DEVO-RELAY and PORT with the IP address and the port of your Devo In-house Relay.
- Sending to the Devo Cloud: Uncomment the SSL config section and be sure the security certificate files are correctly referenced. Replace DEVO-RELAY and PORT with the hostname of the Devo Cloud and port 514.
Other ways to forward events
A log destination, besides being a file, can also be an external process such as local syslog machine or a ydp/tcp server. To send to an external process, you should include program name and full path. Squid uses a minimal protocol in all the messages sent to the process and controls the logs processing by using the first byte of each event. The external process, like logger, should be capable of sending to a remote syslog server.
logfile_daemon /path/to/helper_process access_log daemon:/var/log/squid/access.log access-lt
Another mechanism is to send to the machine's local syslog daemon, by specifying the destination as syslog:facility.priority. In this case, you should apply a rsyslog or syslog-ng filter for the given facility.priority to tag all the events coming through it as proxy.squid.access-lt.myProxyName and send them to the Devo endpoint.
access_log syslog:local4.info access-lt
Although Squid offers a TCP Receiver module for sending log lines to an external server, because the data is sent in text format, not syslog, this is not supported by Devo.