Table of Contents | ||||||
---|---|---|---|---|---|---|
|
Introduction
The tags beginning with proxy.
squid squid
identify log events generated by the Squid caching proxy .
Devo recommends using rsyslog as a means to forward Squid log events to a Devo endpoint. This enables you to correctly tag the events and forward them to either a Devo Relay or directly to the Devo Cloud.
...
belonging to Squid.org.
Valid tags and data tables
The full tag must have at least four 4 levels. The first two are fixed as asproxy.squid
. The The third level identifies the event's log type and must be one of access-clf, access-combine, access-lt, access-squid, access-squid-mime, or cachetype of events sent. The fourth level of the tag is required and should be the hostname of the Squid proxy server that generated the event.
...
Technology
...
Brand
...
Log format/type
...
Subtype
...
proxy
...
squid
...
access-clf
access-combine
access-lt
access-squid
access-squid-mime
cache
...
required - Squid server hostname
Therefore, the valid tags include:
...
indicates the event subtype.
These are the valid tags and corresponding data tables that will receive the parsers' data:
Product / Service | Tags | Data tables |
---|---|---|
Squid caching proxy |
|
|
|
| |
|
| |
|
...
|
...
|
|
...
|
|
...
|
|
...
|
|
...
|
|
...
For more information, read more about Devo tags.
...
Expand | ||
---|---|---|
| ||
Access logsSquid offers several default formats for log events that you can generate (or not) using the logformat and access_log directives. For more information about default formats, see the Squid documentation. To these formats, Devo proposes a more detailed log format called access-lt that is expressed like this: Devo Access:
|
Table structure
These are the fields displayed in these tables:
Rw ui tabs macro | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
proxy.squid.accessClf
proxy.squid.accessCombined
proxy.squid.accessLt
proxy.squid.accessSquid
proxy.squid.accessSquidMime
proxy.squid.cache
|
How is the data sent to Devo?
Devo recommends using rsyslog as a means to forward Squid log events to a Devo endpoint. This enables you to correctly tag the events and forward them to either a Devo Relay or directly to the Devo Cloud.
Sending to the Devo endpoint
We suggest you confirm the location of the Squid log files as dictated in the squid.conf file. Then you can use rsyslog to forward the log events to a Devo endpoint, either a Devo relay or directly to the Devo Cloud.
/etc/squid/squid.conf file extract: Below are examples of how the access_log directive is used in this file to indicate the location for each log file.
/etc/squid/squid.conf file extract
Code Block |
---|
cache_log /var/log/squid3/cache.log cache_store_log /var/log/squid3/store.log #Choose one or more formats for the access log access_log /var/log/squid3/accessSquid.log squid access_log /var/log/squid3/accessMime.log squidmime access_log /var/log/squid3/accessClf.log common access_log /var/log/squid3/accessCombined.log combined access_log /var/log/squid3/access-lt.log access-lt |
/etc/rsyslog.d/45-squid.conf file: Below is an example of how to monitor these log files using a rsyslog configuration file:
...
Code Block |
---|
$template squid,"<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%" # File access $InputFileName /var/log/squid3/cache.log $InputFileTag proxy.squid.cache.myProxyName: $InputFileStateFile stat-file1-SquidCache $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-lt.log $InputFileTag proxy.squid.accesslt.myProxyName: $InputFileStateFile stat-file1-SquidAccessLt $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-clf.log $InputFileTag proxy.squid.accessclf.myProxyName: $InputFileStateFile stat-file1-SquidAccessClf $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-combined.log $InputFileTag proxy.squid.accesscombined.myProxyName: $InputFileStateFile stat-file1-SquidAccessCombined $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/access-squid.log $InputFileTag proxy.squid.accesssquid.myProxyName: $InputFileStateFile stat-file1-SquidAccessSquid $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # File access $InputFileName /var/log/squid3/accessMime.log $InputFileTag proxy.squid.access-squid-mime.myProxyName: $InputFileStateFile stat-file1-SquidAccessSquidMime $InputFileSeverity info $InputFileFacility local7 $InputFilePollInterval 1 $InputFilePersistStateInterval 1 $InputRunFileMonitor # SSL config for sending to Devo Cloud #$DefaultNetstreamDriver gtls # use gtls netstream driver #$DefaultNetstreamDriverCAFile /etc/rsyslog.d/ca.crt #$DefaultNetstreamDriverCertFile /etc/rsyslog.d/user.crt #$DefaultNetstreamDriverKeyFile /etc/rsyslog.d/user.key #$ActionSendStreamDriverMode 1 # require TLS for the connection #$ActionSendStreamDriverAuthMode x509/name #$ActionSendStreamDriverPermittedPeer collector if $syslogtag contains 'proxy.squid.' and $syslogfacility-text == 'local7' then @@DEVO-RELAY:PORT;squid :syslogtag, contains, "proxy.squid." ~ |
Sending to a Devo Relay: Replace DEVO-RELAY and PORT with the IP address and the port of your Devo In-house Relay.
Sending to the Devo Cloud: Uncomment the SSL config section and be sure the security certificate files are correctly referenced. Replace DEVO-RELAY and PORT with the hostname of the Devo Cloud and port 514.
Other ways to forward events
A log destination, besides being a file, can also be an external process such as local syslog machine or a ydp/tcp server. To send to an external process, you should include program name and full path. Squid uses a minimal protocol in all the messages sent to the process and controls the logs processing by using the first byte of each event. The external process, like logger, should be capable of sending to a remote syslog server.
Forwarding to an external process
Code Block |
---|
logfile_daemon /path/to/helper_process access_log daemon:/var/log/squid/access.log access-lt |
Another mechanism is to send to the machine's local syslog daemon, by specifying the destination as syslog:facility.priority
. In this case, you should apply a rsyslog or syslog-ng filter for the given facility.priority
to tag all the events coming through it as proxy.squid.access-lt.myProxyName and send them to the Devo endpoint.
Forwarding to local syslog
Code Block |
---|
access_log syslog:local4.info access-lt |
...