Versions Compared

Old Version 3

changes.mady.by.user Former user

Saved on

compared with

New Version Current

changes.mady.by.user Juan Tomás Alonso Nieto

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
maxLevel2
minLevel2
typeflat

Introduction

The tags beginning with web.

...

nginx identify

...

events generated by the NGINX web server belonging to NGINX.

...

Valid tags and data tables 

The full tag must have at least six 6 levels. The first two are fixed as web.nginx. The third level identifies the log type /format and currently must be one of access-main, access-combined, access-lt, access-lt-xff, or error.

The fourth, fifth and sixth levels are required and should identify the environment type, web application, and instance respectively. 

  • environment - Describes the environment in when the event occurred. For example, development, testing, or production.
  • web application - The name of the web application.
  • clon - This is the instance that generated the event. Depending on your network, this can be a machine name, or the virtual name of an NGINX process.

The values of these levels should be guided by the structure we propose because they will be saved in the events when saved in Devo. When you open the resulting data table, these will appear in the environment, site and clon columns.

...

Technology

...

Brand

...

Log type/format

...

Environment

...

Web application

...

Clon

...

  • access-main
  • access-combined
  • access-lt
  • access-lt-xff
  • error

free but required

...

Therefore, the valid tags include:

of events sent and the rest of them indicate the event subtypes (environment, application and clon).

Product / Service

Tags

Data tables

NGINX web server

web.nginx.access-combined.<env>.<app>.<clon>

web.nginx.accessCombined

web.nginx.access-

...

lt.

...

<env>.

...

<app>.

...

<clon>

web.nginx.

...

accessLt

web.nginx.access-lt

...

-xff.<env>.<app>.<clon>

web.nginx.accessLtXff

web.nginx.access-

...

main.<env>.<app>.<clon>

web.nginx.accessMain

web.nginx.error

...

.<env>.<app>.<clon>

web.nginx.error

For more information, read more about Devo tags.

...

Expand
titleEvent formats

The format and location of the different access logs are defined using the log_format and access_log directives within the

...

http

...

block of the nginx.conf file. Below find the specifications for each of the access log types supported by Devo.

...

  • The log file that corresponds to the web.nginx.access-

...

Code Block
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main;

  • combined tag is defined by default.

  • Specification for web.nginx.access-lt:

    Code Block
    log_format accesslt '$msec $remote_addr - $remote_user $host:$server_port "$request" "$uri" "$http_referer" "$http_user_agent" "$http_cookie" 
					$status $request_completion $request_length $request_time $upstream_response_time $body_bytes_sent $bytes_sent $gzip_ratio 
					"$http_content_type" "$upstream_http_content_type"';
access_log  /var/log/nginx/access.log  accesslt;

  • Specification for web.nginx.access-lt-xff:

    Code Block
    log_format accessltxff '$msec $remote_addr "$http_x_forwarded_for" - $remote_user $host:$server_port "$request" "$uri" "$http_referer" "$http_user_agent" "$http_cookie" 
						$status $request_completion $request_length $request_time $upstream_response_time $body_bytes_sent $bytes_sent $gzip_ratio 
						"$http_content_type" "$upstream_http_content_type"';
access_log  /var/log/nginx/access.log  accessltxff;

...

  • Specification for web.nginx.access

...

  • -main:

    Code Block
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
				'$status $body_bytes_sent "$http_referer" '
				'"$http_user_agent" "$http_x_forwarded_for"';
access_log  /var/log/nginx/access.log  main;

  • The error log that corresponds to the web.nginx.error tag is defined by the error_log directive in the main context of the NGINX configuration file.

For more information about NGINX logging, see the NGINX documentation.

Note

Remember to restart the server after editing the configuration file.

For more information about NGINX logging, see the NGINX documentation.

...

How is the data sent to Devo?

We recommend that you use the rsyslog configuration files on the host machine to send log events to a Devo endpoint. See the Monitoring files using rsyslog and Secure sending using rsyslog articles for details about editing the rsyslog configuration file.

Note

Remember to restart rsyslog after editing the configuration file.

Table structure

These are the fields displayed in these tables:

Anchor
tag1
tag1
web.nginx.accessCombined

Field

Type

Source field name

Extra fields

eventdate

timestamp

 

environment

str

venv

site

str

vsite

clon

str

vclon

serverdate

timestamp

 

srcIp

ip4

 

user

str

 

method

str

 

url

str

 

protocol

str

 

statusCode

int4

 

responseLength

int4

 

referer

str

 

userAgent

str

 

srcIdentd

str

 

rawMessage

str

 

✓

hostchain

str

 

✓

hostname

str

 

tag

str

 

✓

Anchor
tag2
tag2
web.nginx.accessLt

Field

Type

Source field name

Extra fields

eventdate

timestamp

 

environment

str

venv

site

str

vsite

clon

str

vclon

rawMessage

str

 

✓

serverdate

timestamp

 

srcIp

ip4

 

serverName

str

 

serverPort

int4

 

user

str

 

method

str

 

rawUrl

str

 

protocol

str

 

statusCode

int4

 

url

str

 

referer

str

 

userAgent

str

 

connectionState

str

 

responseTime

float8

 

upstreamResponseTime

float8

 

requestLength

int8

 

responseLength

int8

 

bodyLength

int8

 

requestContentType

str

 

responseContentType

str

 

gzipRatio

float8

 

cookies

str

 

hostchain

str

 

✓

tag

str

 

✓

Anchor
tag3
tag3
web.nginx.accessLtXff

Field

Type

Field Transformation

Source field name

Extra fields

eventdate

timestamp

 

 

environment

str

 

venv

site

str

 

vsite

clon

str

 

vclon

serverdate

timestamp

 

 

srcIp

ip4

 

 

clientIp

ip4

Code Block
(length(split(xForwardedFor, ","), as, pc) = 1) ? ip4(xForwardedFor) : (length(pc) >= 2) ? (pc[0] = "127.0.0.1") ? ip4(trim(pc[1])) : ip4(pc[0]) : null

pc

as

xForwardedFor

serverName

str

 

 

serverPort

int4

 

 

user

str

 

 

method

str

 

 

rawUrl

str

 

 

protocol

str

 

 

statusCode

int4

 

 

url

str

 

 

referer

str

 

 

userAgent

str

 

 

connectionState

str

 

 

xForwardedFor

str

 

 

responseTime

float8

 

 

upstreamResponseTime

float8

 

 

requestLength

int8

 

 

responseLength

int8

 

 

bodyLength

int8

 

 

requestContentType

str

 

 

responseContentType

str

 

 

gzipRatio

float8

 

 

cookies

str

 

 

rawMessage

str

 

 

✓

hostchain

str

 

 

✓

tag

str

 

 

✓

Anchor
tag4
tag4
web.nginx.accessMain

Field

Type

Field Transformation

Source field name

Extra fields

eventdate

timestamp

 

 

environment

str

 

venv

site

str

 

vsite

clon

str

 

vclon

serverdate

timestamp

 

 

srcIp

ip4

 

 

proxyChain

str

 

 

clientIp

ip4

Code Block
(length(split(proxyChain, ","), as, pc) = 1) ? ip4(proxyChain) : (length(pc) >= 2) ? (pc[0] = "127.0.0.1") ? ip4(trim(pc[1])) : ip4(pc[0]) : null

proxyChain

pc

as

user

str

 

 

method

str

 

 

url

str

 

 

protocol

str

 

 

statusCode

int4

 

 

responseLength

int8

 

 

referer

str

 

 

userAgent

str

 

 

srcIdentd

str

 

 

rawMessage

str

 

 

✓

hostchain

str

 

 

✓

tag

str

 

 

✓

Anchor
tag5
tag5
web.nginx.

...

...

error

Field

Type

Source field name

Extra fields

eventdate

timestamp

 

environment

str

venv

site

str

vsite

clon

str

vclon

serverdate

timestamp

 

severity

str

 

pid

str

 

tid

str

 

cid

str

 

message

str

 

rawMessage

str

 

✓

hostchain

str

 

✓

tag

str

 

✓