Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Jun 26 20:32:49 relayHostnameHere devo-ng-relay[28831]: File "/opt/devo/ng-relay/conf/relay/run/rules/rule-13030.pconf", line 10, in <module>
Jun 26 20:32:49 relayHostnameHere devo-ng-relay[28831]: rule (
Jun 26 20:32:49 relayHostnameHere devo-ng-relay[28831]: File "/opt/devo/ng-relay/conf/relay/run/rule.py", line 78, in rule
Jun 26 20:32:49 relayHostnameHere devo-ng-relay[28831]: raise "A target tag with data references but no source data"
Jun 26 20:32:49 relayHostnameHere devo-ng-relay[28831]: TypeError: exceptions must be old-style classes or derived from BaseException, not str

Issues pertaining to Java memory error logs

If you query the logs under syslog.scoja.target table and uses Select raw and the following message appears in the “raw” container message (in the Devo UI, this is a column labeled “raw” within the Data Search tab after you run a query) of multiple critical, warnings, and error logs (you can hover over the “raw” column and press “p” for the ‘raw’ message):

Error while sending with [a syslog sender through [a reusing transport syslogger to SSLTransport[on: ConnectTransport[on: NBTCPTransport[on: restrictedproxy.tycoelectronics.com/135.107.16.139:80, with: TCPConf[timeout: 0, reuse address: true, send buffer size: 131072, receive buffer size: <nothing>, keep alive: true, no delay: <nothing>, traffic class: <nothing>]], with: ConnectConf[keep alive: true, user agent: <nothing>]], with: SSLConf[protocols: <nothing>, cipher suites: <nothing>, client authentication: REQUIRE, key managers: [sun.security.ssl.SunX509KeyManagerImpl@3eca82ee], trust managers: [sun.security.ssl.X509TrustManagerImpl@e80505a], secure random: <nothing>]], buffered with FilePairFall[memory max size: 671088640, work directory: /data/logt/buffer/rule, memory flush speed: 16, disk max size: 110100480000, storage size: 502824960, total events: 29477268, done events: 28992817, pending events: 484451, dropped events: 0, is first partial: false, is partial allowed: false], no pending partial, no pending terminator, 0 logs dropped, with no packet limit, with default priority to user.notice, sending timestamp, sending host, with default host usdcalog112, with default tag xcoja, with default terminator \x00]] java.lang.OutOfMemoryError: Java heap space%0A

Try increasing the memory on the Relay. In certain cases increasing it to 6GB of RAM has proven to work.

Relay runs into IndexOutOfBoundsException

When sending events through a relay, it’s possible to run into an IndexOutOfBoundsException if the regex (regular expression) in the relay rule you specified doesn’t conform to the messages that you’re sending to the relay. For example, if you have the event bsssbssssbsss and you have a regex rule that says locate the third b and return everything until the end of the message, the result would be sss. However, if you sent the event bsssbssss using the same regex rule, you would get the IndexOutOfBoundsException because you're trying to access a character that doesn't exist and goes beyond what's contained in the event.

  1. Analyze and verify that the error message indicates an out of bounds exception related to the regex rule.

  2. Analyze the messages that are being sent through the relay and compare them to the regex rule you have defined in the relay to see what inconsistencies there are in terms of the format.

  3. Modify the regex rule in the relay so that it conforms to the format of the events that you’re sending through the relay.

Relay rules not applied

Relay rules don’t seem to be applied on the relay after attempting to add a rule within Devo's web GUI.

  1. Check the table for events showing that a new configuration was downloaded and deployed.

  2. Validate rule added to relay via CLI. Check the syslog.relay.conf table to confirm the rule has been applied to the relay.

  3. If neither shows up then double-check the relay's connectivity.

In the event the buffer becomes full

Usually following a loss in connectivity, it can become impossible for the relay service to resume sending events. In that event, an administrator should:

  1. Stop the relay service: systemctl stop devo-ng-relay

  2. Manually delete the buffer directory located at var/logt/buffer/rule

  3. Manually start the service again and verify that the syslog sender begins shipping logs to the platform: systemctl start devo-ng-relay and checking the sending part with devo-ng-relay-cli and test command

Note

Clearing the buffer will result in events stored in the local buffer not being sent to the platform and being permanently removed.