Relay migration

Introduction

The steps described in this article will guide you through the migration process to the most recent version of Devo Relay in an Ubuntu or CentOS machine.

The process is as follows:

• Install the new relay in the same machine where the old relay is located with exactly the same relay name so the same rules are downloaded for the new one. At this point, both relays will be running at the same time.

• Stop the old relay and wait until the new relay takes over and starts processing the data. This process should take a few seconds. If something goes wrong, start the old relay again and check for issues. Visit support site if you need support.

• Once the new relay is handling the traffic, remove the old relay from the machine.

Following these steps, data loss will be minimized as much as possible.

Getting ready

Make sure there is enough memory left for the new relay to start. In the beginning, both relays will be running at the same time. If the relay is configured to start with 70% of the available RAM, it won’t be possible to start the new relay at the same time. Edit /etc/init.d/devo-scoja-relay and set either a fixed amount of RAM of 30% or below to allow the new relay to run. 

Once you have updated the startup file, restart the old relay with the following commands:

Migration procedure

Phase 1 - Install the new relay

1. In the same machine where the old relay is installed, install the devo-ng-relay and devo-ng-relay-cli packages.

2. Set up the memory for the new relay in case it is needed by editing the following file: /opt/devo/ng-relay/conf/relay/devo-ng-relay.jvmoptions. In this example, we are setting 400 MB for the new relay.

   ## JVM Options File ## The options are expressed as a list of new-line-separated JVM flags -Xms400M -Xmx400M

   If changes in the file are performed, you must restart the service.

   sudo systemctl restart devo-ng-relay.service

3. Check both Java processes are running.

   $ ps aux | grep java root 5953 2.0 23.8 2721048 239612 pts/0 Sl 10:55 0:08 java -XX:+UseConcMarkSweepGC -XX:MaxRAMPercentage=30.0 -XX:InitialRAMPercentage=30.0 -classpath /opt/devo/scoja-server/lib/scoja-server-1.5.1.jar:/opt/devo/scoja-server/lib/scoja-client-1.5.1.jar:/opt/devo/scoja-server/lib/scoja-beep-1.4.0.jar:/opt/devo/scoja-server/lib/scoja-rpc-1.4.0.jar:/opt/devo/scoja-server/lib/scoja-compression-1.4.0.jar:/opt/devo/scoja-server/lib/scoja-rpc-xc-1.4.0.jar:/opt/devo/scoja-server/lib/scoja-cc-1.4.0.jar:/opt/devo/scoja-server/lib/jython-standalone-2.7.2b3.jar:/opt/devo/scoja-server/lib/log4j-1.2.17.jar org.scoja.server.Scoja -r 5s -G /opt/devo/scoja-server/conf/all-me.conf -j /opt/devo/scoja-server/conf/all-var.conf devo 7214 0.2 55.9 4754512 564304 ? Ssl 09:07 1:02 java -Dfile.encoding=UTF8 -DMADURO_CONF=/opt/devo/ng-relay/conf/ @/opt/devo/ng-relay/conf/relay/devo-ng-relay.jvmoptions -classpath /opt/devo/ng-relay/lib/maduro-server-2.1.0-SNAPSHOT.jar:/opt/devo/ng-relay/lib/maduro-api-client-1.4.0-SNAPSHOT.jar:/opt/devo/ng-relay/lib/maduro-common-1.4.0-SNAPSHOT.jar:/opt/devo/ng-relay/lib/scoja-server-.5.2.jar:/opt/devo/ng-relay/lib/api-clients-serrea-0.2.1.jar:/opt/devo/ng-relay/lib/jenga-echo-1.4.5.jar:/opt/devo/ng-relay/lib/jenga-monitoring-1.4.5.jar:/opt/devo/ng-relay/lib/httpclient-4.5.13.jar:/opt/devo/ng-lay/lib/crypass-1.0.2.jar:/opt/devo/ng-relay/lib/logback-syslog4j-1.0.0.jar:/opt/devo/ng-relay/lib/logback-classic-1.2.10.jar:/opt/devo/ng-relay/lib/logback-core-1.2.10.jar:/opt/devo/ng-relay/lib/janino-3.1.2.jar:/opt/devo/ng-relay/lib/commons-compress-1.21.jar:/opt/devo/ng-relay/lib/commons-configuration-1.10.jar:/opt/devo/ng-relay/lib/spring-boot-autoconfigure-2.6.3.jar:/opt/devo/ng-relay/lib/spring-boot-2.6.3.jar:/opt/devo/ng-relay/lib/spring-context-5.3.15.jar:/opt/devo/ng-relay/lib/spring-tx-5.2.19.RELEASE.jar:/opt/devo/ng-relay/lib/jinjava-2.6.0.jar:/opt/devo/ng-relay/lib/feign-jackson-10.2.3.jar:/opt/devo/ng-relay/lib/jackson-dataformat-yaml-.13.1.jar:/opt/devo/ng-relay/lib/jackson-databind-2.13.1.jar:/opt/devo/ng-relay/lib/jackson-annotations-2.13.1.jar:/opt/devo/ng-relay/lib/jackson-core-2.13.1.jar:/opt/devo/ng-relay/lib/feign-slf4j-10.2.3.jar:/opt/devo/ng-relay/lib/slf4j-api-1.7.32.jar:/opt/devo/ng-relay/lib/mysql-connector-java-8.0.28.jar:/opt/devo/ng-relay/lib/javax.annotation-api-1.3.2.jar:/opt/devo/ng-relay/lib/lutier-1.18.0.jar:/opt/devo/ng-relay/lib/scoja-beep-.4.0.jar:/opt/devo/ng-relay/lib/scoja-compression-1.4.0.jar:/opt/devo/ng-relay/lib/scoja-rpc-1.4.0.jar:/opt/devo/ng-relay/lib/scoja-client-1.5.2.jar:/opt/devo/ng-relay/lib/scoja-rpc-xc-1.4.0.jar:/opt/devo/ng-lay/lib/scoja-cc-1.4.0.jar:/opt/devo/ng-relay/lib/jython-standalone-2.7.2b3.jar:/opt/devo/ng-relay/lib/commons-codec-1.13.jar:/opt/devo/ng-relay/lib/feign-okhttp-10.2.3.jar:/opt/devo/ng-relay/lib/feign-core-10.2.3.jar:/opt/devo/ng-lay/lib/groovy-ant-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-astbuilder-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-cli-picocli-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-groovysh-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-console- 3.0.8.jar:/opt/devo/ng-relay/lib/groovy-datetime-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-groovydoc-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-docgenerator-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-jmx-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-json-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-jsr223-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-macro-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-nio-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-servlet-.0.8.jar:/opt/devo/ng-relay/lib/groovy-sql-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-swing-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-templates-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-test-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-test-junit5-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-testng-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-xml-3.0.8.jar:/opt/devo/ng-relay/lib/groovy-3.0.8.jar:/opt/devo/ng-relay/lib/guava-30.0-jre.jar:/opt/devo/ng-relay/lib/javassist-3.26.0-GA.jar:/opt/devo/ng-relay/lib/jsoup-1.14.2.jar:/opt/devo/ng-relay/lib/re2j-1.2.jar:/opt/devo/ng-relay/lib/commons-lang3-3.10.jar:/opt/devo/ng-relay/lib/commons-net-3.3.jar:/opt/devo/ng-relay/lib/java-ipv6-0.17.jar:/opt/devo/ng-relay/lib/annotations-3.0.1.jar:/opt/devo/ng-relay/lib/big-math-2.0.0.jar:/opt/devo/ng-relay/lib/spring-aop-5.3.15.jar:/opt/devo/ng-relay/lib/spring-beans-5.3.15.jar:/opt/devo/ng-relay/lib/spring-expression-5.3.15.jar:/opt/devo/ng-relay/lib/spring-core-5.3.15.jar:/opt/devo/ng-relay/lib/snakeyaml-1.30.jar:/opt/devo/ng-relay/lib/junit-4.13.2.jar:/opt/devo/ng-relay/lib/junit-platform-launcher-1.7.0.jar:/opt/devo/ng-relay/lib/junit-platform-engine-1.7.0.jar:/opt/devo/ng-relay/lib/junit-platform-commons-1.7.0.jar:/opt/devo/ng-relay/lib/junit-jupiter-engine-5.7.0.jar:/opt/devo/ng-relay/lib/junit-jupiter-api-5.7.0.jar:/opt/devo/ng-relay/lib/hamcrest-core-1.3.jar:/opt/devo/ng-relay/lib/micrometer-registry-prometheus-1.6.1.jar:/opt/devo/ng-relay/lib/commons-logging-1.2.jar:/opt/devo/ng-relay/lib/commons-lang-2.6.jar:/opt/devo/ng-relay/lib/okhttp-4.9.1.jar:/opt/devo/ng-relay/lib/protobuf-java-3.19.4.jar:/opt/devo/ng-relay/lib/jsr305-3.0.2.jar:/opt/devo/ng-relay/lib/ant-junit-1.10.9.jar:/opt/devo/ng-relay/lib/ant-1.10.9.jar:/opt/devo/ng-relay/lib/ant-launcher-1.10.9.jar:/opt/devo/ng-relay/lib/ant-antlr-1.10.9.jar:/opt/devo/ng-relay/lib/picocli-4.5.2.jar:/opt/devo/ng-relay/lib/qdox-1.12.1.jar:/opt/devo/ng-relay/lib/javaparser-core-3.18.0.jar:/opt/devo/ng-relay/lib/jline-2.14.6.jar:/opt/devo/ng-relay/lib/testng-7.4.0.jar:/opt/devo/ng-relay/lib/failureaccess-1.0.1.jar:/opt/devo/ng-relay/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/devo/ng-relay/lib/checker -qual-3.5.0.jar:/opt/devo/ng-relay/lib/error_prone_annotations-2.3.4.jar:/opt/devo/ng-relay/lib/j2objc-annotations-1.3.jar:/opt/devo/ng-relay/lib/syslog4j-0.9.30.jar:/opt/devo/ng-relay/lib/commons-compiler-3.1.2.jar:/opt/devo/ng-relay/lib/httpcore-4.4.13.jar:/opt/devo/ng-relay/lib/trove4j-3.0.3.jar:/opt/devo/ng-relay/lib/jcommander-1.78.jar:/opt/devo/ng-relay/lib/jquery-3.5.1.jar:/opt/devo/ng-relay/lib/spring-jcl-5.3.15.jar:/opt/devo/ng-relay/lib/opentest4j-1.2.0.jar:/opt/devo/ng-relay/lib/okio-jvm-2.8.0.jar:/opt/devo/ng-relay/lib/kotlin-stdlib-1.4.10.jar:/opt/devo/ng-relay/lib/micrometer-core-1.6.1.jar:/opt/devo/ng-relay/lib/simpleclient_common-0.9.0.jar:/opt/devo/ng-relay/lib/kotlin-stdlib-common-1.4.10.jar:/opt/devo/ng-relay/lib/annotations-13.0.jar:/opt/devo/ng-relay/lib/HdrHistogram-2.1.12.jar:/opt/devo/ng-relay/lib/LatencyUtils-2.0.3.jar:/opt/devo/ng-relay/lib/simpleclient-0.9.0.jarcom.devo.maduro.Maduro

   Note that PID 5953 is the old relay and PID 7214 is the new one. 

4. Open a new terminal to the machine and check ng-relay logs with the command sudo journalctl -u devo-ng-relay -f. You should see these logs:

   $ sudo journalctl -u devo-ng-relay -f -- Logs begin at Fri 2021-03-12 14:40:57 UTC. -- Mar 12 14:51:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:51:46.071 INFO [none] -- com.devo.lib.jenga.echo.JengaEcho : Starting server Mar 12 14:51:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:51:46.078 INFO [none] -- com.devo.lib.jenga.echo.JengaEcho : Server started on port: 12996 Mar 12 14:51:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:51:46.081 INFO [none] -- com.devo.lib.jenga.echo.JengaEcho : Listening at /0:0:0:0:0:0:0:0:12996 with endpoints: Mar 12 14:51:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:51:46.084 INFO [none] -- com.devo.lib.jenga.echo.JengaEcho : ➤ /system: class com.devo.lib.jenga.echo.handlers.SystemHandler Mar 12 14:51:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:51:46.085 INFO [none] -- com.devo.lib.jenga.echo.JengaEcho : ➤ /system/keepalive: class com.devo.maduro.server.healthcheck.KeepAlive Mar 12 14:51:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:51:46.108 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet. Mar 12 14:52:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:52:46.096 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet. Mar 12 14:53:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:53:46.094 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet. Mar 12 14:54:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:54:46.095 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet. Mar 12 14:55:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:55:46.096 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet. Mar 12 14:56:46 test-relay142-ubuntu20 devo-ng-relay[6379]: 14:56:46.095 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet.

5. Check for ports where the old relay is listening using sudo netstat -atunp | grep <old_relay_PID>

   $ sudo netstat -atunp | grep 5953 tcp6 0 0 :::13000 :::* LISTEN 5953/java tcp6 0 0 :::13001 :::* LISTEN 5953/java tcp6 0 0 :::13002 :::* LISTEN 5953/java tcp6 0 0 :::13003 :::* LISTEN 5953/java tcp6 0 0 :::13010 :::* LISTEN 5953/java tcp6 0 0 127.0.0.1:5140 :::* LISTEN 5953/java tcp6 0 0 127.0.0.1:5140 127.0.0.1:46614 ESTABLISHED 5953/java tcp6 0 0 127.0.0.1:5140 127.0.0.1:46568 ESTABLISHED 5953/java tcp6 0 0 10.0.2.15:52000 172.17.12.66:443 ESTABLISHED 5953/java udp6 0 0 :::12999 :::* 5953/java udp6 0 0 :::13000 :::* 5953/java udp6 0 0 :::13001 :::* 5953/java udp6 0 0 :::13002 :::* 5953/java udp6 0 0 :::13003 :::* 5953/java udp6 0 0 :::13010 :::* 5953/java udp6 0 0 127.0.0.1:5140 :::* 5953/java udp6 0 0 127.0.0.1:5141 :::* 5953/java

6. Run the Devo Relay CLI with the required command for your system (learn how to do it here) and then run the setup command using the exact same name for the relay. The exact same name can be taken from the file opt/devo/relay/conf/relay.conf in the hardware.id field.

7. You will see a message like this asking if you want to override the existing relay. Type Y.

   There already exists a relay with the name test-relay142-ubuntu20-13930 Do you want to override the existing relay test-relay142-ubuntu20-13930? [Y, n]:

   After setup is completed, you will see these messages in the relay logs.

   ... Mar 12 15:08:33 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:08:33.484 INFO [2234fb02de4a] -- com.devo.maduro.server.ActionVisitor : Setting Up.... Mar 12 15:08:34 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:08:34.187 WARN [2234fb02de4a] -- c.devo.maduro.server.relay.SetUpService : Trying to setting up an already existing relay: test-relay142-ubuntu20-13930 Mar 12 15:09:05 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:09:05.876 WARN [none] -- c.d.m.server.relay.ConfigUpdaterService : There is no relay configuration yet. Mar 12 15:09:34 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:09:34.993 INFO [4cd24e4bb7e3] -- com.devo.maduro.server.MaduroServerLite : Accept from /127.0.0.1 Mar 12 15:09:34 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:09:34.994 INFO [4cd24e4bb7e3] -- com.devo.maduro.server.ActionVisitor : Setting Up.... Mar 12 15:10:05 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:05.878 INFO [none] -- o.a.c.c.PropertiesConfiguration : Reloading configuration. URL is file:/opt/devo/ng-relay/conf/relay/relay.properties Mar 12 15:10:05 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:05.893 INFO [58e8d1dc4635] -- c.d.m.server.relay.ConfigUpdaterService : Checking status of the relay test-relay142-ubuntu20-13930 Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.252 INFO [58e8d1dc4635] -- c.d.m.server.relay.ConfigUpdaterService : [SUCCESS] Downloading and deploying configuration set Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.586 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/target.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.587 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/rule-13000.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.590 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/rule-13001.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.592 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/rule-13002.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.593 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/rule-12997.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.593 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/rule-13003.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.594 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/rule-13010.pconf... Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.595 INFO [58e8d1dc4635] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/ports.cfg... ...

8. If there is not enough memory for the syslog engine inside the relay (Scoja) to start, you will see the following error and Scoja will not start. To fix it, increase the heap size for the new relay process by editing the file /opt/devo/ng-relay/conf/relay/devo-ng-relay.jvmoptions

   Mar 12 15:10:06 test-relay142-ubuntu20 devo-ng-relay[6996]: 15:10:06.719 ERROR [58e8d1dc4635] -- c.d.m.server.relay.CorrectiveService : The relay cannot start. Insufficient memory configured. The relay needs at least 128 MB for memory buffering. Please increase the process heap size

   Otherwise, you will see scoja starting normally but the second scoja won’t be able to listen in the ports until the old relay is stopped. Please notice the “cannot listen!“ messages. They are normal.

   Mar 12 16:11:02 test-relay142-ubuntu20 devo-ng-relay[11764]: 16:11:02.131 INFO [595cc453a729] -- c.d.m.server.relay.RelayConfigFileUtil : Extracting rule rules/ports.cfg... Mar 12 16:11:02 test-relay142-ubuntu20 devo-ng-relay[11764]: 16:11:02.199 INFO [595cc453a729] -- c.d.m.server.relay.ConfigUpdaterService : Maduro context status: ACTIVE Mar 12 16:11:02 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:02 test-relay142-ubuntu20 syslog.scoja.configuration: (Re)Loading configuration file /opt/devo/ng-relay/conf/relay/run/all-me.conf Mar 12 16:11:02 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:02 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Queue "internal queue" with 1 queued events, 1 max threads, 1 threads started, 0 threads awaiting Mar 12 16:11:02 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:02 test-relay142-ubuntu20 syslog.scoja.configuration: (Re)Loading configuration file /opt/devo/ng-relay/conf/relay/run/parameters.conf Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/parameters.conf successfully parsed Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/parameters.conf successfully (re)loaded Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: (Re)Loading configuration file /opt/devo/ng-relay/conf/relay/run/me.conf Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/me.conf successfully parsed Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/me.conf successfully (re)loaded Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/all-me.conf successfully (re)loaded Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: (Re)Loading configuration file /opt/devo/ng-relay/conf/relay/run/all-var.conf Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/all-var.conf successfully parsed Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.configuration: Configuration file /opt/devo/ng-relay/conf/relay/run/all-var.conf successfully (re)loaded Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.measurer: Starting Measurer source Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Measurer source Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at localhost:5140 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at localhost:5140 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at localhost:5140 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at localhost:5140 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at localhost:5140 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at localhost:5141 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Opening server socket for Selecting TCP source listening at localhost:5140 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at localhost:5141 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 2 for UDP source listening at localhost:5141 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:12999 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at localhost:5141 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:12999 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 2 for UDP source listening at 0.0.0.0:12999 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at localhost:5141 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at localhost:5140 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Source Selecting TCP source listening at localhost:5140 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use%0A at sun.nio.ch.Net.bind0(Native Method)%0A at sun.nio.ch.Net.bind(Net.java:461)%0A at sun.nio.ch.Net.bind(Net.java:453)%0A at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)%0A at org.scoja.server.source.SelectingTCPSource.openSocket(SelectingTCPSource.java:268)%0A at org.scoja.server.source.SelectingTCPSource.ensureSocket(SelectingTCPSource.java:240)%0A at org.scoja.server.source.SelectingTCPSource.run(SelectingTCPSource.java:225)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:12999 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at 0.0.0.0:12999 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 2 for UDP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 2 for Selecting TCP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Opening server socket for Selecting TCP source listening at 0.0.0.0:12997 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 2 for UDP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at 0.0.0.0:13000 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 2 for Selecting TCP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:13010 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Opening server socket for Selecting TCP source listening at 0.0.0.0:13000 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Source Selecting TCP source listening at 0.0.0.0:13000 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use%0A at sun.nio.ch.Net.bind0(Native Method)%0A at sun.nio.ch.Net.bind(Net.java:461)%0A at sun.nio.ch.Net.bind(Net.java:453)%0A at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)%0A at org.scoja.server.source.SelectingTCPSource.openSocket(SelectingTCPSource.java:268)%0A at org.scoja.server.source.SelectingTCPSource.ensureSocket(SelectingTCPSource.java:240)%0A at org.scoja.server.source.SelectingTCPSource.run(SelectingTCPSource.java:225)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:13010 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at 0.0.0.0:13010 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:13010 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at 0.0.0.0:13010 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at 0.0.0.0:13010 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:13001 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Opening server socket for Selecting TCP source listening at 0.0.0.0:13010 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Source Selecting TCP source listening at 0.0.0.0:13010 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use%0A at sun.nio.ch.Net.bind0(Native Method)%0A at sun.nio.ch.Net.bind(Net.java:461)%0A at sun.nio.ch.Net.bind(Net.java:453)%0A at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)%0A at org.scoja.server.source.SelectingTCPSource.openSocket(SelectingTCPSource.java:268)%0A at org.scoja.server.source.SelectingTCPSource.ensureSocket(SelectingTCPSource.java:240)%0A at org.scoja.server.source.SelectingTCPSource.run(SelectingTCPSource.java:225)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:13001 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at 0.0.0.0:13001 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at 0.0.0.0:13001 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:13002 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Opening server socket for Selecting TCP source listening at 0.0.0.0:13001 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Source Selecting TCP source listening at 0.0.0.0:13001 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use%0A at sun.nio.ch.Net.bind0(Native Method)%0A at sun.nio.ch.Net.bind(Net.java:461)%0A at sun.nio.ch.Net.bind(Net.java:453)%0A at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)%0A at org.scoja.server.source.SelectingTCPSource.openSocket(SelectingTCPSource.java:268)%0A at org.scoja.server.source.SelectingTCPSource.ensureSocket(SelectingTCPSource.java:240)%0A at org.scoja.server.source.SelectingTCPSource.run(SelectingTCPSource.java:225)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:13002 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at 0.0.0.0:13002 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:13002 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at 0.0.0.0:13002 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at 0.0.0.0:13002 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Starting UDP source listening at 0.0.0.0:13003 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Opening server socket for Selecting TCP source listening at 0.0.0.0:13002 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Source Selecting TCP source listening at 0.0.0.0:13002 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use%0A at sun.nio.ch.Net.bind0(Native Method)%0A at sun.nio.ch.Net.bind(Net.java:461)%0A at sun.nio.ch.Net.bind(Net.java:453)%0A at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)%0A at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)%0A at org.scoja.server.source.SelectingTCPSource.openSocket(SelectingTCPSource.java:268)%0A at org.scoja.server.source.SelectingTCPSource.ensureSocket(SelectingTCPSource.java:240)%0A at org.scoja.server.source.SelectingTCPSource.run(SelectingTCPSource.java:225)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for UDP source listening at 0.0.0.0:13003 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.tcp: Starting Selecting TCP source listening at 0.0.0.0:13003 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:13001 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at 0.0.0.0:13001 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Opening server socket for UDP source listening at 0.0.0.0:13003 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.source.net.udp: Source UDP source listening at 0.0.0.0:13003 cannot listen! I will retry after 1.0 seconds. java.net.BindException: Address already in use (Bind failed)%0A at java.net.PlainDatagramSocketImpl.bind0(Native Method)%0A at java.net.AbstractPlainDatagramSocketImpl.bind(AbstractPlainDatagramSocketImpl.java:93)%0A at java.net.DatagramSocket.bind(DatagramSocket.java:392)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:242)%0A at java.net.DatagramSocket.<init>(DatagramSocket.java:299)%0A at org.scoja.server.source.UDPSource.openSocket(UDPSource.java:176)%0A at org.scoja.server.source.UDPSource.ensureSocket(UDPSource.java:144)%0A at org.scoja.server.source.UDPSource.run(UDPSource.java:129)%0A at java.lang.Thread.run(Thread.java:748)%0A Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.thread: Starting thread 1 for Selecting TCP source listening at 0.0.0.0:13003 Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.main: Reloader started to run every 5.0 seconds Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.main: Scoja started Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.target.file: Error while managing `/var/logt/local/2021/03/12/test-relay142-ubuntu20-13930/syslog/scoja/main.log': Permission denied Mar 12 16:11:05 test-relay142-ubuntu20 devo-ng-relay[11764]: Mar 12 16:11:05 test-relay142-ubuntu20 syslog.scoja.target.file: All data to file `/var/logt/local/2021/03/12/test-relay142-ubuntu20-13930/syslog/scoja/main.log' will ignored until Fri Mar 12 16:11:17 UTC 2021

Phase 2 - Stop the old relay and make sure the new one is handling the data

1. Stop the old relay using the following command.

   sudo /etc/init.d/devo-scoja-relay stop

2. Check that the new relay has started to listen to the ports that were being used by the old relay using the following command:

   sudo netstat -atunp | grep <new_relay_PID>.
   $ sudo netstat -atunp | grep 7214 tcp6 0 0 :::12996 :::* LISTEN 7214/java tcp6 0 0 :::12997 :::* LISTEN 7214/java tcp6 0 0 :::12998 :::* LISTEN 7214/java tcp6 0 0 :::13000 :::* LISTEN 7214/java tcp6 0 0 :::13001 :::* LISTEN 7214/java tcp6 0 0 :::13002 :::* LISTEN 7214/java tcp6 0 0 :::13003 :::* LISTEN 7214/java tcp6 0 0 :::13010 :::* LISTEN 7214/java tcp6 0 0 127.0.0.1:5140 :::* LISTEN 7214/java tcp6 0 0 10.0.2.15:38904 52.19.61.231:443 ESTABLISHED 7214/java tcp6 0 0 10.0.2.15:52986 172.17.12.66:443 ESTABLISHED 7214/java tcp6 1 0 127.0.0.1:47520 127.0.0.1:5140 CLOSE_WAIT 7214/java tcp6 0 0 127.0.0.1:12997 127.0.0.1:46414 ESTABLISHED 7214/java tcp6 1 0 127.0.0.1:47528 127.0.0.1:5140 CLOSE_WAIT 7214/java tcp6 0 0 10.0.2.15:52982 172.17.12.66:443 ESTABLISHED 7214/java udp6 0 0 :::12997 :::* 7214/java udp6 0 0 :::12999 :::* 7214/java udp6 0 0 :::13000 :::* 7214/java udp6 0 0 :::13001 :::* 7214/java udp6 0 0 :::13002 :::* 7214/java udp6 0 0 :::13003 :::* 7214/java udp6 0 0 :::13010 :::* 7214/java udp6 0 0 127.0.0.1:5140 :::* 7214/java udp6 0 0 127.0.0.1:5141 :::* 7214/java

   In case the connections don’t show up and the new relay process is not listening in all configured ports or events stop being ingested in Devo, start again the old relay:

   sudo /etc/init.d/devo-scoja-relay start

   Please make sure data is still reaching Devo before moving to the next phase.

3. Once the traffic is being handled by the new relay, adjust the memory allocation of the new relay process. We recommend allocating at least 2GB. Edit the file /opt/devo/ng-relay/conf/relay/devo-ng-relay.jvmoptions in case it's needed and restart the service.

   sudo systemctl restart devo-ng-relay.service

4. A test command from the Devo Relay CLI can also be executed to double-check that the sending from the new relay is working.

Phase 3 - Uninstall the old relay

To uninstall the old relay, execute the required command according to your OS: