Troubleshoot a collector
- Laszlo Frazer
- Virginia Camuñas Núñez
Invalid JSON format
When editing lengthy collector parameters, there may be invalid JSON. Common causes are:
Missing or extra brackets.
Missing quotation marks.
Missing comma after a key-value statement.
Quotation marks have been replaced with smart quotes.
The collector is not the most recent version
Debug mode
Do not enable debug mode in the JSON parameters in production. Debug mode may incur additional costs. Devo may disable debug mode at any time.
An error occurred while creating the collector. Please try again later.
The limit on the number of collectors may have been reached. Contact us to get the limit adjusted.
Verify collector is running
Open the Cloud Collector App.
Select the collector name.
Select “Target pods.”
If the number on the right is zero, change it to one to start the collector.
If the number on the right is one and the number on the left is zero, the collector has not started. Contact us if it takes more than three minutes to start.
If the number on the right is large and the number on the left is smaller, but not zero, then it may take a while for all the collector pods to start. Contact us if pods have not started after an hour. Only increase the target pods if the documentation for the specific collector confirms that multiple pod operation is supported. Most collectors will only function with one pod.
No data in devo.collectors.out
If all collectors with a particular sender keychain are unable to write data to devo.collectors.out, the sender keychain is probably incorrect. The logs of a running a collector can be located in the Cloud Collector App by selecting the collector name, “Collector pods,” and the eye button. Logs stating Error closing connection can be caused by an incorrect sender keychain.
If the sender keychain has stopped working, probably the X.509 certificate has expired. If a new sender keychain is not working, the most common cause is that a wrong file was uploaded.
In “Keychains,” use the pencil button to fix the sender keychain configuration.
Restart all collectors.
No data is being collected
Causes
Incorrect collector credentials.
Replace the credentials in the collector parameters.
Follow the Microsoft credentials instructions to ensure a Secret ID is not used.
Credentials are correct, but permissions are wrong in the data source.
Change the permissions.
Some data sources also require replacement of the credentials in the collector parameters.
Restart the collector to check the credentials quickly.
Credentials and permissions are correct, but no data is available at the data source.
Add some data. Some data sources only have data when malicious activity occurs.
The service is not enabled in the collector parameters.
Check the list of services in the collector parameters against the list of services in the documentation for that collector.
The destination tag has been overridden in the collector parameters.
Query the table set in the collector parameters instead of querying the table named in the documentation.
The data format in the data source is not compatible with the collector’s automatic tagging function.
Configure the data source to have the expected format.
The data may be in the table
unknown.unknown.
The data source is behind a firewall.
The firewall should allow connections to the data source.
The data was collected but the table is not listed in the finder.
Click the refresh button in the finder to make the table visible.
Gather more information
Find the collector ID in the cloud collector app. This is not the ID in the parameters.
Restart the collector. Shortly after the collector restarts, it will attempt to contact the data source.
Run this query.
from devo.collectors.out where weakhas(msg,"error") or weakhas(msg,"exception")Select a time range that covers the five minutes after the collector was restarted.
Filter the
hostnamefield to select the correct collector ID.Usually, the
msgfield will describe the problem. Frequently the error message will include an explanatory HTTP status code. For example,503indicates the data source is not available.
The ID is not unique
Change the ID to a unique five digit number in the collector parameters. In some cases, changing the ID will reset the collector, leading to collection of historical data. If two collectors were using the same ID, a reset of both collectors is necessary.
Data is delayed
When a collector is first created, usually some delayed data will be collected before new data is collected. This may also happen if the collector ID has been changed in the collector parameters.
Some data sources, such as Microsoft Graph, only provide data with a delay. Contact the vendor for a workaround. For example, Entra ID data send with an event hub will arrive much faster than data sent with Graph.
The timestamp of a log may indicate the time a resource was created in the data source. If the log is made available in the data source some time after the resource is created, then the eventdate may be much later than the timestamp.
If the data source was unavailable due to maintenance, data may be delayed for a short period after the maintenance. Query devo.collectors.out for explanatory errors.
Contact us if the time delay is equal to the retention period of the data source.
Data is not parsed or all fields are null
Change the data format in the data source so that it is in a format compatible with Devo’s parser. Contact us for more assistance.
Collector parameters include < or >
The JSON object in the collector parameters has not been completely filled out. Check the documentation for the specific collector to see what the parameters should be.
Getting help
Contact us with this information:
What is the problem?
When did the problem happen? What is the time zone?
What is the ID of the collector in the cloud collector app? This is not the ID in the parameters.
What is the domain?
What is the Devo URL?
In this example, the ID is collector-f40fae61d09be943. The domain is demo. The URL is http://us.devo.com.