- Created by Juan Tomás Alonso Nieto , last modified by MD Tausif on Dec 04, 2024
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 13 Current »
Overview
Cortex XDR delivers enterprise-wide protection by analyzing data from any source to stop sophisticated attacks.
Eliminate blind spots with complete visibility
Simplify security operations to cut mean time to respond (MTTR)
Harness the scale of the cloud for AI and analytics
Lower costs by consolidating tools and improving SOC efficiency.
Cortex XDR collector migration guide (from 1.x.x to 2.x.x)
If you need to migrate an old collector version to a more recent one, please check the migration process in this article.
Devo collector features
Feature | Details |
---|---|
Allow parallel downloading ( |
|
Running environments |
|
Populated Devo events |
|
Flattening preprocessing |
|
Requires IP Whitelisting |
|
Data sources
Data source | Description | API endpoint | Collector service name | Devo table | Available from release |
---|---|---|---|---|---|
Alerts | Get a list of alerts with multiple events. |
|
|
You can override this in |
|
Incidents | Get Incidents: Get Incident Alerts: | List Incidents: List Incident Alerts:
|
| List Incidents:
You can override this in List Incident Alerts:
You can override this in |
|
All Alerts | Get a list of alerts. |
|
|
You can override this in |
|
Audit Managements | Get audit management logs. |
|
|
You can override this in |
|
Vendor configuration
Generate API key
Log in to your Cortex XDR console with an administrative account.
Go to the Settings menu (gear icon).
Go to API Keys under the Integrations section.
Click Add Key to generate a new API key.
Select the roles or permissions you want to assign to this key (these determine the actions the key can perform).
Click Create for generating the API Key along with an API Key ID. Save these securely, as you may not be able to retrieve them later.
API FQDN
The API FQDN (Fully Qualified Domain Name) can be found in the documentation or directly in the URL when you are logged into your Cortex XDR console.
It will usually be in the format:
https://<YOUR_REGION>.xdr.paloaltonetworks.com
Minimum configuration required for basic pulling
Although this collector supports advanced configuration, the fields required to download data with basic configuration are defined below.
This minimum configuration refers exclusively to those specific parameters of this integration. There are more required parameters related to the generic behavior of the collector. Check setting sections for details.
Setting | Details |
---|---|
| The API Key is your unique identifier for Cortex XDR. |
| The API Key ID is your unique token used to authenticate the API key for Cortex XDR. |
| API FQDN is a unique host and domain name associated with each tenant for Cortex XDR. |
See the Accepted authentication methods section to verify what settings are required based on the desired authentication method.
Accepted authentication methods
Depending on how did you obtain your credentials, you will have to either fill or delete the following properties on the JSON credentials
configuration block.
Authentication method | API key | API key ID | API FQDN |
---|---|---|---|
API KEY / API KEY ID / API FQDN | REQUIRED | REQUIRED | REQUIRED |
Run the collector
Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).
Collector services detail
This section is intended to explain how to proceed with specific actions for services.
Verify data collection
Service components
Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.
This service has the following components:
Component | Description |
---|---|
Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |
Setup is common for every service, so its output is displayed in this section.
On the other hand, Pullers are specific to each Service, so their outputs are shown in their corresponding sections below.
Setup output
A successful run has the following output messages for the setup module:
2024-09-06T15:07:25.139 INFO MainProcess::MainThread -> Started all objects from "MainProcess" process 2024-09-06T15:07:25.144 INFO InputProcess::MainThread -> Process started (pid=19803, ppid=19784, multiprocessing.start_method="fork") 2024-09-06T15:07:25.220 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSenderManager;standard_senders;devo_1.json.gz 2024-09-06T15:07:25.223 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.224 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/5bdbabbd30e1988d6e2b0b74b6ec3289" 2024-09-06T15:07:25.227 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/cortex_xdr;1243598;incidents;predefined;CortexXdrBasePuller.json 2024-09-06T15:07:25.231 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.231 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/6a3859facec8d617720d39590e169164" 2024-09-06T15:07:25.232 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/c620dfb5bdc3fc1a386e4924e99aa3e7" 2024-09-06T15:07:25.232 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSender;standard_senders;devo_sender_0.json.gz 2024-09-06T15:07:25.233 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-09-06T15:07:25.233 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/6a3859facec8d617720d39590e169164" 2024-09-06T15:07:25.233 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/c620dfb5bdc3fc1a386e4924e99aa3e7" 2024-09-06T15:07:25.235 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.235 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/b1ee30ea2c4b3be1eb50c8b2ca80d8d8" 2024-09-06T15:07:25.235 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) Starting the execution of init_variables() 2024-09-06T15:07:25.236 INFO InputProcess::MainThread -> Validating service metadata 2024-09-06T15:07:25.238 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/OutputStandardConsumer;standard_senders;0.json.gz 2024-09-06T15:07:25.241 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.241 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/de28663278a264d05d531fbc1db51a93" 2024-09-06T15:07:25.242 INFO InputProcess::MainThread -> Validating defined module definition 2024-09-06T15:07:25.243 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSenderManager;lookup_senders;devo_1.json.gz 2024-09-06T15:07:25.246 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.246 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/a28c135f3fc69ece87d8fc2fdb029e8c" 2024-09-06T15:07:25.251 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSender;lookup_senders;devo_sender_0.json.gz 2024-09-06T15:07:25.252 INFO InputProcess::MainThread -> Validating common input config 2024-09-06T15:07:25.253 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.253 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/df8895fef2a509cbd87fcc9850dc0c81" 2024-09-06T15:07:25.255 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/OutputLookupConsumer;lookup_senders;0.json.gz 2024-09-06T15:07:25.256 INFO InputProcess::MainThread -> Validating service input config 2024-09-06T15:07:25.257 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.257 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/865a79c1b99ad39b22becc235c9732cb" 2024-09-06T15:07:25.259 INFO InputProcess::MainThread -> Running overriding rules 2024-09-06T15:07:25.259 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSenderManager;internal_senders;devo_1.json.gz 2024-09-06T15:07:25.260 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-09-06T15:07:25.260 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-09-06T15:07:25.261 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-09-06T15:07:25.261 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.261 INFO InputProcess::MainThread -> Running custom validation rules 2024-09-06T15:07:25.261 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/e784c94a6fe28de5937e50674407d82e" 2024-09-06T15:07:25.265 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) Finalizing the execution of init_variables() 2024-09-06T15:07:25.265 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSender;internal_senders;devo_sender_0.json.gz 2024-09-06T15:07:25.268 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.268 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/4ff7b345dc444ac050cf75f93e5dcb3b" 2024-09-06T15:07:25.269 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/cortex_xdr;1243598;alerts;predefined;CortexXdrBasePuller.json 2024-09-06T15:07:25.271 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/OutputInternalConsumer;internal_senders;0.json.gz 2024-09-06T15:07:25.272 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.272 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/50e7b9f5b3320841b223e83e5b156b5f" 2024-09-06T15:07:25.272 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.272 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/5a07c27e666ce7c4400ef58d5b89f174" 2024-09-06T15:07:25.273 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/10dd360c86621afd5a28a029a0dddcf6" 2024-09-06T15:07:25.273 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> Starting thread 2024-09-06T15:07:25.274 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-09-06T15:07:25.274 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/50e7b9f5b3320841b223e83e5b156b5f" 2024-09-06T15:07:25.274 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/5a07c27e666ce7c4400ef58d5b89f174" 2024-09-06T15:07:25.274 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(standard_senders,devo_1) -> Starting thread (every 300 seconds) 2024-09-06T15:07:25.275 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> Starting thread 2024-09-06T15:07:25.275 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) Starting the execution of init_variables() 2024-09-06T15:07:25.275 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.276 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.276 INFO InputProcess::MainThread -> Validating service metadata 2024-09-06T15:07:25.276 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> Starting thread 2024-09-06T15:07:25.277 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(lookup_senders,devo_1) -> Starting thread (every 300 seconds) 2024-09-06T15:07:25.279 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.279 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-09-06T15:07:25.279 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> Starting thread 2024-09-06T15:07:25.279 INFO InputProcess::MainThread -> Validating defined module definition 2024-09-06T15:07:25.280 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.280 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.280 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> Starting thread 2024-09-06T15:07:25.282 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(internal_senders,devo_1) -> Starting thread (every 300 seconds) 2024-09-06T15:07:25.283 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.283 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-09-06T15:07:25.283 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.283 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-09-06T15:07:25.285 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> Starting thread 2024-09-06T15:07:25.285 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.285 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-09-06T15:07:25.285 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.286 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.287 INFO InputProcess::MainThread -> Validating common input config 2024-09-06T15:07:25.289 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.289 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-09-06T15:07:25.294 INFO InputProcess::MainThread -> Validating service input config 2024-09-06T15:07:25.298 INFO InputProcess::MainThread -> Running overriding rules 2024-09-06T15:07:25.299 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-09-06T15:07:25.300 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-09-06T15:07:25.301 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-09-06T15:07:25.302 INFO InputProcess::MainThread -> Running custom validation rules 2024-09-06T15:07:25.302 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) Finalizing the execution of init_variables() 2024-09-06T15:07:25.307 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/cortex_xdr;1243598;all_alerts;predefined;CortexXdrBasePuller.json 2024-09-06T15:07:25.310 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.311 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/30e04fb750908b6e56aa65c0390708bf" 2024-09-06T15:07:25.311 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/9e181361d214f7eaac0dcc0853396aef" 2024-09-06T15:07:25.312 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-09-06T15:07:25.313 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/30e04fb750908b6e56aa65c0390708bf" 2024-09-06T15:07:25.313 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/9e181361d214f7eaac0dcc0853396aef" 2024-09-06T15:07:25.314 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) Starting the execution of init_variables() 2024-09-06T15:07:25.315 INFO InputProcess::MainThread -> Validating service metadata 2024-09-06T15:07:25.321 INFO InputProcess::MainThread -> Validating defined module definition 2024-09-06T15:07:25.339 INFO InputProcess::MainThread -> Validating common input config 2024-09-06T15:07:25.339 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.340 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.05 2024-09-06T15:07:25.341 INFO OutputProcess::MainThread -> [GC] global: 42.0% -> 42.0%, process: RSS(41.21MiB -> 41.21MiB), VMS(929.01MiB -> 929.01MiB) 2024-09-06T15:07:25.349 INFO InputProcess::MainThread -> Validating service input config 2024-09-06T15:07:25.353 INFO InputProcess::MainThread -> Running overriding rules 2024-09-06T15:07:25.354 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-09-06T15:07:25.356 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-09-06T15:07:25.357 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-09-06T15:07:25.358 INFO InputProcess::MainThread -> Running custom validation rules 2024-09-06T15:07:25.359 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) Finalizing the execution of init_variables() 2024-09-06T15:07:25.367 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/cortex_xdr;1243598;audit_managements;predefined;CortexXdrBasePuller.json 2024-09-06T15:07:25.373 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.373 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/ae374cefc744c8268f07d37056e660e2" 2024-09-06T15:07:25.373 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/93607031cbbdac722344eb2b8746cfdc" 2024-09-06T15:07:25.375 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-09-06T15:07:25.376 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/ae374cefc744c8268f07d37056e660e2" 2024-09-06T15:07:25.376 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/93607031cbbdac722344eb2b8746cfdc" 2024-09-06T15:07:25.378 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) Starting the execution of init_variables() 2024-09-06T15:07:25.379 INFO InputProcess::MainThread -> Validating service metadata 2024-09-06T15:07:25.386 INFO InputProcess::MainThread -> Validating defined module definition 2024-09-06T15:07:25.399 INFO InputProcess::MainThread -> Validating common input config 2024-09-06T15:07:25.405 INFO InputProcess::MainThread -> Validating service input config 2024-09-06T15:07:25.408 INFO InputProcess::MainThread -> Running overriding rules 2024-09-06T15:07:25.409 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-09-06T15:07:25.410 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-09-06T15:07:25.411 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-09-06T15:07:25.413 INFO InputProcess::MainThread -> Running custom validation rules 2024-09-06T15:07:25.415 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) Finalizing the execution of init_variables() 2024-09-06T15:07:25.419 INFO InputProcess::MainThread -> InputThread(cortex_xdr,1243598) - Starting thread (execution_period=60s) 2024-09-06T15:07:25.420 INFO InputProcess::MainThread -> ServiceThread(cortex_xdr,1243598,incidents,predefined) - Starting thread (execution_period=60s) 2024-09-06T15:07:25.421 INFO InputProcess::MainThread -> CortexXdrPullerSetup(cortex_xdr,cortex_xdr#1243598,incidents#predefined) -> Starting thread 2024-09-06T15:07:25.422 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) - Starting thread 2024-09-06T15:07:25.430 WARNING InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> Waiting until setup will be executed 2024-09-06T15:07:25.431 INFO InputProcess::MainThread -> ServiceThread(cortex_xdr,1243598,alerts,predefined) - Starting thread (execution_period=60s) 2024-09-06T15:07:25.432 INFO InputProcess::MainThread -> CortexXdrPullerSetup(cortex_xdr,cortex_xdr#1243598,alerts#predefined) -> Starting thread 2024-09-06T15:07:25.433 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) - Starting thread 2024-09-06T15:07:25.435 WARNING InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> Waiting until setup will be executed 2024-09-06T15:07:25.438 INFO InputProcess::MainThread -> ServiceThread(cortex_xdr,1243598,all_alerts,predefined) - Starting thread (execution_period=60s) 2024-09-06T15:07:25.440 INFO InputProcess::MainThread -> CortexXdrPullerSetup(cortex_xdr,cortex_xdr#1243598,all_alerts#predefined) -> Starting thread 2024-09-06T15:07:25.440 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) - Starting thread 2024-09-06T15:07:25.444 WARNING InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> Waiting until setup will be executed 2024-09-06T15:07:25.445 INFO InputProcess::MainThread -> ServiceThread(cortex_xdr,1243598,audit_managements,predefined) - Starting thread (execution_period=60s) 2024-09-06T15:07:25.446 INFO InputProcess::MainThread -> CortexXdrPullerSetup(cortex_xdr,cortex_xdr#1243598,audit_managements#predefined) -> Starting thread 2024-09-06T15:07:25.448 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) - Starting thread 2024-09-06T15:07:25.451 WARNING InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> Waiting until setup will be executed 2024-09-06T15:07:25.494 INFO InputProcess::MainThread -> [GC] global: 42.1% -> 42.1%, process: RSS(41.61MiB -> 41.73MiB), VMS(1.05GiB -> 1.05GiB) 2024-09-06T15:07:25.780 INFO OutputProcess::DevoSender(internal_senders,devo_sender_0) -> Created a sender: {"name": "DevoSender(internal_senders,devo_sender_0)", "url": "collector-eu.devo.io:443", "chain_path": "/home/md_tausif/gitlab/devo-collector-cortex-xdr/certs/chain.crt", "cert_path": "/home/md_tausif/gitlab/devo-collector-cortex-xdr/certs/int-if-integrations-india.crt", "key_path": "/home/md_tausif/gitlab/devo-collector-cortex-xdr/certs/int-if-integrations-india.key", "transport_layer_type": "SSL", "last_usage_timestamp": null, "socket_status": null}, hostname: "2023-apac-0046", session_id: "124794842200288" 2024-09-06T15:07:25.781 INFO OutputProcess::DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:28.870 INFO InputProcess::CortexXdrPullerSetup(cortex_xdr,cortex_xdr#1243598,incidents#predefined) -> Setup for module <CortexXdrBasePuller> has been successfully executed
Restart the persistence
Some services in this collector use persistent storage to download events in an orderly fashion and avoid duplicates. In case you want to re-ingest historical data or recreate the persistence, you can restart the persistence of this collector by following these steps:
Edit the configuration file.
Change the value of the
start_date
parameter to a different one.Save the changes.
Restart the collector.
The collector will detect this change and will restart the persistence using the parameters of the configuration file or the default configuration in case it has not been provided.
Note that this action clears the persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.
This Service no longer uses
edr.cortex_xdr.alerts_multi_event
table to ingest the events Data. Now, All the Data will be ingested inedr.cortex_xdr.alerts_multi
.We are using
v2/alerts/get_alerts_multi_events
endpoint now but earlier we were usingv1/alerts/get_alerts_multi_events
endpoint to fetch the data.
Verify data collection
Puller output
A successful initial run has the following output messages for the puller module:
Note that the PrePull
action is executed only one time before the first run of the Pull
action.
024-09-06T15:30:53.884 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> Pull Started 2024-09-06T15:30:53.886 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> start time_stamp to fetch the data is: 1725616793881 2024-09-06T15:30:53.887 INFO OutputProcess::DevoSender(internal_senders,devo_sender_0) -> Consumed messages: 11 messages (60.001491 seconds) => 0 msg/sec 2024-09-06T15:30:59.379 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725616853882):Number of requests made: 1; Number of events received: 0; Number of duplicated events filtered out: 0; Number of events generated and sent: 0; Average of events per second: 0.000. 2024-09-06T15:30:59.380 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> Sent 0 events to Devo. 2024-09-06T15:30:59.382 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725616853882):Number of requests made: 1; Number of events received: 0; Number of duplicated events filtered out: 0; Number of events generated and sent: 0; Average of events per second: 0.000. 2024-09-06T15:30:59.383 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725616853882):Number of requests made: 1; Number of events received: 0; Number of duplicated events filtered out: 0; Number of events generated and sent: 0; Average of events per second: 0.000. 2024-09-06T15:30:59.383 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> The data is up to date!
After a successful collector’s execution (this is, no error logs were found), you should be able to see the following log message:
2024-09-06T15:30:59.382 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725616853882):Number of requests made: 1; Number of events received: 0; Number of duplicated events filtered out: 0; Number of events generated and sent: 0; Average of events per second: 0.000.
The @devo_pulling_id
value is injected into each event to allow grouping all events ingested by the same pull action. You can use it to get the exact events downloaded on that Pull
action in the Data Search area of Devo.
Restart the persistence
This service makes use of persistence. You can check how to restart it above.
Table edr.cortex_xdr.alerts
has been deprecated. Now, the data of incident Alerts (v1/incident/get_incident_alert_data)
is ingested in the table edr.cortex_xdr.incident_alert
Verify data collection
Puller output
A successful initial run has the following output messages for the puller module:
Note that the PrePull
action is executed only one time before the first run of the Pull
action.
2024-09-06T17:13:57.587 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> Pull Started 2024-09-06T17:13:57.588 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> start time_stamp to fetch the data is: 1724098800000 2024-09-06T17:13:59.093 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725623037578):Number of requests made: 1; Number of events received: 15; Number of duplicated events filtered out: 0; Number of events generated and sent: 15; Average of events per second: 9.958. 2024-09-06T17:13:59.094 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> Sent 15 events to Devo. 2024-09-06T17:13:59.095 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725623037578):Number of requests made: 1; Number of events received: 15; Number of duplicated events filtered out: 0; Number of events generated and sent: 15; Average of events per second: 9.950. 2024-09-06T17:13:59.095 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725623037578):Number of requests made: 1; Number of events received: 15; Number of duplicated events filtered out: 0; Number of events generated and sent: 15; Average of events per second: 9.948. 2024-09-06T17:13:59.095 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> The data is up to date! 2024-09-06T17:13:59.096 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> Data collection completed. Elapsed time: 1.518 seconds. Waiting for 58.482 second(s) until the next one
After a successful collector’s execution (this is, no error logs were found), you should be able to see the following log message:
2024-09-06T17:13:59.095 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725623037578):Number of requests made: 1; Number of events received: 15; Number of duplicated events filtered out: 0; Number of events generated and sent: 15; Average of events per second: 9.948.
The @devo_pulling_id
value is injected into each event to allow grouping all events ingested by the same pull action. You can use it to get the exact events downloaded on that Pull
action in Loxcope.
Restart the persistence
This service makes use of persistence. You can check how to restart it above.
Devo categorization and destination
All events of this service are ingested into table edr.cortex_xdr.audit_management
Verify data collection
Puller output
A successful initial run has the following output messages for the puller module:
Note that the PrePull
action is executed only one time before the first run of the Pull
action.
2024-09-06T17:17:05.936 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> Pull Started 2024-09-06T17:17:05.939 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> start time_stamp to fetch the data is: 1725623165933 2024-09-06T17:16:09.137 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725623165933):Number of requests made: 1; Number of events received: 2; Number of duplicated events filtered out: 0; Number of events generated and sent: 2; Average of events per second: 0.627. 2024-09-06T17:16:09.139 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> Sent 2 events to Devo. 2024-09-06T17:16:09.143 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725623165933):Number of requests made: 1; Number of events received: 2; Number of duplicated events filtered out: 0; Number of events generated and sent: 2; Average of events per second: 0.626. 2024-09-06T17:16:09.145 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725623165933):Number of requests made: 1; Number of events received: 2; Number of duplicated events filtered out: 0; Number of events generated and sent: 2; Average of events per second: 0.626. 2024-09-06T17:17:09.094 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> The data is up to date! 2024-09-06T17:17:09.094 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> Data collection completed. Elapsed time: 3.160 seconds. Waiting for 56.840 second(s) until the next one
After a successful collector’s execution (this is, no error logs were found), you should be able to see the following log message:
2024-09-06T17:16:09.145 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,audit_managements#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725623165933):Number of requests made: 1; Number of events received: 2; Number of duplicated events filtered out: 0; Number of events generated and sent: 2; Average of events per second: 0.626.
The @devo_pulling_id
value is injected into each event to allow grouping all events ingested by the same pull action. You can use it to get the exact events downloaded on that Pull
action in Loxcope.
Restart the persistence
This service makes use of persistence. You can check how to restart it above.
Devo categorization and destination
All events of this service are ingested into table edr.cortex_xdr.all_alert
Verify data collection
Puller output
A successful initial run has the following output messages for the puller module:
Note that the PrePull
action is executed only one time before the first run of the Pull
action.
2024-09-06T17:24:44.923 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> Pull Started 2024-09-06T17:24:44.925 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> start time_stamp to fetch the data is: 1725308400000 2024-09-06T17:24:52.766 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725623684899):Number of requests made: 1; Number of events received: 77; Number of duplicated events filtered out: 0; Number of events generated and sent: 77; Average of events per second: 9.819. 2024-09-06T17:24:52.767 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> Sent 77 events to Devo. 2024-09-06T17:24:52.768 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1725623684899):Number of requests made: 1; Number of events received: 77; Number of duplicated events filtered out: 0; Number of events generated and sent: 77; Average of events per second: 9.817. 2024-09-06T17:24:52.768 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725623684899):Number of requests made: 1; Number of events received: 77; Number of duplicated events filtered out: 0; Number of events generated and sent: 77; Average of events per second: 9.816. 2024-09-06T17:24:52.769 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> The data is up to date! 2024-09-06T17:24:52.770 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> Data collection completed. Elapsed time: 7.871 seconds. Waiting for 52.129 second(s) until the next one
After a successful collector’s execution (this is, no error logs were found), you should be able to see the following log message:
2024-09-06T17:24:52.768 INFO InputProcess::CortexXdrBasePuller(cortex_xdr#1243598,all_alerts#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1725623684899):Number of requests made: 1; Number of events received: 77; Number of duplicated events filtered out: 0; Number of events generated and sent: 77; Average of events per second: 9.816.
The @devo_pulling_id
value is injected into each event to allow grouping all events ingested by the same pull action. You can use it to get the exact events downloaded on that Pull
action in Loxcope.
Restart the persistence
This service makes use of persistence. You can check how to restart it above.
Collector operations
This section is intended to explain how to proceed with specific operations of this collector.
The initialization module is in charge of setup and running the input (pulling logic) and output (delivering logic) services and validating the given configuration.
The initialization module is in charge of setup and running the input (pulling logic) and output (delivering logic) services and validating the given configuration.
A successful run has the following output messages for the initializer module, including all services together:
2024-09-06T15:07:25.139 INFO MainProcess::MainThread -> Started all objects from "MainProcess" process 2024-09-06T15:07:25.144 INFO InputProcess::MainThread -> Process started (pid=19803, ppid=19784, multiprocessing.start_method="fork") 2024-09-06T15:07:25.220 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSenderManager;standard_senders;devo_1.json.gz 2024-09-06T15:07:25.223 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.224 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/5bdbabbd30e1988d6e2b0b74b6ec3289" 2024-09-06T15:07:25.227 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/cortex_xdr;1243598;incidents;predefined;CortexXdrBasePuller.json 2024-09-06T15:07:25.231 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.231 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/6a3859facec8d617720d39590e169164" 2024-09-06T15:07:25.232 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/c620dfb5bdc3fc1a386e4924e99aa3e7" 2024-09-06T15:07:25.232 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSender;standard_senders;devo_sender_0.json.gz 2024-09-06T15:07:25.233 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-09-06T15:07:25.233 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/6a3859facec8d617720d39590e169164" 2024-09-06T15:07:25.233 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/c620dfb5bdc3fc1a386e4924e99aa3e7" 2024-09-06T15:07:25.235 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.235 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/b1ee30ea2c4b3be1eb50c8b2ca80d8d8" 2024-09-06T15:07:25.235 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) Starting the execution of init_variables() 2024-09-06T15:07:25.236 INFO InputProcess::MainThread -> Validating service metadata 2024-09-06T15:07:25.238 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/OutputStandardConsumer;standard_senders;0.json.gz 2024-09-06T15:07:25.241 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.241 INFO OutputProcess::MainThread -> OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/de28663278a264d05d531fbc1db51a93" 2024-09-06T15:07:25.242 INFO InputProcess::MainThread -> Validating defined module definition 2024-09-06T15:07:25.243 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSenderManager;lookup_senders;devo_1.json.gz 2024-09-06T15:07:25.246 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.246 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/a28c135f3fc69ece87d8fc2fdb029e8c" 2024-09-06T15:07:25.251 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSender;lookup_senders;devo_sender_0.json.gz 2024-09-06T15:07:25.252 INFO InputProcess::MainThread -> Validating common input config 2024-09-06T15:07:25.253 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.253 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/df8895fef2a509cbd87fcc9850dc0c81" 2024-09-06T15:07:25.255 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/OutputLookupConsumer;lookup_senders;0.json.gz 2024-09-06T15:07:25.256 INFO InputProcess::MainThread -> Validating service input config 2024-09-06T15:07:25.257 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.257 INFO OutputProcess::MainThread -> OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/865a79c1b99ad39b22becc235c9732cb" 2024-09-06T15:07:25.259 INFO InputProcess::MainThread -> Running overriding rules 2024-09-06T15:07:25.259 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSenderManager;internal_senders;devo_1.json.gz 2024-09-06T15:07:25.260 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-09-06T15:07:25.260 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-09-06T15:07:25.261 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-09-06T15:07:25.261 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.261 INFO InputProcess::MainThread -> Running custom validation rules 2024-09-06T15:07:25.261 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/e784c94a6fe28de5937e50674407d82e" 2024-09-06T15:07:25.265 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,incidents#predefined) Finalizing the execution of init_variables() 2024-09-06T15:07:25.265 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/DevoSender;internal_senders;devo_sender_0.json.gz 2024-09-06T15:07:25.268 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.268 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/4ff7b345dc444ac050cf75f93e5dcb3b" 2024-09-06T15:07:25.269 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/cortex_xdr;1243598;alerts;predefined;CortexXdrBasePuller.json 2024-09-06T15:07:25.271 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Created persistence instance, filename_path: /home/md_tausif/gitlab/devo-collector-cortex-xdr/state/not_used/OutputInternalConsumer;internal_senders;0.json.gz 2024-09-06T15:07:25.272 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.272 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/50e7b9f5b3320841b223e83e5b156b5f" 2024-09-06T15:07:25.272 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] There is no data persisted with the latest format, any previous persisted data will be migrated 2024-09-06T15:07:25.272 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/5a07c27e666ce7c4400ef58d5b89f174" 2024-09-06T15:07:25.273 INFO OutputProcess::MainThread -> OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] No previous persistence file exists to migrate (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/10dd360c86621afd5a28a029a0dddcf6" 2024-09-06T15:07:25.273 INFO OutputProcess::MainThread -> DevoSender(standard_senders,devo_sender_0) -> Starting thread 2024-09-06T15:07:25.274 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] Checking if old persisted data must be removed 2024-09-06T15:07:25.274 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 2), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/50e7b9f5b3320841b223e83e5b156b5f" 2024-09-06T15:07:25.274 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) -> [PERSISTENCE_SYSTEM] No previous persistence file exists (Version 1), filename_path: "/home/md_tausif/gitlab/devo-collector-cortex-xdr/state/5a07c27e666ce7c4400ef58d5b89f174" 2024-09-06T15:07:25.274 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(standard_senders,devo_1) -> Starting thread (every 300 seconds) 2024-09-06T15:07:25.275 INFO OutputProcess::MainThread -> DevoSenderManager(standard_senders,manager,devo_1) -> Starting thread 2024-09-06T15:07:25.275 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) Starting the execution of init_variables() 2024-09-06T15:07:25.275 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.276 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.276 INFO InputProcess::MainThread -> Validating service metadata 2024-09-06T15:07:25.276 INFO OutputProcess::MainThread -> DevoSender(lookup_senders,devo_sender_0) -> Starting thread 2024-09-06T15:07:25.277 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(lookup_senders,devo_1) -> Starting thread (every 300 seconds) 2024-09-06T15:07:25.279 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.279 INFO OutputProcess::DevoSenderManager(standard_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-09-06T15:07:25.279 INFO OutputProcess::MainThread -> DevoSenderManager(lookup_senders,manager,devo_1) -> Starting thread 2024-09-06T15:07:25.279 INFO InputProcess::MainThread -> Validating defined module definition 2024-09-06T15:07:25.280 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.280 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.280 INFO OutputProcess::MainThread -> DevoSender(internal_senders,devo_sender_0) -> Starting thread 2024-09-06T15:07:25.282 INFO OutputProcess::MainThread -> DevoSenderManagerMonitor(internal_senders,devo_1) -> Starting thread (every 300 seconds) 2024-09-06T15:07:25.283 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.283 INFO OutputProcess::OutputStandardConsumer(standard_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-09-06T15:07:25.283 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.283 INFO OutputProcess::OutputLookupConsumer(lookup_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-09-06T15:07:25.285 INFO OutputProcess::MainThread -> DevoSenderManager(internal_senders,manager,devo_1) -> Starting thread 2024-09-06T15:07:25.285 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.285 INFO OutputProcess::DevoSenderManager(lookup_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.01 2024-09-06T15:07:25.285 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.286 INFO OutputProcess::OutputInternalConsumer(internal_senders_consumer_0) -> [EMERGENCY_PERSISTENCE_SYSTEM] Recovering any available content from the persistence system 2024-09-06T15:07:25.287 INFO InputProcess::MainThread -> Validating common input config 2024-09-06T15:07:25.289 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Nothing available in the persistence system 2024-09-06T15:07:25.289 INFO OutputProcess::DevoSenderManager(internal_senders,manager,devo_1) -> [EMERGENCY_PERSISTENCE_SYSTEM] Elapsed seconds: 0.00 2024-09-06T15:07:25.294 INFO InputProcess::MainThread -> Validating service input config 2024-09-06T15:07:25.298 INFO InputProcess::MainThread -> Running overriding rules 2024-09-06T15:07:25.299 INFO InputProcess::MainThread -> Validating the rate limiter config given by the user 2024-09-06T15:07:25.300 INFO InputProcess::MainThread -> <requests_limits> setting has not been defined. The generic settings will be used instead. 2024-09-06T15:07:25.301 INFO InputProcess::MainThread -> Adding raw config to the collector store 2024-09-06T15:07:25.302 INFO InputProcess::MainThread -> Running custom validation rules 2024-09-06T15:07:25.302 INFO InputProcess::MainThread -> CortexXdrBasePuller(cortex_xdr#1243598,alerts#predefined) Finalizing the execution of init_variables()
The event delivery module is in charge of receiving the events from the internal queues where all events are injected by the pullers and delivering them using the selected compatible delivery method.
A successful run has the following output messages for the initializer module:
2024-09-06T15:32:47.810 INFO OutputProcess::DevoSenderManagerMonitor(lookup_senders,devo_1) -> Number of available senders: 1, sender manager internal queue size: 0 2024-09-06T15:32:47.810 INFO OutputProcess::DevoSenderManagerMonitor(lookup_senders,devo_1) -> enqueued_elapsed_times_in_seconds_stats: {} 2024-09-06T15:32:47.810 INFO OutputProcess::DevoSenderManagerMonitor(lookup_senders,devo_1) -> Sender: DevoSender(lookup_senders,devo_sender_0), status: {"internal_queue_size": 0, "is_connection_open": False} 2024-09-06T15:32:47.811 INFO OutputProcess::DevoSenderManagerMonitor(lookup_senders,devo_1) -> Lookup - Total number of messages sent: 0, messages sent since "2024-09-06 09:57:47.810025+00:00": 0 (elapsed 0.000 seconds)
By default, these information traces will be displayed every 10 minutes.
Sender statistics
Each service displays its own performance statistics that allow checking how many events have been delivered to Devo by type:
Logging trace | Description |
---|---|
| Displays the number of concurrent senders available for the given Sender Service. |
| Displays the items available in the internal sender queue. This value helps detect bottlenecks and needs to increase the performance of data delivery to Devo. This last can be made by increasing the concurrent senders. |
| Displayes the number of events from the last time and following the given example, the following conclusions can be obtained:
By default these traces will be shown every 10 minutes. |
To check the memory usage of this collector, look for the following log records in the collector which are displayed every 5 minutes by default, always after running the memory-free process.
Some collectors can be tuned to reduce the memory impact when pulling data in large infrastructures.
The used memory is displayed by running processes and the sum of both values will give the total used memory for the collector.
The global pressure of the available memory is displayed in the
global
value.Al metrics (Global, RSS, VMS) include the value before freeing and after:
previous -> after freeing memory
2024-09-06T15:07:25.494 INFO InputProcess::MainThread -> [GC] global: 42.1% -> 42.1%, process: RSS(41.61MiB -> 41.73MiB), VMS(1.05GiB -> 1.05GiB) 2024-09-06T15:09:25.375 INFO OutputProcess::MainThread -> [GC] global: 42.4% -> 42.4%, process: RSS(44.46MiB -> 44.46MiB), VMS(929.14MiB -> 929.14MiB)
Differences between RSS
and VMS
memory usage:
RSS
is the Resident Set Size, which is the actual physical memory the process is usingVMS
is the Virtual Memory Size which is the virtual memory that process is using
Sometimes it is necessary to activate the debug mode of the collector's logging. This debug mode increases the verbosity of the log and allows you to print execution traces that are very helpful in resolving incidents or detecting bottlenecks in heavy download processes.
To enable this option you just need to edit the configuration file and change the debug_status parameter from false to true and restart the collector.
To disable this option, you just need to update the configuration file and change the debug_status parameter from true to false and restart the collector.
For more information, visit the configuration and parameterization section corresponding to the chosen deployment mode.
This collector has different security layers that detect both an invalid configuration and abnormal operation. This table will help you detect and resolve the common errors for the current services.
Error type | Error ID | Error message | Cause | Solution |
---|---|---|---|---|
| 100 |
| You will get the status code with error msg. | Ensure that the collector has the necessary permissions and proper credentials to access the Cortex xdr API and contact the developer with exact error message |
| 300 |
| You will get the status code with error msg. | This is an internal issue. Contact with Devo Support team. |
| 400 |
| Bad Request | Kindly reach out to the developer |
401 |
| Credentials provided is not correct | Please check and provide the correct credentials | |
402 |
| User doesn't have the required permission | Kindly get required permission and reach the developer with the exact msg. | |
403 |
| Internal server error. An unverified status of API communication type error | Kindly reach the developer with the exact msg. | |
404 |
| You will get the status code with error msg. | Kindly reach the developer with the exact msg. | |
405 |
| Number of request has exceeded for the cortex api | Kindly reach the developer with the exact msg. | |
406 |
| Number of request has exceeded for the cortex api | Kindly reach the developer with the exact msg. | |
407 |
| Number of request has exceeded for the cortex api | Kindly reach the developer with the exact msg. | |
408 |
| You will get the status code with error msg | Kindly reach the developer with the exact msg. | |
409 |
| You will get the status code with error msg | Kindly reach the developer with the exact msg. |
Change log
Release | Released on | Release type | Recommendations |
---|---|---|---|
| IMPROVEMENT |
| |
Details Improvements
| |||
| IMPROVEMENT |
| |
Details Improvements
Bug
| |||
| BUG FIXING IMPROVEMENT |
| |
Details Improvements
Bug fixing
| |||
| BUG FIXING IMPROVEMENT |
| |
Details Improvements
| |||
| IMPROVEMENT |
| |
Details Improvements:
| |||
| IMPROVEMENT |
| |
Details Improvements:
| |||
| IMPROVEMENT | | |
Details Improvements:
|
- No labels