Document toolboxDocument toolbox

Cisco eStreamer collector

Service description

The Cisco Event Streamer (also known as Cisco eStreamer) allows you to stream Firepower System events to external client applications. You can stream host, discovery, correlation, compliance allow list, intrusion, user activity, file, malware, and connection data from a Management Center and you can stream intrusion data from 7000 and 8000 series devices.

Data source description

Currently, the Cisco eStreamer collector generates host, discovery, correlation, compliance allow list, intrusion, user activity, file, malware, and connection events. The collector processes the eStreamer responses and sends them to the Devo platform, which will categorize all the information received on the following tables:

Group name

Details

Data tables

Group name

Details

Data tables

Metadata

Context information for codes and numeric identifiers in the event records

firewall.cisco.fmc_estreamer.metadata

Packet

Packets associated with intrusion events

firewall.cisco.fmc_estreamer.packet

Intrusion

Intrusion events generated by managed devices

firewall.cisco.fmc_estreamer.intrusion

File malware

Malware events

firewall.cisco.fmc_estreamer.file_malware

Correlation

Correlation and allow list events

firewall.cisco.fmc_estreamer.correlation

Connection

Connection events

firewall.cisco.fmc_estreamer.connection

RNA

Realtime Network Awareness events

firewall.cisco.fmc_estreamer.rna

RUA

Realtime User Awareness events

firewall.cisco.fmc_estreamer.rua

Event

Additional data for intrusion events

firewall.cisco.fmc_estreamer.event

For more info about the Cisco eStreamer, visit the Firepower System Event Streamer Integration Guide.

Setup

The Cisco eStreamer data collector works over the Cisco FMC (Firepower Management Center) devices. To start receiving data from the eStreamer protocol, you need to set up the eStreamer service in the FMC.

Setting up eStreamer

  1. Access the FMC web console.

  2. Go to System → Integration → eStreamer

  3. Check the events that you want to receive and save the changes.

  4. Create a new client and save the certificate (and password) to use later in the collector. Using a password is mandatory. The IP to use is the IP of the collector, the client, not the one from FMC server.

Note that if you are using the collector through the Cloud collector, you must use a public IP. Internal IP addresses will not work if you are using the on-premise collector.

Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).

Troubleshooting

Some common points to check in case of problems:

  • Doublecheck that the IP of the FMC is the parameter host, and the certificate was created for the IP of the client (the collector).

  • The FMC has the API server listening and the IP address that is used can be checked by executing this command from a terminal in the FMC: netstat -an | grep 8302

  • It is needed that both FMC and the collector have network visibility. Check if there is a firewall between the FMC and the collector. Port 8302 must be open for the collector. You can check it by installing nmap utility in the collector computer (for instance, in Ubuntu or Debian, sudo apt install nmap). Then execute from a terminal in the collector computer: nmap -p8302 x.y.ip.fmc where x.y.ip.fmc should be the IP of the FMC.

  • The certificate generated should use a password. A password-less certificate can cause an error in the collector

  • It is possible to discover if the FMC is rejecting the certificate. Execute in a terminal in the FMC the command cat /var/log/messages | grep "EventStreamer" | grep "Certificate" or cat /var/log/messages | grep "EventStreamer" | grep "Certificate" | grep ERROR

  • Other interesting points to check can be found on this website.

Change log

Release

Released on

Release type

Details

Recommendations

Release

Released on

Release type

Details

Recommendations

v1.3.0

Jul 5, 2023

IMPROVEMENTS

Improvements:

  • Upgraded DCSDK from 1.6.1 to 1.8.0:

    • Ability to validate collector setup and exit without pulling any data

    • Ability to store in the persistence the messages that couldn't be sent after the collector stopped

    • Ability to send messages from the persistence when the collector starts and before the puller begins working

    • Ensure special characters are properly sent to the platform

    • Added a lock to enhance sender object

    • Added new class attrs to the setstate and getstate queue methods

    • Fix sending attribute value to the setstate and getstate queue methods

    • Added log traces when queues are full and have to wait

    • Added log traces of queues time waiting every minute in debug mode

    • Added method to calculate queue size in bytes

    • Block incoming events in queues when there are no space left

    • Send telemetry events to Devo platform

    • Upgraded internal Python dependency Redis to v4.5.4

    • pgraded internal Python dependency DevoSDK to v5.1.3

    • Fixed obfuscation not working when messages are sent from templates

    • New method to figure out if a puller thread is stopping

    • pgraded internal Python dependency DevoSDK to v5.0.6

    • Improved logging on messages/bytes sent to Devo platform

    • Fixed wrong bytes size calculation for queues

    • New functionality to count bytes sent to Devo Platform (shown in console log)

    • Upgraded internal Python dependency DevoSDK to v5.0.4

    • Fixed bug in persistence management process, related to persistence reset

    • Aligned source code typing to be aligned with Python 3.9.x

Recommended version

v1.2.0

Feb 16, 2023

IMPROVEMENTS

Improvements:

  • Enabled additional logs for the eStreamer client.

  • DevoCollectorSDK upgraded to 1.6.1:

    • Added

      • A new key called @devo_environment will be added to the event(only for JSON events).

      • Obfuscation service can be now configured from user config and module definition.

      • Obfuscation service can now obfuscate items inside arrays.

Upgrade

v1.1.3

Jul 11, 2022

BUG FIX

Bug fixes:

  • Fixed a bug that prevented the use of the functionality to read the Cisco certificate file from the /certificates folder when running on-prem.

Recommended version

v1.1.2

May 24, 2022

IMPROVEMENTS

Improvements:

  • Validated base64 variables from config.yaml. A new function was created to check if the base64 token in the configuration file has a valid format.

  • Added standard exceptions (InitVariablesError, PrePullError, PullError...) to improve the troubleshooting. The collector was throwing generic exceptions.

Upgrade

v1.1.0

Apr 13, 2022

IMPROVEMENTS

VULNS

Improvements:

  • The underlay IFC SDK has been updated from v1.1.2 to v1.1.3.

  • The resilience has been improved with a new feature that restart the collector when the Devo connections is lost and it cannot be recovered.

Vulnerabilities mitigation:

  • All critical and high vulnerabilities have been mitigated.

Upgrade