Proofpoint Tap collector migration guide (2.x.x. to 3.x.x)

[ 1 About migration ] [ 2 Overview of Changes ] [ 3 Preparing for Migration ] [ 4 Migration Steps ]

About migration

This guide will walk you through the process of updating your configuration from the old version (2.x.x) to the new version (3.x.x). The new version introduces significant improvements and changes to the configuration style to enhance performance, usability, and security.

Overview of Changes

The new configuration format introduces several key changes:

New Services: The configuration now supports new services (threats, campaigns, people_topclicks, people_vap).
Added new puller for campaign, threats, people_topclicks, people_vap.

Preparing for Migration

Before starting the migration process, we recommend the following steps:

Backup Your Current Configuration: Always ensure you have a backup of your existing configuration files to prevent any data loss.
Review the New Configuration Documentation: Familiarize yourself with the new configuration options available in version 3.x.x.

Migration Steps

Step 1: Update the Service Configuration Parameter Field Name

An example of the old and new configuration is shown below:

# Old Version (2.x.x)
  services:
       clicksBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
       clicksPermitted:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag>  #optional
       messagesBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag>  #optional
       messagesDelivered:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag>  #optional

↓

# New Version (3.x.x)
    services:
       clicksBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       clicksPermitted:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       messagesBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       messagesDelivered:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       threats:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         forensics: <boolean>
         include_campaign_forensics: <boolean>
       campaigns:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         forensics: <boolean>
       people_topclicks:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       people_vap:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z

Step 2: Update in the services Configuration

The new services have been added to the proofpoint_tap collector like threats, campaigns, people_topclicks and people_vap.

An example of the new configuration is shown below:

# New version (3.x.x)
  services:
       clicksBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
       clicksPermitted:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
       messagesBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
       messagesDelivered:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
       threats:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
         forensics: <boolean>
         include_campaign_forensics: <boolean>
       campaigns:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
         forensics: <boolean>
       people_topclicks:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional
       people_vap:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         override_tag: <override_tag> #optional

Each of the services have override_tag, request_period_in_seconds as an optional parameter.

Step 3: Before and After Configuration

Putting it all together, see below for an example of the old and new configuration:

↓

Step 4: Changes in the of the old and new version

4 new services have been added threats, campaigns, people_topclicks, people_topclicks.
threats service have additional params forensics and include_campaign_forensics both are boolean, can be true and false. It is a mandatory parameter.
For threats service, forensics is a mandatory parameter and it changes the API endpoint to include forensic information.
For threats service, include_campaign_forensics is a mandatory parameter and it gives additional information for Campaign forensics only.
campaigns service have additional params forensics. it is boolean, can be true and false
For campaigns service, forensics is a mandatory parameter and it changes the API endpoint to include forensic information.