Proofpoint Tap collector migration guide (1.x.x. to 2.x.x)

[ 1 About migration ] [ 2 Overview of Changes ] [ 3 Preparing for Migration ] [ 4 Migration Steps ]

About migration

This guide will walk you through the process of updating your configuration from the old version (1.x.x) to the new version (2.x.x). The new version introduces significant improvements and changes to the configuration style to enhance performance, usability, and security.

Overview of Changes

The new configuration format introduces several key changes:

In the new version, to ingest the data we need to add that service name in the config.
user is changed into username.
credential is changed into password.
username and password of the new version are under credentials subheading.
There is no separate puller for any services all the services have a unified puller.

Preparing for Migration

Before starting the migration process, we recommend the following steps:

Backup Your Current Configuration: Always ensure you have a backup of your existing configuration files to prevent any data loss.
Review the New Configuration Documentation: Familiarize yourself with the new configuration options available in version 2.0.0.

Migration Steps

Step 1: Update the Service Configuration Parameter Field Name

An example of the old and new configuration is shown below:

# Old Version (1.x.x)
  user: <user>
  credential: <credentials>

↓

# New Version (2.x.x)
    services:
       clicksBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       clicksPermitted:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       messagesBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
       messagesDelivered:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z

There are some changes in the params we are providing for each services in old and new configurations.

In the old version we just had to put user and credential in order to get the data of all the services
In the new version we have to put the name of the service to get the data of that particular service.

Step 2: Update in the services Configuration

An example of the new configuration is shown below:

# New version (2.0.0)
  services:
       clicksBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         <override_tag>: <override_tag>
       clicksPermitted:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         <override_tag>: <override_tag>
       messagesBlocked:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         <override_tag>: <override_tag>
       messagesDelivered:
         start_time_in_utc_format: <start_time_in_utc_format> # example 2022-05-14T00:00:00Z
         <override_tag>: <override_tag>

Each of the services have override_tag, request_period_in_seconds as an optional parameter. override_tag will override the devo_tag for that service and request_period_in_seconds will tell the waiting time of the service before starting another pull cycle after completion of a pull cycle.

Step 3: Before and After Configuration

Putting it all together, see below for an example of the old and new configuration:

↓

Step 4: Changes in the old and new service

user is changed into username
credential is changed into password
username and password in the new version must be put under credentials subheading.
Number of requests all the services can make is 1800 in a day after that new calls will be blocked for a day.
By default request_period_in_seconds is 240 second so if no request_period_in_seconds param is provided in the user_config each service will starting new pull cycle after waiting for 240 seconds.