Versions Compared

Version Old Version 35 New Version 36
Changes made by

Former user

Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

There are minimal requirements to setup this collector:

  1. GPC console access: Credentials are required to access the GCP console.

  2. Owner or Administrator permissions within the GCP console.

  3. Have a delegated email.

In order to retrieve the data, we need to create and authorize a Service account to authenticate the collector.

Expand
title1. Create a Project

  1. Login to Google APIs console.

  2. In the search bar, search Create a Project.

  3. Click on Create a Project.

  4. Fill in the required fields.

  5. Click on Create.

Expand
title2.Enable Google Workspace Alert Center API

  1. Login to Google APIs console.

  2. In the search bar, search Workspace Alert Center API.

  3. Click on Google Workspace Alert Center API.

  4. Click on Enable to activate the API.

Expand
title3. Create credentials

  1. In the search bar, search Credentials.

  2. Click on Credentials (APIs & Services).

  3. In the Service Accounts section, click on Manage service accounts.

  4. Click on the + Create service account button.

  5. Enter a Service account name and click Done.

  6. Click on the Email field of the created service account to access its details.

  7. Copy and save the Unique ID of the created service account.

  8. Click on the Keys tab.

  9. Click on Add key button.

  10. Click on Create new key. A pop-up window will open to select the details of the key.

  11. In key type select JSON.

  12. Click on the Create button.

  13. The file with the credentials will be downloaded automatically.

  14. Rename the file credentials.json.

  15. Save the file in <any_directory>/devo-collectors/gsuite-google-workspace-alerts/credentials/

  16. Convert the content of the credentials.json to base64. Recommended method using Python version higher or equal 3.6

    1. Copy the Python script below:

      Code Block
      import base64

def main():
    with open('credentials.json', 'r') as credentials:
        creds_as_bytes = credentials.read().encode('utf-8')
        base64creds = base64.b64encode(creds_as_bytes).decode("utf-8")
        print(f'Base64 encoded credentials.json: {base64creds}')

if __name__ == '__main__':
    main()

    2. Save the script to <any_directory>/devo-collectors/gsuite-google-workspace-alerts/credentials/.

    3. Rename it to b64encoder.py.

    4. Run the command below in the <any_directory>/devo-collectors/gsuite-google-workspace-alerts/credentials/ directory.

      Code Block
      $ python b64encoder.py

    5. The script will output a line starting with Base64 encoded credentials.json: Copy the base64 value as this will be required for the collector configuration.

  17. Copy the content of the json file. You can use any free software to convert the content of the json file to base64.

  18. Paste it into a base64 encoder and copy the result.

Info

Save base64 value

It is important to save the base64 value to later run the collector on-premise and in the collector server.

Info

Unique ID

The Unique ID (step 7) will be used later in the domain delegation.

Note

Don’t use base64 encoders

Online base64 encoders are not recommended for security reasons.

Expand
title4. Authorize the service account through Domain Wide Delegation.

Once the service account is created and with credentials, it is necessary to authorize it through Domain Wide Delegation.

  1. Login in the google administration console: https://admin.google.com/.

  2. In the left menu select Security → Access and data control → API controls.

  3. Click Manage domain wide delegation in the Domain wide delegation section.

  4. Click on Add new. A pop-up window will open to enter the details.

    1. In Client ID field enter the previously copied Service Account Unique ID.

    2. In OAuth scopes field add the following: https://www.googleapis.com/auth/apps.alerts.

  5. Finally click on Authorize.

Info

Delegated email

You can copy the email used to login to https://accounts.google.com/ to later use it as delegated_email in the collector configuration.

Note

Delegated email

delegated_email field should contain the email of a real user (you cannot use a service account here) with enough access to display the alerts in the Google WorkSpace Admin Console → Alert Center (https://admin.google.com/ac/ac).

...

Expand
titleRestart the persistence

This collector uses persistent storage to download events in an orderly fashion and avoid duplicates. In case you want to re-ingest historical data or recreate the persistence, you can restart the persistence of this collector by following these steps:

  1. Edit the configuration file.

  2. Change the value of the start_time parameter to a different one.

  3. Save the changes.

  4. Restart the collector.

The collector will detect this change and will restart the persistence using the parameters of the configuration file or the default configuration in case it has not been provided.

Note

Note that this action clears the persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.

Expand
titleTroubleshooting

This collector has different security layers that detect both an invalid configuration and abnormal operation. This table will help you detect and resolve the most common errors.

ErrorType

Error Id

Error Message

Cause

Solution

GSuiteAlertsSetupException

1

Error loading token pickle file: <exception_message>

This error is raised when unknown error occurs while loading token.pickle

Check that the token.pickle is in the correct format and if not, regenerate it again.

GSuiteAlertsPullerCredentialsException

0

HTTP error: <error_message>

This error is raised when an HTTP error appears during setup.

The solution depends on the type of error.. Contact with Devo Support team.

2

File <filename> does not exist. Please, learn how to generate a token pickle on: https://docs.devo.com/confluence/ndt/latest/sending-data-to-devo/collectors/g-suite-collectors/g-suite-alerts-collector

This error is raised when token.pickle does not exist.

Regenerate the token.json and save it in the file devo-collector-gsuite-google-workspace-alerts/credentials/.

To know how the token is regenerated, consult the section Vendor setup.

1

Unable to refresh token or client Auth was deleted. Detail: <error_message>

This error is raised when the token cannot be refreshed. The reason may be that it has been deleted.

Regenerate the token.json and save it in the file devo-collector-gsuite-google-workspace-alerts/credentials/.

To know how the token is regenerated, consult the section Vendor setup.

InputConfigurationError

70

<input_config_environment_key_path> property must be a string

This error is raised when optional value environment is not in the str format.

Edit the value of environment in the configuration file so it is of valid str format.

71

<input_config_environment_key_path> property must fulfil the regex pattern r'^[A-Za-z0-9_-]{1,32}$'

This error is raised when optional value environment does not match the required regex.

Make the value of environment in the configuration file match the indicated format.

GSuiteAlertsPullerConnectionLostException

0

Operation timed out: <error_message>

This error is raised when the maximum time to wait for the connection has been exceeded.

Check that the connection is working properly.

1

HTTP/1.1 503 Service Unavailable at moment - Retrying reconnection: <error_message>

This error is raised when the service is not available.

This is an internal issue. Contact the support team.

2

DefaultCredentials: <error_message>

This error is raised when credentials cannot be automatically determined.

This is an internal issue. Contact the support team.

0

Operation timed out

This error is raised when the maximum time to wait for the connection has been exceeded.

Check that the connection is working properly.

1

HTTP/1.1 503 Service Unavailable at moment - Retrying reconnection.

This error is raised when the service is not available.

This is an internal issue. Contact the support team.

3

<error_message>

This error is raised when credentials cannot be automatically determined.

This is an internal issue. Contact the support team.

GSuiteAlertsPullerCreationException

1

<module_properties_key_path> mandatory property is missing or empty'

This error is raised when module_properties property is not found in collector_definitions.yaml

This is an internal issue. Contact the support team.

2

<module_properties_key_path> property must be a dictionary

This error is raised when module_properties is defined in the collector_definitions.yaml file but the format is not dict.

This is an internal issue. Contact the support team.

3

<module_properties_key_path>.alert_type mandatory property is missing or empty

This error is raised when alert_type property is not found in collector_definitions.yaml

This is an internal issue. Contact the support team.

4

<module_properties_key_path>.alert_type property must be a string

This error is raised when alert_type is defined in the collector_definitions.yaml file but the format is not str.

This is an internal issue. Contact the support team.

5

<module_properties_key_path>.tag_base mandatory property is missing or empty

This error is raised when tag_base property is not found in collector_definitions.yaml

This is an internal issue. Contact the support team.

6

<module_properties_key_path>.tag_base property must be a string

This error is raised when tag_base is defined in the collector_definitions.yaml file but the format is not str.

This is an internal issue. Contact the support team.

7

<module_properties_key_path>.credentials_scopes" mandatory property is missing or empty

This error is raised when credentials_scopes property is not found in collector_definitions.yaml

This is an internal issue. Contact the support team.

8

<module_properties_key_path>.credentials_scopes" property must be a string

This error is raised when credentials_scopes is defined in the collector_definitions.yaml file but the format is not str.

This is an internal issue. Contact the support team.

9

<input_config_key_path> property must be a dictionary

This error is raised when the required property input_config_key_path is not found in the config file.

Add input_config_key_path to config file, for example: gsuit_alerts:

10

<input_config_credentials_key_path> mandatory property is missing or empty

This error is raised when the required property credentials is not found in the config file.

Add credentials dictionary in config.

11

<input_config_credentials_key_path> property must be a dictionary

This error is raised when credentials is defined in the configfile but the format is not dict.

Edit the value of credentials in config file, so it is of valid dict format.

12

<input_config_credentials_key_path>.filename mandatory property is missing or empty

This error is raised when the required property filename is not found in the config file, into credentials dictionary.

Add filename property in config file, into credentials dictionary.

13

<input_config_credentials_key_path>.filename property must be a string

This error is raised when filename is defined in the configfile but the format is not str.

Edit the value of filename in config file, so it is of valid str format.

14

<input_config_credentials_key_path>.content_base64" property must be a string

This error is raised when content_base64 is defined in the configfile but the format is not str.

Edit the value of content_base64 in config file, so it is of valid str format.

15

<input_config_credentials_key_path>.content_base64" must be in a valid base64 format

This error is raised when content_base64 is defined in the configfile but the format is not base64 string.

Edit the value of content_base64 in config file, so it is of valid base64 string format.

16

<input_config_credentials_key_path>.delegated_email mandatory property is missing or empty

This error is raised when the required property delegated_email is not found in the config file, into credentials dictionary.

Add delegated_email property in config file, into credentials dictionary.

17

<input_config_credentials_key_path>.delegated_email property must be a string

This error is raised when delegated_email is defined in the configfile but the format is not str.

Edit the value of delegated_email in config file, so it is of valid str format.

18

<input_config_credentials_key_path>.source_id mandatory property is missing or empty

This error is raised when the required property source_id is not found in the config file, into credentials dictionary.

Add source_id property in config file, into credentials dictionary.

19

<input_config_credentials_key_path}.source_id> property must be a string'

This error is raised when source_id is defined in the configfile but the format is not str.

Edit the value of source_id in config file, so it is of valid str format.

20

\"start_time\" property from configuration file format is invalid

This error is raised when start_time is defined in the configfile but the format is not "%Y-%m-%dT%H:%M:%S.%fZ" or "%Y-%m-%dT%H:%M:%SZ".

Edit the value of start_time in config file, so it is of valid "%Y-%m-%dT%H:%M:%S.%fZ" or "%Y-%m-%dT%H:%M:%SZ" format.

PrePullError

200

The <state> does not meet the structure requirements of this collector version. '
f'The keys <last_alert_timestamp> and <last_alerts_id> are mandatory. State = <state>

This error is raised when the collector is upgraded or downgraded to an incompatible version.

  1. Go back to a compatible version.

  2. Remove persistence by modifying the start_time property.

PullError

300

Puller action cannot be executed when PrePull step failed

This error is raised when the prepull fails for whatever reason.

Check the logs to find the PrePull error.

301

Could not automatically determine credentials. Cause:<exception_message>. Aborting...

This error is raised when credentials cannot be automatically determined.

This is an internal issue. Contact the support team.

302

The <next_timestamp> value cannot be calculated, please check the related error logs.

This error is raised when next_timestampt cannot be calculated for the next request.

Check the logs to find the related errors.

...