...
Mimecast is a cloud-based, anti-spam, and archive filtering service for securing email accounts and communications for businesses. This collector
Mimecast protects an enterprise’s email infrastructure from viruses, malware, phishing, and the rise of deep-fake attacks. It does this by deploying a layered cyber resilience solution that prevents email-borne infections and reduces data loss by archiving emails. This cloud-based cybersecurity solution also makes it possible to automate the recovery of archived and affected emails for continuous use. It
The Mimecast approach to protecting email structures means it can predict and or anticipate attacks and deal with losses in order to handle real-time threats. It also deals with data loss from ransomware attacks using data archiving. The Devo Mimecast Collector uses the Mimecast API to extract all the relevant information and send it as events to Devo., which eliminates the need to meet ransom demands, as well as struggle with downtime. Mimecast can also be deployed to tackle those annoyingly ‘spammy’ messages that keep cluttering inboxes.
For those who already use any of the popular email management brands such as Microsoft Office 365, Outlook, or Google’s Gsuite, Mimecast’s cloud-based nature makes it compatible with them. It can be deployed to tackle spam, ransomware, or other cybersecurity challenges.
The Devo Mimecast Collector uses the Mimecast API to extract all the relevant information an send it as events to Devo.
Data sources
Data Sourcesource | Description | API Endpointendpoint | Devo table | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Audit | Attachments |
|
| ||||||||||||
Audit |
|
| |||||||||||||
Attachments | Attachment Protection LogsDashboard |
|
| ||||||||||||
Impersonation |
|
| Url | ||||||||||||
TTP URL LogsMessageholdlist |
|
| |||||||||||||
Search | Search LogsMessageholdsummary |
|
| ||||||||||||
ViewSearch | Archive Message View Search Logs |
|
| Threatfeed | |||||||||||
|
| Messageholdlist |
| Messageholdsummary |
| Dashboard |
| Siem |
| ||||||
Siem (API v2) |
|
|
...
| |||
Threatfeed |
|
| |
Url |
|
| |
View |
|
|
For more information on how the events are parsed, visit our page.
Vendor setup
There are some requirements to configure the Mimecast collector:
Accessing your API applications.
Creating user API keys. Refer to the Mimecast official documentation for more information.
Expand | ||
---|---|---|
| ||
Once your API applications display you can:
|
Expand | ||
---|---|---|
| ||
Scroll to the middle of API Concepts for detailed instructions. |
Authentication
The Mimecast Collector API 2.0 needs two keys that the API uses:
Client ID (
client_id
).Client secret (
client_secret
)
Expand | ||
---|---|---|
| ||
Steps and information to generate these keys can be found in this article. |
Expand | ||
---|---|---|
| ||
Each API call has a prerequisite section that tells you what permissions are needed for the call. Usually, a basic administrator role will be enough, which should allow you to use the same API keys generated for multiple API calls under the application. If you want to create a custom administrative role for this API service account user, follow these steps:
If you want to add the service account user to an existing role:
Find more details in the Customer Community. |
Run the collector
Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).
...
Rw tab | ||
---|---|---|
|
This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.
Structure
The following directory structure should be created for being used when running the collector:
Code Block |
---|
<any_directory>
└── devo-collectors/
└── <product_name>/
├── certs/
│ ├── chain.crt
│ ├── <your_domain>.key
│ └── <your_domain>.crt
├── state/
└── config/
└── config.yaml |
Note |
---|
Replace |
Devo credentials
In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/
. Learn more about security credentials in Devo here.
...
Note |
---|
Replace |
Editing the config.yaml file
...
Expand | ||
---|---|---|
| ||
OverviewMimecast API 2.0 uses OAuth 2.0 to authenticate with the new Mimecast API Gateway using a dedicated Application (created and configured by the customer). To register and configure an Application:
AuthenticationAfter this process, the two keys that the Mimecast Collector API 2.0 needs are created, the keys are:
|
Expand | ||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||
OverviewFollowing steps are necessary for setup at the Mimecast side. Log in from https://www.mimecast.com/tech-connect/documentation/api-overview/api-concepts/ Accessing your API applications:
With your API applications displayed you can:
Further information may be found here: https://community.mimecast.com/s/article/Managing-API-Applications-505230018 Creating user API keys: Scroll to middle of: https://www.mimecast.com/tech-connect/documentation/api-overview/api-concepts/ for detailed instructions. AuthenticationThe Mimecast Collector needs four keys that the API uses, the four keys are:
Credentials
More details https://community.mimecast.com/s/article/Managing-API-Applications-505230018#Creating-an-API-user-Authentication-Profile .
More details https://community.mimecast.com/s/article/Managing-API-Applications-505230018#Creating-an-API-user-Authentication-Profile . |
Permissions (both API 1 and 2)
Each API call has a prerequisite section that tells you what permissions are needed for the call. Usually, a Basic Administrator role will suffice, which should allow you to use the same API keys generated for multiple API calls under the application.
Service | Permissions |
---|---|
SIEM Audit | Gateway | Tracking | Read |
Audit | Account | Logs | Read |
TTP attachment | Monitoring | Attachment Protection | Read |
TTP impersonation | Monitoring | Impersonation Protection | Read |
TTP URL | Monitoring | URL Protection | Read |
Archive search | Archive | Search Logs | Read |
Archive view | Archive | View Logs | Read |
TTP Thread intel | Services | Gateway | Tracking | Read |
Message Hold List | Account | Dashboard | Read |
Message Hold Summary | Account | Monitoring | Held Summary | Read |
Dashboard | Account | Dashboard | Read |
If you want to create a custom administrative role for this API service account user:
Navigate to Administration | Account | Roles.
Click New Role.
Enter a Role Name and Description.
In the Application Permissions section, select the boxes for each required role to be used by the service user account.
Click Save and Exit.
Locate the newly created role and click on the role name.
Click Add User to Role.
Click on the email address of the API service user account.
If you want to add the service account user to an existing role:
Navigate to Administration | Account | Roles.
Click on the administrator role the user will be added to.
Click Add User to Role.
Click on the email address of the API service user account.
More details https://community.mimecast.com/s/article/Managing-API-Applications-505230018#Creating-an-API-user-Authentication-Profile .
Run the collector
Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).
...
Rw ui tabs macro | ||||||||
---|---|---|---|---|---|---|---|---|
We use a piece of software called Collector Server to host and manage all our available collectors. To enable the collector for a customer:
Editing the JSON configuration
Replace the placeholders with your required values following the description table below: | ||||||||
Parameter | Data type | Type | Value range/ Format | Details | ||||
|
|
| false / true | This will make the collector generate (or not) log messages with the DEBUG level. | ||||
|
| Mandatory | Minimum length: 1 | Alphanumeric identifier. | ||||
|
| Mandatory |
| Enables or disables the input. | ||||
|
| Mandatory | The region related to the API credentials, the string value has to be one of the valid URL’s in Global Base URLs | Mimecast. |
|
| Mandatory | ||
Code Block |
" } }, { "endpoints_5": { "name": "search", "initial_lookback_period": "1d" } }, { "endpoints_6": { "name": "view", "initial_lookback_period": "1d" } }, { |
"endpoints_ |
Credentials to use the API.
endpoints
list
Mandatory
Minimum length: 1
Posible values:
7": { |
|
|
"name": "threatfeed", |
|
"initial_lookback_period": "1d" |
|
|
|
|
|
|
} |
|
|
|
}, |
|
|
{ |
|
|
|
|
"endpoints_ |
8": |
{ |
|
|
|
|
|
"name": "messageholdlist |
", |
|
|
|
|
|
"initial_lookback_period": "1d" |
|
|
|
An array with at least one endpoint, the collector will pull from the selected endpoints.
last_configuration_timestamp
str
Mandatory
Date following the next format:
yyyy-mm-ddThh:mm:ss.000Z
Change this value to a date after the initial configuration to reset the state of the collector.
initial_lookback_period
str
Mandatory
Number of days, Example:
1d
This value will be subtracted from the current date to execute all queries in that range if no state is detected (Initial execution for example).
This value only has an effect for mimecast_input
, mimecast_siem_input
always pull from the last 7 days.
Download the Docker image
The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:
Collector Docker image | SHA-256 hash |
---|---|
|
Use the following command to add the Docker image to the system:
Code Block |
---|
gunzip -c <image_file>-<version>.tgz | docker load |
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace <image_file>
and <version>
with a proper value.
The Docker image can be deployed on the following services:
Docker
Execute the following command on the root directory <any_directory>/devo-collectors/<product_name>/
Code Block |
---|
docker run \
--name <YOUR_COLLECTOR_NAME>\
--volume $PWD/certs:/devo-collector/certs \
--volume $PWD/config:/devo-collector/config \
--volume $PWD/state:/devo-collector/state \
--env CONFIG_FILE=<YOUR-CONFIG-FILE>.yaml \
--rm -it <YOUR_DEVO_IMAGE_NAME>:<version> |
Note |
---|
Replace |
Docker Compose
The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/<product_name>/
directory.
Code Block |
---|
version: '3'
services:
<YOUR_COLLECTOR_NAME>:
image: <YOUR_DEVO_IMAGE_NAME>:${IMAGE_VERSION:-latest}
volumes:
- ./certs:/devo-collector/certs
- ./config:/devo-collector/config
- ./state:/devo-collector/state
environment:
- CONFIG_FILE=${CONFIG_FILE:-<YOUR-CONFIG-FILE>.yaml} |
To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/<product_name>/
directory:
Code Block |
---|
IMAGE_VERSION=<version> docker-compose up -d |
Note |
---|
Replace |
Rw tab | ||
---|---|---|
|
We use a piece of software called Collector Server to host and manage all our available collectors.
To enable the collector for a customer:
In the Collector Server GUI, access to the domain in which you want this instance to be created in, click on Add Collector and search for “Mimecast Collector - Integrations Factory”, then click on the result.
In the Version field, select the latest value.
In the Collector Name field, set the value you prefer (this name must be unique inside the same Collector Server domain).
In the Parameters section, establish the Collector Parameters as follows below:
Editing the JSON configuration
Code Block |
---|
{ "mimecast_input": {} }, { "endpoints_9": { "name": "messageholdsummary", "initial_lookback_period": "1d" } }, { "endpoints_10": { "name": "dashboard", "initial_lookback_period": "1d" } } ] } } }, "mimecast_siem_input": { "id": "<short_unique_identifier>", "enabled": true, "requests_per_second": "requests_per_second_value", "base_url": "your_base_url", "auth_url": "your_auth_url", "idpageSize": "<shortpage_uniquesize_identifier>value", "enabledautoconfig": { true, "base_urlrefresh_interval_in_seconds": "yourrefresh_baseinterval_urlvalue", "auth_urlcreation_timeout_in_second": "yourcreation_authtimeout_urlvalue", "pageSize": "<page_size_value>"}, "autoconfigcredentials": { "refresh_interval_in_secondsclient_id": "refreshyour_intervalclient_valueid", "creation_timeout_in_secondclient_secret": "creationyour_timeoutclient_value" secret", }, "credentialsapp_id": {"your_app_id", "clientapp_idkey": "your_clientapp_idkey", "clientaccess_secretkey": "your_clientaccess_secret" key", }, "secret_key": "your_secret_key" }, "services": { "service_mimecast_siem_client_api": { "last_configuration_timestamp": "last_configuration_timestamp_value", "endpoints": [ { "endpoints_1siem": { "nameinitial_lookback_period": "audit0d", "initialpage_lookback_periodtoken": "1d<page_token>" } }, } { } } "endpoints_2": { "name": "attachments", "initial_lookback_period": "1d" }} } |
Info |
---|
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the |
Please replace the placeholders with real world values following the description table below:
Parameter | Data Type | Type | Value Range/ Format | Details | |
---|---|---|---|---|---|
|
| Mandatory | Minimum length: 1 | Alphanumeric identifier. | |
|
| Mandatory |
| Enables or disables the input. | |
|
| Mandatory | For v2 API For v1 API, see: | Base url for all the APIs | |
|
| Mandatory | For v2 API: Delete this parameter | Auth url to generated auth token. | |
|
| Mandatory | For v2 API:
|
|
For v1:
|
|
|
|
|
|
|
|
|
| Credentials to use the API. | ||||
|
| Mandatory | Minimum length: 1 Posible values:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| An array with at least one endpoint, the collector will pull from the selected endpoints. | |||
|
| Mandatory | Date following the next format:
| Change this value to a date after the initial configuration to reset the state of the collector. |
|
| Mandatory | Number of days, Example:
| This value will be subtracted from the current date to execute all queries in that range if no state is detected (Initial execution for example). This value only has an effect for |
|
| Optional | Token from app log | Advanced theme: it is possible to put a pagination token from the collector to start fetching data from a given page. |
We recommend to leave parameters not in the list with their default values.
Keep in mind that the Mimecast collector has two different inputs:
mimecast_input
mimecast_siem_input
The collector can use both inputs or just one. Each input uses different endpoints and feeds different tables in Devo. Make sure to check the credentials given to determine the inputs and endpoints to use.
Rw tab | ||
---|---|---|
|
This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.
Structure
The following directory structure should be created for being used when running the collector:
"search",Code Block |
---|
<any_directory> └── devo-collectors/ "initial_lookback_period": "1d"└── <product_name>/ } }, { "endpoints_6": {├── certs/ │ ├── chain.crt "name": "view", │ ├── <your_domain>.key │ └── "initial_lookback_period": "1d"<your_domain>.crt ├── state/ } └── config/ }, └── { "endpoints_7": { "name": "threatfeed", "initial_lookback_period": "1d" } }, { "endpoints_8": { "name": "messageholdlist", "initial_lookback_period": "1d" } }, { "endpoints_9": { "name": "messageholdsummary", "initial_lookback_period": "1d" } }, { "endpoints_10": { config.yaml |
Note |
---|
Replace |
Devo credentials
In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <product_name>/certs/
. Learn more about security credentials in Devo here.
Note |
---|
Replace |
Editing the config.yaml file
Code Block |
---|
globals: debug: false id: not_used name: mimecast_collector persistence: type: filesystem config: directory_name: state queue_max_size_in_messages: 1000 queue_wrap_max_size_in_messages: 100 outputs: # devo_1: # type: devo_platform # config: # address: collector-us.devo.io # port: 443 # type: SSL # chain: chain.crt # cert: <devo_domain>.crt # key: <devo_domain>.key inputs: mimecast_input: id: 1 enabled: true requests_per_second: 5 base_url: your_base_url auth_url: your_auth_url pageSize: 1000 autoconfig: refresh_interval_in_seconds: 60 # Runs the setup every x seconds (default 600) creation_timeout_in_second: 60 # Set up the setup timeout (default 60) credentials: client_id: <client_id_value> client_secret: <client_secret_value> app_id: <app_id_value> app_key: <app_key_value> access_key: <access_key_value> secret_key: <secret_key_value> services: service_mimecast_client_api: "name": "dashboard", last_configuration_timestamp: 2021-12-02T13:10:00Z # change this if you want to get your state changed! "initial_lookback_period": "1d" endpoints: - endpoints_1: } }name: audit ] initial_lookback_period: 1d } } - endpoints_2: }, "mimecast_siem_input": { "id"name: "<short_unique_identifier>",attachments "enabled": true, "requestsinitial_perlookback_second"period: "requests_per_second_value", 1d "base_url": "your_base_url", - "authendpoints_url": "your_auth_url",3: "pageSize": "page_size_value", "autoconfig": {name: impersonation "refreshinitial_interval_in_seconds": "refresh_interval_value",lookback_period: 1d - "creation_timeout_in_second": "creation_timeout_value"endpoints_4: }, "credentials": {name: url "client_id": "your_client_id", initial_lookback_period: 1d "client_secret": "your_client_secret" - endpoints_5: }, "services": { name: search "service_mimecast_siem_client_api": { "lastinitial_configurationlookback_timestamp"period: "last_configuration_timestamp_value", 1d - "endpoints"_6: { "siem"name: {view "initial_lookback_period": "1d" } } - endpoints_7: } } name: threatfeed } } |
Info |
---|
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the |
Please replace the placeholders with real world values following the description table below:
Parameter
Data Type
Type
Value Range/ Format
Details
id
str
Mandatory
Minimum length: 1
Maximum length: 5
Alphanumeric identifier.
enabled
bool
Mandatory
true
/false
Enables or disables the input.
base_url
str
Mandatory
The region related to the API credentials, the string value has to be one of the valid URL’s in https://integrations.mimecast.com/documentation/api-overview/global-base-urls/.
credentials
dictionary
Mandatory
Code Block |
---|
"credentials": {
"client_id": "your_client_id",
"client_secret": "your_client_secret"
}, |
Credentials to use the API.
endpoints
list
Mandatory
Minimum length: 1
Posible values:
initial_lookback_period: 1d - endpoints_8: name: messageholdlist initial_lookback_period: 1d - endpoints_9: name: messageholdsummary initial_lookback_period: 1d - endpoints_10: name: dashboard initial_lookback_period: 1d mimecast_siem_input: id: 2 enabled: false requests_per_second: 5 base_url: your_base_url auth_url: your_auth_url pageSize: 10 autoconfig: refresh_interval_in_seconds: 60 # Runs the setup every x seconds (default 600) creation_timeout_in_second: 60 # Set up the setup timeout (default 60) credentials: client_id: <client_id_value> |
client_secret: <client_secret_value> |
app_id: <app_id_value> |
app_key: <app_key_value> |
access_ |
key: |
<access_key_value> secret_key: <secret_key_value> |
services: |
service_mimecast_siem_client_api: |
last_ |
configuration_ |
timestamp: |
2021-12-02T13:10:00Z # change this if you want to get your state changed! endpoints: |
siem: |
|
initial_lookback_period: 0d |
page_token: <page_token> |
Info |
---|
All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the |
Replace the placeholders with your required values following the description table below:
Parameter | Data type | Type | Value range/ Format | Details | |
---|---|---|---|---|---|
|
| Mandatory | Minimum length: 1 | Alphanumeric identifier. | |
|
| Mandatory |
| Enables or disables the input. | |
|
| Mandatory | For v2 API For v1 API, see: | Base url for all the APIs | |
|
| Mandatory | For v2 API: Delete this parameter | Auth url to generated auth token. | |
|
| Mandatory |
|
|
| Credentials to use the API. | ||||
|
| Mandatory | Minimum length: 1 Posible values:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
An array with at least one endpoint, the collector will pull from the selected endpoints.
last_configuration_timestamp
str
Mandatory
Date following the next format:
yyyy-mm-ddThh:mm:ss.000Z
Change this value to a date after the initial configuration to reset the state of the collector.
initial_lookback_period
str
Mandatory
Number of days, Example:
1d
This value will be subtracted from the current date to execute all queries in that range if no state is detected (Initial execution for example).
This value only has an effect for mimecast_input
, mimecast_siem_input
always pull from the last 7 days.
We recommend to leave parameters not in the list with their default values.
Keep in mind that the Mimecast collector has two different inputs:
mimecast_input
mimecast_siem_input
The collector can use both inputs or just one. Each input uses different endpoints and feeds different tables in Devo. Make sure to check the credentials given to determine the inputs and endpoints to use. Using Postman is a great way to check the credentials to each input. Read more here.
Input
Endpoint
Tables
mimecast_input
/api/audit/get-audit-events
/api/ttp/attachment/get-logs
/api/ttp/impersonation/get-logs
/api/ttp/url/get-logs
/api/archive/get-search-logs
/api/archive/get-view-logs
/api/ttp/threat-intel/get-feed
/api/gateway/get-hold-message-list
/api/gateway/get-hold-summary-list
/api/account/get-dashboard-notifications
mail.mimecast.audit.events
mail.mimecast.ttp.attachment
mail.mimecast.ttp.impersonation
mail.mimecast.ttp.url
mail.mimecast.archive.search
mail.mimecast.archive.messageview
mail.mimecast.threat.feed
mail.mimecast.message.list
mail.mimecast.message.summary
mail.mimecast.account.dashboard
mimecast_siem_input
/api/audit/get-siem-logs
mail.mimecast.siem.receipt
mail.mimecast.siem.process
mail.mimecast.siem.delivery
mail.mimecast.siem.jrnl
mail.mimecast.siem.av
mail.mimecast.siem.iep
mail.mimecast.siem.impersonation
mail.mimecast.siem.spameventthread
mail.mimecast.siem.ttp
Change log
...
Release
...
Released on
...
Release type
...
Details
Download the Docker imageThe collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:
Use the following command to add the Docker image to the system:
Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace The Docker image can be deployed on the following services: DockerExecute the following command on the root directory
Docker ComposeThe following Docker Compose file can be used to execute the Docker container. It must be created in the
To run the container using docker-compose, execute the following command from the
|
API limits and duplicates
The Mimecast API has some call rate limits. When a limit is reached, the collector shows a 429 error. More details about MImecast limits can be found here and here
The Mimecast API sometimes sends duplicate events (it is not common). The collector tries to filter out the duplicates, but it is not possible to guarantee that all duplicates are deleted.
Change log
Release | Released on | Release type | Recommendations | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
| ||||||||||||
| |||||||||||||||
|
|
| |||||||||||||
| |||||||||||||||
|
|
| |||||||||||||
| |||||||||||||||
|
|
|
|
| ||||||
|
Recommended version