Salesforce collector

[ 1 Service description ] [ 2 Data source description ] [ 3 Setup ] [ 4 Devo collector features ] [ 5 Minimum configuration required for basic pulling ] [ 6 Accepted authentication methods ] [ 7 Run the collector ]

Service description

Salesforce is a customer relationship management (CRM) solution that brings companies and customers together. It's one integrated CRM platform that gives all the departments, including marketing, sales, commerce, and service. This collector provides the possibility to integrate Salesforce with the Devo Platform making it easy to query and analyze the Salesforce data, view it in the pre-configured Activeboards, or customize them to enable Enterprise IT and Cybersecurity teams to make impactful data-driven decisions.

The Devo Salesforce Collector enables customers to retrieve data from the “Salesforce Objects API“ (a.k.a. SObjects). The collector processes the Salesforce API responses and sends them to the Devo platform which then categorizes all data received on tables along rows and columns in your Devo domain.

This collector works only with the Enterprise, Performance, Unlimited, Developer, and Database Salesforce editions. Make sure that you have one of the listed editions if you want to use the collector.

Data source description

Listed in the table below are the SObject names, their details, and how the Devo Platform treats the data. You can see that all the SObject events (JSON documents) fall under the crm.salesforceobjects technologies, except for the “event log files” processed events, which fall under the crm.salesforce one:

Data source	Table	Collector service	Related remote endpoint	Description

Data source	Table	Collector service	Related remote endpoint	Description
Account	`crm.salesforceobjects.account`	`Account`	The main endpoint is `GET https://{instance}/services/data/v{version}/{query_method}?q={query}`, where: `{instance}` is your Salesforce domain name. `{version}` is an internal variable that controls the API's versioning. `{query_method}` can be `query` or `queryAll`. `{query}` represents an SQL-like query that is composed in the following way: `SELECT @fields FROM @table Where @key >= @date_filter ORDER BY @key ASC NULLS FIRST` `@fields` represents the list of fields to be retrieved by each Collector Service. `@table` is each Collector Service. `@key` should be a date field and is used for filtering and ordering. `@date_filter` is internally controlled by the collector. Check the Running the data collector section for more information.	Represents an individual account, which is an organization or person involved with your business (such as customers, competitors, and partners).
Case	`crm.salesforceobjects.case`	`Case`		Represents a case, which is a customer issue or problem.
Content Version	`crm.salesforceobjects.contentversion`	`ContentVersion`		Represents a specific version of a document in Salesforce CRM Content or Salesforce Files.
Dashboard	`crm.salesforceobjects.dashboard`	`Dashboard`		Represents a dashboard, which shows data from custom reports as visual components.
Login History	`crm.salesforceobjects.loginhistory`	`LoginHistory`		Represents the login history for all successful and failed login attempts for organizations and enabled portals.
Opportunity	`crm.salesforceobjects.opportunity`	`Opportunity`		Represents an opportunity, which is a sale or pending deal.
Report	`crm.salesforceobjects.report`	`Report`		Represents a report, a set of data that meets certain criteria, displayed in an organized way.
User	`crm.salesforceobjects.users`	`User`		Represents a user in your organization.
Event Log Files	`crm.salesforceobjects.eventlogfiles` `crm.salesforce.<event_type>` if file processing is enabled	`EventLogFiles`		Represents event log files for event monitoring.

For more information, check the following sites:

Salesforce SObjects - API guidelines and documentation for Salesforce Objects. Learn more
Event types for EventLogFiles - List all the possible event types that are ingested in the crm.salesforce.<event_type> tables. All event_types will be in lowercase and without spaces. Learn more

Setup

The Salesforce collector works over the Salesforce API. You need to enable the API in your Salesforce account to allow the collector to get the data and make sure you have the required permissions.

Enable API access on the account

Log in to your Salesforce account.
Click the gear icon and select Service Setup.
Open the Users section in the left tree and select Users. In this section, search your user and click your user profile.
In the profile screen, click Edit.
Scroll down to Administrative Permissions and search for the API Enabled checkbox. Check it and save the changes.

Authentication

The Salesforce collector supports User/Password and OAuth2 authentication methods.

Getting credentials (User/password)

This method needs a username, password, and a security token.

Usually, all come together in the same mail when the user creates a Salesforce account. However, if the security token does not come in the mail or you can not find it, follow the next steps to obtain a new one:

Log in to your Salesforce account with the user that will be used to access the API.
Click the user icon at the top right of the screen and select Settings.
In the setup interface, go to the left tree and navigate to Users → Users.
In the left menu, go to the My personal information section and select Reset my security token.
On this screen, click Reset Security Token. You will receive a mail with the new security token.

Getting credentials (OAuth2)

To enable OAuth2 authentication, you need to create a Connected App and obtain the client_id, client_secret, and redirect_uri to use it.

You may implement OAuth2 in two different ways for this collector:

Using all the parameters (username, password, security_token, client_id, client_secret, and redirect_uri) to automatically authorize the OAuth2 client to access the information.
Authorizing the OAuth client manually, so you will only need the client_id, client_secret, and the access_token file obtained with the following script.

Log in to your Salesforce account.
Click the gear icon and select Service Setup.
Type Apps in the left search bar, then click App Manager → New Connected App.eso estaba pensando ja
Fill all the mandatory fields, check the Enable OAuth settings option and fill the new fields as shown in the following capture:
Save the changes and note down the client_id (Consumer Key), client_secret (Consumer Secret), and redirect_uri (Callback URL)

Getting the access token file

To get the access token file, we need to execute this script and follow these steps.

In order to execute the access_token file generation script, you need the following Python 3.6 (or above) dependencies installed:

click 7.1.2 (or above)
requests 2.25.1 (or above)
JSON (normally comes by default in the python standard library)
urllib (normally comes by default in the python standard library)
webbrowser (normally comes by default in the python standard library)

Execute the script using the following command:
python salesforce_access_token_generator.py --client_id <salesforce_client_id> --client_secret <salesforce_client_secret> --redirect_uri <salesforce_client_uri>
This script will open your predefined browser to log in to Salesforce and authorize the client in your Salesforce domain, and will redirect to another URL like in the following image:
.
Now, we need to copy the URL and paste it into the terminal where we have executed the script, which was waiting for this URL. Press ENTER and the script will finalize the token file creation. Once finalized the execution, we should see an access_token.json file in the same directory as the script.
Next, we need to convert the content of this file into base64 format. This can be done using an online tool such as Base64 Encode and Decode.
The base64 encoded string is then used as the value of the initial_access_token_base64 key in the collector's user configuration.

Permissions

In order to retrieve the data, we need to have some minimal permissions in the user configured for this collector. This user must be under a “Salesforce” user license and we need to provide the “standard user” profile for almost all the services/SObjects (with some exceptions) after enabling the API access.

The special permissions required for each SObject are specified below:

LoginHistory and Users	We can use a profile inherited from the standard user. To add the missing permissions, we just need to enable the Manage Users permission in the user’s profile.
EventLogFiles	We can use a profile inherited from the standard user. To add the missing permissions, we just need to enable the View Event Log Files permission in the user’s profile.

Devo collector features

Feature	Details

Feature	Details
Allow parallel downloading (`multipod`)	`Not allowed`
Running Environments	`Collector Server`, `On-Premise`

Minimum configuration required for basic pulling

Although this collector supports advanced configuration, the fields required to download data with basic configuration are defined below.

Setting	Details

Setting	Details
`token_value`	Set up here your access token created in the GitHub console.
`<salesforce_user_if_using_user_and_password>`	Username to authenticate the service, if using the User/Password authentication mechanism.
`<salesforce_password_if_using_user_and_password>`	Password to authenticate the service, if using the User/Password authentication mechanism.
`<salesforce_security_token_if_using_user_and_password>`	Security token to authenticate the service, if using the User/Password authentication mechanism.
`<client_id_if_using_oauth>`	Client ID to authenticate the service, if using the OAuth 2 authentication mechanism.
`<client_secret_if_using_oauth>`	Client Secret to authenticate the service, if using the OAuth 2 authentication mechanism.
`<base_64_encoded_content_from_acess_token_file_if_using_oauth>`	The result obtained from the script described in the Getting the access token file section, if using the OAuth 2 authentication mechanism.

Accepted authentication methods

Depending on how did you obtain your credentials, you will have to either fill or delete the following properties on the JSON credentials configuration block.

Authentication method	Username	Password	Security token	Client ID	Client Secret	Salesforce domain	Initial Access Token in Base 64

Authentication method	Username	Password	Security token	Client ID	Client Secret	Salesforce domain	Initial Access Token in Base 64
Username/Password	REQUIRED	REQUIRED	REQUIRED			REQUIRED
OAuth 2 with all the parameters (without `access_token`)	REQUIRED	REQUIRED	REQUIRED	REQUIRED	REQUIRED	REQUIRED
OAuth 2 with the `access_token` file manually generated				REQUIRED	REQUIRED	REQUIRED	REQUIRED

Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).