Document toolboxDocument toolbox

Add a Step to Transform Data

Owned by Jonas Gonzalez
Nov 29, 2024
3 min read
[ 1 What You'll Learn ] [ 2 How to Transform Data ] [ 2.1 Example of Using 'Extract' to Transform Data ] [ 3 Related ]

Playbooks are workflows that can be automated. In these workflows, there are two situations where you would want to 'Transform Data'. The first one is when you need to trigger actions in response to a specific event or a data point. However, sometimes identifying the specific data point is not straightforward, you will have to transform the input data like a filter, convert formats, join, and so on. The second situation is when you want to send data out. Imagine you have a playbook set to do a lookup of the source IP addresses. You may want to take the results and send that information as an email. You can do most kinds of such simple and advanced data transformations in the playbook.

What You'll Learn

  • How to transform or manipulate output data of steps?

How to Transform Data

  1. Hover over the step whose output data you want to transform. Click + button.

  2. Open Transform Data under Browse All in the step catalog. The following options are available.

 

 

 

 

Extract

Extract a single field or multiple fields into a new column from JSON.

Available Steps: Extract JSON fields, JSON to Columns, JSON to Table, Split Array

 

Filter

Filter rows from the input data using boolean expressions and filter columns directly by name.

Available Steps: Filter, Drop Columns, Select

 

Add New Field

Add a new field/column to the input data. The new field's value can be expressed using Spark Functions. Learn more about Spark functions.

Parse

 

Join

Combine outputs of one or more steps in many different ways.

Lookup

 

Combine Columns

Combine multiple individual columns to form one single JSON column using the steps available under Combine Columns

Format

Convert between different formats of data like HTML to JSON, JSON list to Array, Table to HTML, JSON to Table and JSON to Markup Jira Text.

Group by

Group the same values of a column together. Aggregate the rest of the column values for each grouped row using a function like Count, Average etc

SQL Functions

Add a new field/column to the input data. The new field's value can be expressed using Spark Functions. Learn more about Spark functions.

Other

 

Example of Using 'Extract' to Transform Data

Extract JSON fields allow you to extract fields from a JSON column in the table.

  1. In the Playbook builder, click + on any step to extract the JSON fields.

  2. Search for extract JSON fields and select the Extract JSON Fields step from the results.

  3. Suppose you want to extract the specific column names from the table, search or enter a new field name that you want to extract as JSON fields in the Fields to Output.

  4. After you select the fields, the values and paths will be displayed.

  5. Drag the fields inside the table to rearrange and also click X to delete a field.

  6. When you have completed building the table by listing the output fields, click Run.

  7. On the input/output data menu bar:

  8. Click on any field name and select Extract JSON Field; this will open up the extract to JSON field form.

  9. In the search field, enter the name of the field to search by field name or click + to add a new field.

Related