Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
minLevel1
maxLevel6
outlinefalse
typeflat
separatorbrackets
printablefalse

Currently, we have a columnsToJson operator will converts the columns to json object where the column names become key and values in the cell become values in the json. The drawback of this operator is that if you want to provide a constant field, e.g. version which should be replicated in each of the fields, you have to create a separate column in order to be able to see it in the output. Another limitation, if you want to create a nested object you have to call this operator twice, but the problem is when you call it second time it will treat the output of the first call as a string and so it might not be a json object. There are other limitations but I think is enough to create a new operator.

...

  • constant value: {"Version":"1.0.1"}, this will pass constant value "1.0.1" where Version is used

  • column name: {"Id": "$lhub_id$"} this will pass values in lhub_id column where Id is used

  • casting value: {"Description": "$details$ as json"}, details columns values will be cast to json and then pass to Description.

Examples

Input
inputTable

lhub_id

user

path

domain_info

1

emil

Downloads

gmail.com

2

kumar

backend

http://logichub.com

Example 1

LQL command

Code Block
toJson(input, "resultJson", "", {"A":"$user$", "B": "$path$"})

Output

lhub_id

user

path

domain_info

resultJson

1

emil

Downloads

gmail.com

{"A":"emil", "B":"Downloads"}

2

kumar

backend

http://logichub.com

{"A":"kumar", "B":"backend"}

  • Explanation *

When jsonfileOrTemplate is empty string (e.g. "" in the args), this command will create JSON from Map parameter = {"A":"$user$", "B": "$path$"}, where "A" and "B" are keys, and "$user$", "$path$" are the values of Map parameter, since value part contains "$.*$" pattern, string between dollar signs is assumed to be a column name, if the column name does not exist in the table it will throw an error.

...

"$lhub_id as int" mean cast values in lhub_id column to int
"$resultJson$ as json means cast resultJson column to json object
"#id#" is identifier, it can be anything e.g. "&tmp&", "blabla", "x$y$z" etc, but make sure that binding key is unique and does not exist in anywhere in the values of the JSON template that you are going to replace, and remember the variable name, because it will be replaced by the values in the column that is provided in the binding, in this examples #id# in the template will be replaced with cast(lhub_id as int) values, because it is provided in the binding, if we dont find it will be left as it is, e.g. output will be "#id#" for "id" column
Here is the output of the LQL command.

lhub_id

user

path

domain_info

resultJson

res

1

emil

Downloads

gmail.com

{"A":"emil", "B":"Downloads"}

{"id":1, json:{"A":"emil", "B":"Downloads"} }

2

kumar

backend

http://logichub.com

{"A":"kumar", "B":"backend"}

{"id":2, json:{"A":"kumar", "B":"backend"} }

Example 3 Passing template as a file
Save template as json file (testTemplate.json = {"id":"#id#", "json":"#jsonObject#"}), and store it inside /opt/docker/data/shared

...