XML: Combining Documents and Retaining Other Elements

Document created by thanh_n88 Employee on May 3, 2018Last modified by thanh_n88 Employee on May 17, 2018
Version 3Show Document
  • View in full screen mode

Sometimes you need to combine multiple documents of the same structure by a certain element but you would like to keep other elements that are outside of the combining element. The current combine in the data process will remove all other elements except for what is inside of the combining element and its parent element(s) and it will repeat that combining element.

Combine Behavior

Here's a quick overview of what the combine will do. Let's say this is your current data structure, we'll call it StartData and you have multiple of it:

<DataA>

<Request>
  <CustomerID>ABC</CustomerID>

  <ProjectID>145</ProjectID>
  <Records>
      <RequestRecord>
        <Address>123 Boomi Drive</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>
    </Records>
</Request>

</DataA>

Here are the resulting data depending on what you combine by:

Combine by <Records>1Combine by <RequestRecord>2Combine by <Address>3

<DataA>

<Request>

  <Records>
      <RequestRecord>
        <Address>123 Boomi Drive</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>
  </Records>

  <Records>
      <RequestRecord>
        <Address>456 Boomi Lane</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>
  </Records>

  <Records>

     ....

  </Records>

  ...
</Request>

</DataA>

<DataA>

<Request>

  <Records>
      <RequestRecord>

        <Address>123 Boomi Drive</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>

      <RequestRecord>
        <Address>456 Boomi Lane</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>

      <RequestRecord>

        ...

      <RequestRecord>

     ...

</Request>

</DataA>

<DataA>

<Request>

  <Records>

    <RequestRecord>
        <Address>123 Boomi Drive</Address>

        <Address>456 Boomi Lane</Address>

        <Address>789 Boomi Street</Address>

        ....

    </RequestRecord>

  </Records>
</Request>

</DataA>

 

1 Combine by <Records>, it will be repeated and everything within will be present, the <Request> and <DataA> parent tags will remain, but CustomerID and ProjectID are gone

2 Combine by <RequestRecord>, the looping will start/end with <RequestRecord> instead of <Records>. If <Records> had any additional children elements, those children elements would no longer be present after combining based

3 Combine by <Address>, <Address> it will be looped and <PostalCode> and <Country> are removed

 

Retaining Other Elements Via Dynamic Process Property/Default Values

We will use the same StartData at the very beginning. Let's say you have five of those documents and you want all of the <Records> elements together but at the same time retain other information like CustomerID and ProjectID to look something like this (we'll call it ResultData):

<DataA>

<Request>
  <CustomerID>ABC</CustomerID> 

  <ProjectID>145</ProjectID>

  <Records>
      <RequestRecord>
        <Address>123 Boomi Drive</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>
  </Records>

  <Records>
      <RequestRecord>
        <Address>456 Boomi Lane</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>
  </Records>

  <Records>

     ....

  </Records>

  ...


</Request>

</DataA>

Before we get started, in the above example, let's say you get all of your documents from some Connector A using profile A. If your profile A has a lot fields like NetSuite connector, I would recommend mapping to a simpler profile B with just the elements you need. 

 

Important - make sure the combining element property in profile B has a "Max Occurs" of "Unbounded" and "Looping Option" of "Occurrence". Unbounded lets multiple of that element exist. Looping by occurrence lets repeats exist.

 

Here's what a sample of what the process could look like

Process A

 

1. Store the CustomerID and ProjectID values as a dynamic process property A and B respectively since there are just a few. You can do this via a set properties or in the map. Alternatively you can set it later as default values if it's always a known value

2. Map profile A to B to extract the information you want. In this example, profile B will produce StartData (here it is again):

<DataA>

<Request>
  <CustomerID>ABC</CustomerID>

  <ProjectID>145</ProjectID>
    <Records>
      <RequestRecord>
        <Address>123 Boomi Drive</Address>
        <PostalCode>800M1</PostalCode>
        <Country>US</Country>
      </RequestRecord>
    </Records>
</Request>

</DataA>

3. Use the data combine shape and combine on the desired element, <Records> for this example

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4. Now map Profile B to Profile B straight across and populate the other elements either via getting the dynamic process property set in 1. or use default values

 

Retaining Other Elements Via Doc Cache

Another common way to pull the data for other elements is to load it into a cache to be used in the map later. Here's what the process could look like (we'll call it Process B), version 1. Branch 1 gets all of the data from the connector and then branch 2 would populate the data. Alternatively, you can load the cache after mapping to profile B as long as you mapped whatever data you wanted to pull in profile B, in this case I would map to CustomerID and ProjectID so it is available later even though the combine will erase it. View version 2 for reference.

 

Process B v1

Process B v2

 

The majority of the steps would be for the same reason and purpose. The map now pulls data from the cache in the map and then you can point it to the right place. This is supposed to be for multiple documents that have the same values. The five documents in this example all have the same CustomerID and ProjectID value. If you have different values, so you will have to pick which ones you want to use. 

 

2 people found this helpful

Attachments

    Outcomes