Business Data Extraction

FlowOn CI/CD Tools can extract and version business data (reference data, lookup tables) for deployment across environments.

Data Specification File

The data specification file defines which entities and columns to extract:

Structure

<?xml version="1.0" encoding="utf-16"?>
<DataSpecs>
  <Entities>
    <Entity logicalName="[EntityLogicalName]">
      <Columns>
        <Column name="[ColumnName]" />
        <Column name="[LookupColumn]" importBehavior="CreateIfNotExists" />
      </Columns>
    </Entity>
  </Entities>
</DataSpecs>

Import Behaviors

Import behaviors control how the tool handles lookup references during data import. When importing records that reference other records (lookups), the target record may or may not exist in the destination environment.

Behavior	Description	Use Case
Default	Standard upsert: Update if exists, create if not	General purpose, most common
CreateIfNotExists	Only create the referenced record if it doesn't exist; if it exists, use the existing one	Reference data that should not be overwritten
SkipRecordImportIfNotExists	Skip importing the entire parent record if the referenced lookup doesn't exist	Records that are invalid without their dependencies
FailIfNotExists	Fail the import with an error if the referenced record doesn't exist	Critical dependencies that must be present
SetToNullIfNotExists	Set the lookup field to null if the referenced record doesn't exist	Optional relationships that can be cleared

Import Behavior Details

Default

<Column name="crm_category" />

The standard behavior. If the record exists, it will be updated. If it doesn't exist, it will be created. This is used when you want full synchronization of data.

Example: Importing service definitions where you want to ensure the target environment matches the source exactly.

CreateIfNotExists

<Column name="crm_assignedteam" importBehavior="CreateIfNotExists" />

Use this when the lookup references a record that should only be created if it's missing, but never updated if it already exists. This protects existing data in the target environment.

Example: A task references a team. If the team exists in the target environment (perhaps with different settings), keep the existing team. If it doesn't exist, create it from the source data.

SkipRecordImportIfNotExists

<Column name="crm_parentcase" importBehavior="SkipRecordImportIfNotExists" />

Use this when a record is meaningless without its parent or dependency. If the referenced record doesn't exist, the entire record being imported is skipped (not just the field).

Example: Importing case comments that reference a parent case. If the parent case doesn't exist in the target environment, there's no point importing the comment—skip the entire comment record.

FailIfNotExists

<Column name="crm_workflowdefinition" importBehavior="FailIfNotExists" />

Use this for critical dependencies where the import should stop if a required reference is missing. This ensures data integrity by preventing partial imports.

Example: Importing task actions that must reference a specific workflow definition. If the workflow definition is missing, the import should fail rather than create orphaned or invalid data.

SetToNullIfNotExists

<Column name="crm_optionalapprover" importBehavior="SetToNullIfNotExists" />

Use this for optional lookup fields where it's acceptable to clear the reference if the target record doesn't exist. The parent record is still imported, but with a null value in this field.

Example: A document template has an optional "Reviewed By" user reference. If that user doesn't exist in the target environment, import the template anyway with the reviewer field set to null.

Import Behavior Decision Matrix

┌─────────────────────────────────────────────────────────────────────┐
│                    IMPORT BEHAVIOR DECISION TREE                     │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  Is the lookup reference found in the target environment?           │
│                                                                      │
│                    ┌─────────┴─────────┐                            │
│                   YES                  NO                            │
│                    │                    │                            │
│                    ▼                    ▼                            │
│            Use existing         Check importBehavior                 │
│            record reference            │                             │
│                              ┌─────────┴─────────────┐              │
│                              │                       │               │
│                    ┌─────────┼─────────┬────────────┤               │
│                    ▼         ▼         ▼            ▼               │
│               Default/   SkipRecord  Fail      SetToNull            │
│               CreateIf   IfNotExists IfNot     IfNotExists          │
│               NotExists              Exists                          │
│                    │         │         │            │                │
│                    ▼         ▼         ▼            ▼                │
│               Create     Skip the   Throw      Set field            │
│               the        entire     error,     to NULL,             │
│               record     parent     stop       continue             │
│                          record     import     import               │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Practical Example

<?xml version="1.0" encoding="utf-16"?>
<DataSpecs>
  <Entities>
    <!-- Workflow Task Definitions - core reference data -->
    <Entity logicalName="crm_workflowtaskdefinition">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_tasktype" />
        <Column name="crm_sladuration" />
        <!-- Default team should exist, fail if not -->
        <Column name="crm_defaultteam" importBehavior="FailIfNotExists" />
      </Columns>
    </Entity>

    <!-- Task Actions - depend on task definitions -->
    <Entity logicalName="crm_taskaction">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_actiontype" />
        <!-- Skip action if parent definition doesn't exist -->
        <Column name="crm_taskdefinition" importBehavior="SkipRecordImportIfNotExists" />
        <!-- Create notification template if missing -->
        <Column name="crm_notificationtemplate" importBehavior="CreateIfNotExists" />
      </Columns>
    </Entity>

    <!-- Document Templates -->
    <Entity logicalName="crm_documenttemplate">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_content" />
        <!-- Category must exist -->
        <Column name="crm_category" importBehavior="FailIfNotExists" />
        <!-- Reviewer is optional, clear if user doesn't exist -->
        <Column name="crm_reviewedby" importBehavior="SetToNullIfNotExists" />
        <!-- Owner team - create if doesn't exist -->
        <Column name="crm_owningteam" importBehavior="CreateIfNotExists" />
      </Columns>
    </Entity>
  </Entities>
</DataSpecs>

Complete Data Specification Example

<?xml version="1.0" encoding="utf-16"?>
<DataSpecs>
  <Entities>
    <!-- Countries Reference Data -->
    <Entity logicalName="crm_country">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_localizedname" />
        <Column name="crm_isocode" />
        <Column name="crm_isactive" />
      </Columns>
    </Entity>

    <!-- Document Types -->
    <Entity logicalName="crm_documenttype">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_description" />
        <Column name="crm_maxfilesize" />
        <Column name="crm_allowedextensions" />
        <Column name="crm_isrequired" />
        <Column name="crm_category" importBehavior="CreateIfNotExists" />
      </Columns>
    </Entity>

    <!-- Service Definitions -->
    <Entity logicalName="crm_servicedefinition">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_servicecode" />
        <Column name="crm_description" />
        <Column name="crm_category" importBehavior="CreateIfNotExists" />
        <Column name="crm_servicetype" importBehavior="CreateIfNotExists" />
        <Column name="crm_feeamount" />
        <Column name="crm_isactive" />
        <Column name="crm_sladuration" />
      </Columns>
    </Entity>

    <!-- Notification Templates -->
    <Entity logicalName="crm_notificationtemplate">
      <Columns>
        <Column name="crm_name" />
        <Column name="crm_templatekey" />
        <Column name="crm_subject" />
        <Column name="crm_body" />
        <Column name="crm_channel" />
        <Column name="crm_language" />
      </Columns>
    </Entity>
  </Entities>
</DataSpecs>

Extract Data Command

flowon export \
  --include-data \
  --data-spec "./specs/data-spec.xml" \
  --output "./artifacts/data"

Extracted Data Format

The extracted data is stored as XML files, one per entity:

<?xml version="1.0" encoding="utf-16"?>
<Data entity="crm_country">
  <Records>
    <Record id="a1b2c3d4-e5f6-7890-abcd-000000000001">
      <crm_name>United States</crm_name>
      <crm_localizedname>الولايات المتحدة</crm_localizedname>
      <crm_isocode>US</crm_isocode>
      <crm_isactive>true</crm_isactive>
    </Record>
    <Record id="a1b2c3d4-e5f6-7890-abcd-000000000002">
      <crm_name>United Kingdom</crm_name>
      <crm_localizedname>المملكة المتحدة</crm_localizedname>
      <crm_isocode>GB</crm_isocode>
      <crm_isactive>true</crm_isactive>
    </Record>
  </Records>
</Data>

Data Specification File​

Structure​

Import Behaviors​

Import Behavior Details​

Default​

CreateIfNotExists​

SkipRecordImportIfNotExists​

FailIfNotExists​

SetToNullIfNotExists​

Import Behavior Decision Matrix​

Practical Example​

Complete Data Specification Example​

Extract Data Command​

Extracted Data Format​