Add and Configure a Transform
Once the source schema is successfully configured, a Transform node is automatically added to the canvas. This node represents the next stage in the pipeline, where you define how the incoming data should be processed. In the Transform step, you configure rules such as parsing, normalization, filtering, or reshaping the data. These transformations ensure that the raw input files are structured and cleaned appropriately before they move into downstream validation and ingestion workflows.
Begin by adding a transform to the source whose data you want to reshape or load into a new structure.
Choose a Transform Type
You will be prompted to select between:
Projection (Low‑Code)
Use this when you need:
- Simple field mappings
- Renaming or retyping fields
- Light transformations without custom code
Ideal for fast, declarative shaping of data.
Transformer (High‑Code)
Use this when you need:
- Complex logic or multi‑step transformations
- Custom functions or advanced validation
- Output that differs significantly from the input schema
Ideal for advanced or conditional processing.
Configure the Transform
Provide a name and review the incoming fields listed on the left panel that the transform will operate on.
Choose whether the transform writes to an existing structure or a new one.
Select a Target for the Transform
The middle panel displays all available target types that the transform can write to. These represent the destination data structures within the application model.
Each row corresponds to a target type and includes:
- Target Name – The Canonical or Entity type that will receive the transformed data.
- Description – A brief explanation of the type’s purpose and how it is typically used.
- Details – Opens additional metadata about the selected type, including field definitions.
- Select – Assigns the chosen type as the destination for the transform.
Understanding Target Types
You may see two broad categories of targets:
Canonical Types
Canonical types are intermediate data representations used for reshaping, validation, or staging before writing to a final entity. They are commonly used when:
- Additional transformation logic is required.
- Validation rules need to be applied.
- Data must conform to a standardized integration format.
Selecting a Canonical type means the pipeline may continue through additional transformation or validation steps before final ingestion.
Entity Types
Entity types represent persistent objects in the application’s data model. Selecting an Entity allows the pipeline to write data directly into the application model without additional canonical staging.
This is typically used when:
- The source data already matches the target structure.
- Minimal reshaping is required.
- Direct ingestion is sufficient.
Search and Filtering
Use the search bar to quickly locate a specific type by name. This is especially helpful in environments with a large application model where hundreds of types may be available.
Pagination
If many target types exist, the list is paginated. You can navigate between pages to browse all available targets.
Create a New Target
If an appropriate target does not exist:
- Click Create new target to define a new Canonical or Entity type.
This allows you to tailor the data model to your ingestion needs before mapping fields.
By clearly selecting the appropriate target type, you define where and how transformed data will be written within the application model. This step determines whether the pipeline writes directly to persistent entities or flows through additional canonical processing layers.
If creating a new target, you will be prompted to choose its type:
Canonical
Use this when:
- You need a flexible, reusable structure
- The data is semi‑structured or evolving
- You plan to feed multiple downstream processes
Entity
Use this when:
- You are loading into a well‑defined business object
- You want schema enforcement and validation
- The structure is stable and part of your data model
Save and Validate
Once created, the new target type is automatically selected and becomes available for field mapping in the transform.
This step establishes the destination schema for the transform and determines how the transformed data is materialized in the application model.