Confirm Data Fusion Pipeline Run Completion
After running a data integration pipeline, verify that the ingestion process completed successfully and that all source files were processed without errors. The View Run Status feature provides operational visibility into the batch jobs and queues responsible for processing the source data. By reviewing this information, users can confirm that files were processed correctly, identify potential ingestion issues, and trace the execution of the underlying batch jobs. Run status is accessed from the Source Collection node because file ingestion and synchronization are managed at the collection level.
Steps to verify that the pipeline run completed successfully
- Open the Data Integration canvas and locate the relevant Source Collection node in the pipeline.
- Click the ellipsis menu (⋯) on the source collection node and select View run status.
- Review the File Sync Status to confirm the synchronization state and the last time files were processed.
- Inspect the queue metrics (BatchQueue, SourceQueue, and SourceStatusQueue) to verify that no jobs are pending or failed.
- Select View Batch Job to open the batch job details and confirm that the ingestion jobs completed successfully.
Preview the target type node on the Data Integration canvas to verify that records were ingested successfully.
Checking the run status ensures that the ingestion pipeline executed as expected and that all discovered files were processed without errors. It also provides visibility into the underlying execution infrastructure, queues and batch jobs, which helps diagnose issues such as stalled jobs, failed processing tasks, or incomplete file synchronization. Regularly reviewing the run status allows operators to detect problems early and maintain reliable data ingestion workflows.
Validate data output
After the pipeline completes successfully you can verify that data has been written to the configured target Type using the Object Model.
Using the Object Model
After configuring the transform and selecting a target entity, use the Object Model view to verify that the target entity is defined correctly and matches the fields expected by the transform.
- In C3 AI Studio, click Object Model in the top navigation bar.
- Use the search bar to locate the target entity (for example,
SampleEntity). - Click the entity on the canvas to open its details panel.
- Review the fields and data types defined for the entity.
- Optionally, click Preview Data to check whether records exist for the entity.
Why this is important
Validating the target entity in the Object Model ensures that:
- the entity structure matches the fields defined in the transform mapping
- the data types are compatible with the incoming data
- the target entity exists and is correctly configured
- data is successfully written to the entity after pipeline execution
This step is useful for confirming the target configuration and identifying potential issues early before running or troubleshooting the pipeline. Use downstream applications, queries, or previews to confirm correctness.
Key takeaway
Runtime parameters let you control how data is processed without changing the pipeline structure. By separating pipeline design from execution behavior, Data Fusion enables safe iteration, repeatable runs, and controlled reprocessing—all from the UI.
Notes and best practices
- Always preview source data and transforms before executing the pipeline.
- Use reprocessing options carefully, especially in production environments.
- Monitor execution metrics regularly to identify data quality or performance issues.
The completed pipeline—Source System → Source Collection → Schema → Transform → Target—appears fully connected on the canvas.