Monitor the Unstructured Data Integration (UDI) Job
Data Fusion is in Beta. Please contact your C3 AI representative to enable this feature.
After the Unstructured Data Integration (UDI) pipeline is fully configured, the system automatically creates a batch job to run the end-to-end data ingestion and embedding process.
During Execution
- The Source System SourceCollection fetches files.
- The Document Parser processes and chunks text.
- The Embedder converts text to embeddings.
- The Vector Store stores embeddings linked to the Entity.
The final output is a populated Entity with searchable vector embeddings ready for retrieval and analysis.
You can monitor these job runs from Monitor → Jobs → Batch in the C3 AI Studio sidebar.
Viewing Job Status
The Batch Jobs page displays key details for each UDI job run:
- Run ID – Unique identifier for the job execution.
- Status – Indicates whether the job is Running, Completed, or Failed.
- Job Type – Displays the underlying job class (for example,
HierarchyDenormBatchJob). - Start Time and Elapsed Time – Show when the job began and how long it took to complete.
Understanding Job Execution
Each UDI pipeline runs as an independent batch job that:
- Reads files from the configured Source System.
- Parses and extracts metadata using the Document Parser and Metadata Extractor.
- Generates embeddings through the Embedder.
- Stores results in the Vector Store and linked Entity.
You can filter job runs by status, type, or date, and use Auto-Refresh to update the job list periodically.
Tip:
If a job fails, review system logs under Monitor → Logs, or verify credentials and configurations for the Source System and Embedder.