Monitor the Unstructured Data Integration (UDI) Job
After the Unstructured Data Integration (UDI) pipeline is fully configured, the system automatically creates a batch job to run the end-to-end data ingestion and embedding process.
During Execution
- The Source System SourceCollection fetches files.
- The Document Parser processes and chunks text.
- The Embedder converts text to embeddings.
- The Vector Store stores embeddings linked to the Entity.
The final output is a populated Entity with searchable vector embeddings ready for retrieval and analysis.
You can monitor these job runs from Monitor → Jobs → Batch in the C3 AI Studio sidebar.
Viewing Job Status
The Batch Jobs page displays key details for each UDI job run:
- Run ID – Unique identifier for the job execution.
- Status – Indicates whether the job is Running, Completed, or Failed.
- Job Type – Displays the underlying job class (for example,
HierarchyDenormBatchJob). - Start Time and Elapsed Time – Show when the job began and how long it took to complete.
Understanding Job Execution
Each UDI pipeline runs as an independent batch job that:
- Reads files from the configured Source System.
- Parses and extracts metadata using the Document Parser and Metadata Extractor.
- Generates embeddings through the Embedder.
- Stores results in the Vector Store and linked Entity.
You can filter job runs by status, type, or date, and use Auto-Refresh to update the job list periodically.
Tip:
If a job fails, review system logs under Monitor → Logs, or verify credentials and configurations for the Source System and Embedder.