Document AI REST API Developer Guide

You have files that must become searchable, retrievable content in your application. Submit them to the Document AI REST API, and it parses, chunks, and/or embeds each one according to a strategy you choose, then reports progress until every file lands or fails. For what Document AI is and when to use it, see the overview linked at the end of this guide.

The Document AI application's base URL isn't a fixed path; it depends on where Document AI is deployed on your cluster. Look it up from the application's microservice configuration:

// From any application's Console
Microservice.Config.forName('DocAi')

// Example result
# {
#   "type": "Microservice.Config",
#   "name": "DocAi",
#   "serviceAppUrl": "https://<cluster-host>/<docai-app-id>",
#   ...
# }

Append /docai/v1 to the returned serviceAppUrl to build the API base path. The rest of this guide refers to that base path as <docai-base-url>, for example https://<cluster-host>/<docai-app-id>/docai/v1.

All endpoints require a valid C3 session (cookie or bearer token). You can call the API from your own application's Console, from a script, or from any HTTP client on the cluster. The DocumentAi.User role, granted automatically to authenticated users, covers all document processing endpoints; the administrative usage dashboard requires the DocumentAi.Admin role.

Copy link to this section

Prerequisites

Before you submit your first job, confirm the following requirements:

1. Your target application is running on C3 AI Platform 8.10 or later. Document AI requires presignedUrl support for file transport, which was introduced in platform 8.10. Earlier platform versions aren't supported.
2. Document AI is deployed on your cluster. It runs as a central application; look up its base URL as shown above with Microservice.Config.forName('DocAi'). If the base URL doesn't respond, ask an administrator to deploy it.
3. Your files are in storage Document AI can read. Use absolute storage URLs, such as azure://<container>/<path> or s3://<bucket>/<path>, or upload the files to your application's cloud file system and use the resulting absolute URL. Don't use app-relative c3fs:/// paths; Document AI can't read them, and the run finishes with empty results instead of an error.
4. You know your retriever ID. The retriever lives in your target application and receives the embedded passages. The conventional ID is default-pg. To list the retrievers in your application, fetch Genai.Retriever from its Console.
5. The retriever's embedding model matches Document AI's. By default the embedding model is the one set at Document AI deployment time; a single job can override it (see Override the models for one job). Either way, a retriever whose model or dimension count differs from the one the run embedded with accepts the submission but fails later, at persistence or search time.

Copy link to this section

Submit a processing job

Send a POST request to /process with the files, a strategy name, and your target details:

curl -X POST "<docai-base-url>/process" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d @process-request.json

{
  "source": { "files": ["azure://<container>/<path>/report.pdf"] },
  "strategyName": "passage_level",
  "retrieverId": "default-pg",
  "sourceApp": "<target-app-id>"
}

The request body accepts the following fields:

The response confirms what was accepted:

{
  "jobId": "<jobId>",
  "runs": [{ "runId": "<runId>", "file": "azure://<container>/<path>/report.pdf", "status": "PENDING" }],
  "rejected": []
}

Record the jobId and runId values; you use them for all monitoring calls. Document AI routes each accepted file to a small, medium, or large processing fleet by file size (by default: up to 50 MB, 200 MB, and 1 GiB). Files that exceed the size gate, 1 GiB by default, appear in rejected with a reason and aren't processed.

Copy link to this section

Override the models for one job

Every run uses the LLM and embedding models the Document AI deployment is configured with. To run one job on different models, add an llm block. Reasons to do this include comparing completion quality or cost, or embedding a corpus for a retriever built on another embedding model. Nothing about the deployment changes, and other jobs are unaffected.

{
  "source": { "files": ["azure://<container>/<path>/report.pdf"] },
  "strategyName": "passage_level",
  "sourceApp": "<target-app-id>",
  "llm": {
    "completionModel": "bedrock/us.anthropic.claude-sonnet-4-6",
    "embeddingModel": "azure/plat-cohere-embed-v4",
    "embeddingDimensions": 1024
  }
}

Each model is independently optional: overriding only completionModel leaves embedding on the deployment's model, and vice versa. Omitting llm, or sending {}, submits exactly the request you submitted before this field existed.

Document AI validates the block before creating any job or run, so a mistake costs you a 400 instead of a batch of failed runs. A submission is rejected when:

• a model name isn't one the cluster's LLM proxy serves. Register the exact route first; a provider wildcard such as anthropic/* doesn't count as serving a specific model;
• embeddingModel is set without embeddingDimensions, or embeddingDimensions without embeddingModel;
• embeddingDimensions isn't a positive integer;
• the two models are swapped: an embedding model in completionModel, or a completion model in embeddingModel;
• the block carries an unrecognized key. Unlike ingestionSettings, a misspelled key here is rejected, not ignored. A dropped model name runs the whole job, and bills it, on the deployment's model instead of yours.

Document AI records the models you asked for on the job as llmCompletionModel, llmEmbeddingModel, and llmEmbeddingDimensions. This lets you read a job's token usage and cost next to the models that produced them. A null field means the run used the deployment's model.

Copy link to this section

Monitor the job

Processing is asynchronous. Poll the job until every run reaches a terminal state:

curl "<docai-base-url>/jobs/<jobId>" \
  -H "Authorization: Bearer <token>"

The response includes the derived job status, a progress object with run counts keyed by lowercase status, the child runs with per-file errors, and aggregated token usage. Job status is derived from the runs:

For a single file, poll the run directly:

curl "<docai-base-url>/runs/<runId>" \
  -H "Authorization: Bearer <token>"

Each run moves through a fixed lifecycle:

Small files typically complete in under a minute, so start polling right away with a short interval and back off gradually. Medium and large files may wait briefly if that fleet is scaling up from zero after idle time; keep polling. A job status of FAILED doesn't mean every file failed: check progress for how many runs reached COMPLETED or PERSISTED (and how many are PERSIST_FAILED, meaning processed but not fully persisted), and report the failed files and their error messages alongside the successes.

To list all jobs, send a GET request to /jobs. The endpoint supports standard FetchSpec query parameters: filter (such as status == 'RUNNING'), order (such as descending(submittedAt)), limit, and offset.

Copy link to this section

Persist the results

Persistence writes a completed run's output into the target application: one Genai.Vector.SourcePassage record per chunk under the retriever, one Genai.SourceFile record for the document, and any extracted tags and entities.

If you set retrieverId at submission, persistence is automatic and each successful run ends in PERSISTED. You don't have to do anything else.

Persist manually when you submitted without a retrieverId, or when automatic persistence reported errors and you want to retry:

curl -X POST "<docai-base-url>/persist" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "runId": "<runId>", "retrieverId": "default-pg" }'

The response reports { "persisted": <count>, "errors": [] }. Persisting the same document into the same retriever again is safe: it replaces stale passages instead of duplicating them. A persistence failure never fails the run; the run stays COMPLETED and you can retry after fixing the cause.

Copy link to this section

Verify the results

After a run reaches PERSISTED, the passages are live in your target application. To confirm, fetch the newest source file records from your application's Console:

var files = Genai.SourceFile.fetch({
  limit: 10,
  order: 'descending(meta.created)',
}).objs;

Each ingested document appears as one source file record whose original file URL matches the file you submitted. The embedded passages are available to your application's retriever and to any agent or search feature built on it.

Copy link to this section

Configure tag and entity extraction

The ingestionSettings field controls what a strategy extracts while it processes. It accepts a documentExtractor block, which runs one time against the whole document, and a passageExtractor block, which runs against every chunk. Each block defines a tagsToExtract schema for descriptive tags and an entitiesToExtract schema for typed entities. Settings keys aren't validated at submission: an unrecognized key is logged and ignored, while a malformed value, such as a tag without a type, fails the run during processing.

For schema syntax, a worked example, and guidance on where the extracted data lands, see Configure Tag and Entity Extraction.

Copy link to this section

Use a custom strategy

When the built-in strategies don't fit, your application supplies its own: a single Python function passed inline on each /process request. There is no registration step and no redeployment—the service runs the code you send with the run.

Pass the function source as the script field on /process instead of strategyName (the two are mutually exclusive):

{
  "source": { "files": ["azure://<container>/<path>/report.pdf"] },
  "script": "def process_document(document, settings):\n    ...\n",
  "sourceApp": "<cluster>-<env>-<app>"
}

The script must define a top-level process_document(document, settings); a script without that signature is rejected at submission. The code ships inline with each run, so your application is the registry—version and store the source wherever you keep your application code.

For the authoring contract, the building blocks, and the write-test-iterate workflow, see Develop Custom Strategies.

Copy link to this section

Retrieve logs and track usage

Every run writes structured pipeline logs. Retrieve them per job or per run:

curl "<docai-base-url>/jobs/<jobId>/logs" \
  -H "Authorization: Bearer <token>"

curl -X POST "<docai-base-url>/runs/logs" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "runIds": ["<runId>"] }'

The job endpoint returns a flat list of log entries. The run endpoint returns an object keyed by run ID, with a missing array naming any run IDs that had no logs; check it before concluding a run produced none. Each entry carries a timestamp, level, and message. Pipeline stages appear as markers in the messages: prepare start:, parse start:, chunk start:, extract_from_chunks start:, extract_from_document start:, and embed start:. The most recent marker tells you which stage a run is in.

Every job and run also reports a usage object with promptTokens, completionTokens, totalTokens, and an estimated cost in US dollars. Job-level usage aggregates across all child runs, so you can attribute processing cost per batch.

Copy link to this section

Troubleshoot common failures

Match what you see against a symptom below for the likely cause and fix:

Copy link to this section

Endpoint reference

All paths are relative to <docai-base-url>.

Copy link to this section

See also

• Document AI Overview
• Quick Start - Process Your First Document
• Configure Tag and Entity Extraction
• Develop Custom Strategies