C3 AI Documentation Home

Use Partial Indexes in Vector Store

A partial index on a vector field indexes only the rows that match a filter condition, instead of the whole table. Choose a partial index when your similarity searches always target a known subset of rows, such as one category or one source. The index stays smaller than a full vector index and covers only the rows those searches read.

The C3 Agentic AI Platform stores and indexes vector embeddings in PostgreSQL with pgvector, a PostgreSQL extension for vector similarity search. On this page, you declare a partial index on an entity Type, load data, check the index, and query with the index filter.

Prerequisites

Before you start, check that your application meets these requirements:

  • The application stores its entity Types in PostgreSQL. The platform creates vector indexes only in PostgreSQL.

  • The pgvector extension is installed in that database. To check the installed version, run the following command in the C3 AI Console of your application:

    JavaScript
    DbAdmin.inst().getPgVectorVersion()

    If pgvector is not installed, ask your database administrator to install it.

Declare the partial index on your Type

To create a partial index, add an index entry with a filter key to the @db annotation of an entity Type. The index covers only the rows for which the filter is true.

This example Type stores product documents with a five-dimension vector embedding in the emb field. Use your own Type, field, and filter values:

Type
@db(index = [{fields:[{name:emb, opClass:L2}], filter: "title == 'Pump manual'"}])
entity type VectorStorePartialIndex {
  /**
   * The id of the doc.
   */
  docid: string
  /**
   * The title of the doc.
   */
  title: string
  /**
   * The text of the doc.
   */
  text: string
  /**
   * The vector embedding.
   */
  @vector(dimension=5)
  emb: string
}

The index covers the emb field for rows where title == 'Pump manual', and uses the L2 (Euclidean distance) operator class. The entry sets no index parameters, so the platform creates a Hierarchical Navigable Small Worlds (HNSW) index with default settings. To choose the index type and its parameters, see the section Choose an index type in Vector Store.

Write the filter as a C3 filter expression that evaluates to true or false. The filter can use comparison and arithmetic operators, &&, ||, and most expression engine functions other than time series functions, such as startsWith(title, 'Pump').

Create multiple partial indexes

To index several subsets of the same field, add one index entry per filter:

Type
@db(index = [{fields:[{name:emb, opClass:L2}], filter: "title == 'Pump manual'"},
             {fields:[{name:emb, opClass:L2}], filter: "title == 'Valve manual'"}])

Load data

Sync or deploy the package that contains the Type. When the platform provisions the Type, it creates the table and one partial index for each index entry.

Then load data into the Type with any standard method, such as createBatch() or a data integration pipeline. Each emb value must be a vector string with as many numbers as the dimension of the field. To load the two example rows, run the following command in the C3 AI Console of your application:

JavaScript
VectorStorePartialIndex.createBatch([
  {docid: "1", title: "Pump manual", text: "Sample text", emb: "[0,1,1,1,1]"},
  {docid: "2", title: "Valve manual", text: "Another sample text", emb: "[1,0,1,0,1]"}
])

Check the partial index

To list the indexes on the table of the Type, call DbAdmin.inst().getTableIndexes() with the table name. Run the following command in the C3 AI Console of your application:

JavaScript
DbAdmin.inst().getTableIndexes(VectorStorePartialIndex.meta().rootC3TableName())

The result has one entry per index. For a partial index, indexType shows the index method, such as hnsw, and filterSql shows the filter condition.

Query with the partial index

Include the index filter condition in each query that should use the partial index. You can query in two ways:

  • Nearest-neighbor search: Choose it to find the rows closest to a query vector. Order by vectorDistance() and set a limit. Nearest-neighbor search is the query shape that uses the vector index.
  • Distance threshold: Choose it to find every row within a set distance of a query vector. Filter on vectorDistance(). This query shape does not use the vector index.

Nearest-neighbor search

This query returns the three rows with title Pump manual that are closest to the query vector, nearest first:

JavaScript
var dist = "vectorDistance('emb', '[0,1,1,1,2]', {'metric': 'L2'})";
VectorStorePartialIndex.evaluate({
  filter: "title == 'Pump manual'",
  projection: "id, text, " + dist,
  order: dist,
  limit: 3
});

Include the distance expression in the projection when you order by it. Use the metric that matches the operator class of the index.

The docid field in the example is a regular field that holds the document ID from your source data. The platform also gives each record its own id, which the query returns.

Distance threshold

This query returns the rows with title Pump manual whose distance from the query vector is less than 2:

JavaScript
VectorStorePartialIndex.fetch({filter: "vectorDistance('emb','[0,1,1,1,2]') < 2 && title == 'Pump manual'"});

See also

Was this page helpful?