Specify a KV Store

Every entity Type stores its data in a datastore. The C3 Agentic AI Platform provides two kinds: a SQL store, which uses Java Database Connectivity (JDBC), and a key-value (KV) store. You choose the datastore for a Type with the datastore parameter of the Ann.Db annotation. On this page, you check the default databases for your environment, choose a store, and declare a Type for it. Administrators manage the store configurations and datastore settings that some options need.

Copy link to this section

SQL store and KV store

The SQL store is a relational database. The SQL store is also the default: a Type without a datastore value stores its data there.

A KV store saves each record under a key. A KV Type usually mixes Partitionable and names two of its fields in the @db annotation:

• partitionKeyField: the field that groups records into partitions. A Type that mixes Partitionable must set it.
• persistenceOrder: the field that orders records within a partition.

A KV Type without a partition key supports filtering by id only.

Copy link to this section

Default databases

The default database for each kind of store depends on your environment:

An SNE with a shared database is a single node environment whose environment settings select a shared database instead of H2. Like an SE, an SNE with a shared database uses PostgreSQL for the SQL store and Cassandra for the KV store. An SNE can also be set up with PostgreSQL or Cassandra on its own. An SNE uses one database for each environment, so all applications in that environment use the same database. An SE uses one database for each application.

To see which KV store your environment uses, run the following code in the C3 AI Console. The result is a datastore object of Type Cassandra or SqlKvStore.

KvStore.forName('kv')

To learn more about each database, see PostgreSQL Database and Cassandra Database.

Copy link to this section

Choose a store

Set the datastore value in the @db annotation to one of the following options:

The platform matches the datastore value to a KV store configuration with the same name. A value selects a KV store only when it is kv, fs, a name that starts with fs-, or the name of a KV store Type in any letter case, such as cassandra or dynamodb. The platform treats any other value, such as mykv, as the name of a SQL (JDBC) store.

The platform provides the kv and fs configurations. A user with the C3.ClusterAdmin role creates the configuration for any other value. For the method and its requirements, see Configure a datastore. If no configuration with that name exists, the Type uses the default KV store for your environment.

Copy link to this section

Use the default KV store

To store a Type's data in the default KV store for your environment, add the @db annotation with datastore="kv":

@db(datastore="kv", partitionKeyField='pk', persistenceOrder='timestamp')
entity type MyMeasurement mixes Partitionable<string> {
  pk: string
  timestamp: datetime
}

In a default SNE, the Type stores its data in H2. In an SE or an SNE with a shared database, the Type stores its data in Cassandra.

Copy link to this section

Use a named KV store

To store a Type's data in a specific KV store, set datastore to the name of that store's configuration. The name must follow the rule in Choose a store. For example, to use DynamoDB:

@db(datastore="dynamodb", partitionKeyField='pk', persistenceOrder='timestamp')
entity type MyMeasurement mixes Partitionable<string> {
  pk: string
  timestamp: datetime
}

Use the same pattern with datastore="cassandra", datastore="sqlkvstore", or datastore="KvStore.BigTable". Each value needs a store configuration with the same name. Without a matching configuration, the Type uses the default KV store for your environment.

Copy link to this section

Use the file system KV store

The file system KV store keeps data as files in your application's default file system. The store puts each Type's data in its own folder, with one entry for each record.

To use the file system KV store, add the @db annotation with datastore="fs". For example, the platform Type User.Event.Login declares the file system store:

@db(datastore="fs",
    partitionKeyField='userId',
    persistenceOrder='date')
entity type User.Event.Login mixes User.Event, Partitionable<string>, Configurable<User.Event.Login.Config>

To store the files under a different root folder, a user with the C3.ClusterAdmin role sets rootUrl in KvStore.FileSystem.Config. The URL must be fully qualified and include the file system scheme. Also add the mount for that URL with FileSystem#setMount. Data already written under the old root folder stays there. To learn more about file systems, see the FileSystem and File Type documentation.

Copy link to this section

Configure a datastore

This section is for administrators. Datastore configuration includes the store configurations that named KV stores need, credentials, and non-secret settings. For example, perTenant is true when each environment has its own Cassandra keyspace.

Copy link to this section

Roles for datastore configuration

The role you need depends on the task and the configuration level:

• Create a store configuration: Requires the C3.ClusterAdmin role, for every store type.
• Read credentials and other secret values: Requires the C3.ClusterAdmin role.
• Set credentials: Use the setCredentials method of the datastore object. The role depends on the level: APP needs no administrator role, ENV needs the C3.EnvAdmin role, and CLUSTER needs the C3.ClusterAdmin role.
• Read or change non-secret settings: An application administrator (C3.AppAdmin) can read non-secret settings and change them at the APP level. Changes at the ENV level need the C3.EnvAdmin role, and changes at the CLUSTER level need the C3.ClusterAdmin role.

After a user with the C3.ClusterAdmin role creates or changes a store configuration, restart the nodes or the applications that use it. The platform caches each store lookup by name, so it uses the new configuration only after the restart. Developers then select the configuration with @db(datastore="<name>"), where <name> follows the rule in Choose a store.

Copy link to this section

Store configuration for a named KV store

A store configuration has a name that equals the datastore value in the @db annotation. The configuration's kvstore field names the store Type, such as Cassandra, SqlKvStore, DynamoDB, or KvStore.BigTable. The platform ignores a configuration that does not set kvstore. DynamoDB uses the DynamoDBConfig Type, and Bigtable uses the BigTableConfig Type.

To create a store configuration, call the public Config#setConfigValue method on a new instance of the store's configuration Type, with the new name. This call has two requirements:

• The caller has the C3.ClusterAdmin role.
• After the call, you restart the nodes or the applications that use the configuration.

After the configuration exists, set its credentials: get the datastore object with KvStore.forName('<name>'), then call its setCredentials method with the credentials and the level. The setCredentials method changes only an existing configuration.

Copy link to this section

Read and change non-secret settings

Get the datastore object, then read its settings. The following code reads the perTenant setting of the default KV store. To read the default SQL store, use JdbcStore.forName('sql').

var datastore = KvStore.forName('kv');
datastore.isPerTenant();
datastore.config().perTenant;

To change a non-secret setting, use the setDatastoreConfigValue method with the path, the value, and the level. For example, setting external to true stops the platform from making schema changes on that datastore:

datastore.setDatastoreConfigValue("external", true, ConfigOverride.APP);

To undo the change, set the value back with the same method. The setDatastoreConfigValue method does not accept the credentials path; use setCredentials for credentials.

Copy link to this section

See also

• Database Indexes
• Create, Update, and Delete Data from a Database Using JavaScript
• Create, Update, and Delete Data From the Database Using Python
• Perform Database Maintenance to Reclaim Disk Space
• BatchExportSpec and BatchImportSpec Type documentation, to export and import a Type's data