C3 AI Documentation Home

Application-Level Encryption

The C3 Agentic AI Platform includes application-level encryption. The server is responsible for encrypting and decrypting column values. As sensitive data is written (using data loading), the C3 Agentic AI Platform can encrypt or decrypt the data using a well-known, secure algorithm. A benefit of transparent data encryption is that it is database independent and can encrypt data managed by Postgres and Cassandra.

Key terminology

  • Application-Level Encryption: A security measure where encryption and decryption processes are performed within the application, rather than at the database or disk level, securing sensitive data by converting it into a non-readable form for unauthorized users.

  • Encryption Algorithm: A set of mathematical procedures for performing encryption and decryption.

  • Transparent Data Encryption (TDE): An encryption method that occurs seamlessly during data read and write operations at the application level, without requiring changes to the application's code.

  • Symmetric Encryption: An encryption system in which a single key is used for both encryption and decryption of data.

  • PII Annotation: An annotation identifying Personally Identifiable Information (PII) that you should encrypt.

  • Encryption Key: Encryption algorithms use a sequence of bits to transform plain text into ciphertext and to reverse the process during decryption.

  • HashiCorp Vault: A tool for secrets management, encrypting sensitive data, and dynamic secrets.

  • C3 AI Vault: A proprietary vault system within the C3 Agentic AI Platform for managing encryption keys and sensitive data. For more information refer to the topic on Performing Operations on C3 AI Vault.

  • Master Encryption Key (MEK): A primary key you can use to secure other keys within a hierarchy of encryption keys, specifically within the C3 AI Vault.

  • Key Encryption Key (KEK): An encryption key you can use to encrypt other keys, such as a Data Encryption Key (DEK), for added security.

  • Data Encryption Key (DEK): A key you can use specifically to encrypt and decrypt data.

  • Vault: A secure storage mechanism for encryption keys, which can be either C3 AI Vault or Hashicorp Vault.

  • Java Keystore: A storage facility for cryptographic keys and certificates in Java.

  • Data Encryption Engine: A component of the C3 Agentic AI Platform responsible for managing the encryption and decryption operations.

  • SymmetricCipher: An object in the C3 Agentic AI Platform representing symmetric encryption mechanisms, used to set keys for encryption and hashing.

Possible user scenario

Bob, a Data Engineer at Bank Corp, must implement the C3 AI Anti-Money Laundering application.

One data source provides data about customer financial accounts, including the account numbers. Account numbers are considered sensitive information within banks. Bank Corp’s security policy mandates that the encryption of financial account numbers in any target data store (such as Postgres or Cassandra).

When configuring the application, Bob indicates in the application type definition the accountNumber field is sensitive and must be encrypted.

Encryption rules

  • If you indicate a field has sensitive data, that data is stored encrypted and then decrypted on fetch.
Type
entity type SmartBulb extends LightBulb mixes ODataSvc, MetricEvaluatable type key "SMRT_BLB"  { 
  ...
  @PII(encrypt = true)
  sensitiveField_1: string
  @PII(encrypt=true)
  sensitiveBalance: double
}

  • In the code snippet example above, the @PII annotation indicates the field has sensitive information. The encrypt attribute indicates the field should be encrypted on write and decrypted on read.

  • You do not need to specify the encryption method or key used for encryption. These parameters are the responsibility of the C3 Agentic AI Platform or C3 AI Operations.

  • Data identified as sensitive are stored in an encrypted manner in the database. Note that this is not disk-level encryption. If you perform a select on the underlying database column, encrypted results are returned (see example below).

  • Data identified as sensitive are decrypted when read and the unencrypted values are returned to the client.

  • The master encryption key must be registered by C3 AI Operations with the platform if you're using HashiCorp Vault. If the C3 AI Vault is used, the master encryption key is managed by the C3 Agentic AI Platform.

Limitations

Side effects of transparent data encryption are:

  • Application performance can be affected when sensitive data are read and or written.

  • Filtering on sensitive data is not supported and this causes an error. Alternative approaches are possible (such as creating another column that has a hash of a subset of the data) but requires additional application configuration.

  • Depending on the implementation, encrypting and decrypting data causes the database or application server to incur additional load.

  • Database storage can increase, since encrypted fields are stored as text columns.

  • Encrypted Foreign key references are not supported.

Key types

  • KEK – Key Encryption Key

  • DEK – Data Encryption Key

  • MEK – Master Encryption Key (relevant only when using C3 AI Vault)

Key management

  • KEK stores the Vault. The Vault is either a C3 AI Vault or a Hashicorp Vault.

  • The KEK is used to encrypt DEK, and this encrypted DEK can be stored in Postgres.

  • If you are using Hashicorp Vault, there is no concept of MEK.

  • If you are using C3 AI Vault, MEK is protecting the contents of the disk. KEK is in the disk, so MEK is protecting the KEK. MEK has to be seeded in memory using Kubernetes Secret or Java Keystore.

  • Integration with third-party KMS solutions is not supported.

High-level flow

  1. A user/action requests to access encrypted data to the Data Encryption Engine.
  2. On encrypt request: a. The Data Encryption Engine requests the DEK, and looks in memory for a DEK
    • If it exists, it fetches a DEK.
    • If it does not exist, it generates a new DEK. b. System uses the DEK to first encrypt the data, and then immediately encrypts the DEK using KEK (KEK is stored in the Config framework). c. System persists encrypted DEK in database so that any node can decrypt the data.
  3. On decrypt request: a. Node first reads the encrypted DEK from the database b. Reads KEK from the config c. Decrypts DEK using KEK d. Decrypts the data using DEK. e. The platform then encrypts or decrypts the requested data with the plaintext information

To enable data at rest encryption on C3 AI Application Platform, execute the following commands to set the KEK and key for hashing:

JavaScript
// Set key for hashing
SymmetricCipher.setKeyForHash(SecretKey.generateKey(), true) 

// set KEK 
SymmetricCipher.activate(SecretKey.generateKey())

Related content

Was this page helpful?