C3 AI Documentation Home

Google Cloud Storage Connector

The C3 Agentic AI Platform has a built-in connector for integrating with Google Cloud Storage. To integrate with a Google Cloud Storage bucket, you must:

  1. If a Google Cloud Storage bucket does not already exist, create one with the appropriate bucket access policies.
  2. If you do not have a mount path for the Google Cloud Storage bucket, create one in the C3 Agentic AI Platform.
  3. Set the required credentials to access the contents of the Google Cloud Storage bucket.
  4. Validate the connection.

This page also lists the minimum permissions for read-only access and shows how to clear credentials.

C3 AI Studio offers Data Fusion, a visual tool for building data pipelines. Data Fusion configures data sources and connects them to C3 Canonicals and Entities. For more information, refer to Data Fusion Overview.

Create a Google Cloud Storage bucket

Before connecting to Google Cloud Storage, create or use an existing bucket configured for uniform bucket-level access. With uniform bucket-level access, Identity and Access Management (IAM) alone controls access to every object in the bucket. For more information on creating a Google Cloud Storage bucket, see Create storage buckets in the Google Cloud documentation.

To access the bucket from the C3 Agentic AI Platform, you must also create a credential with the appropriate IAM roles. These roles let the platform read from and write to the Google Cloud Storage bucket. To create and edit IAM roles, see the Overview of access control and IAM permissions for Cloud Storage topics in the Google Cloud documentation.

Grant the minimum permissions for read-only access

To provide read-only access, assign the Storage Object Viewer role in the IAM policy. This role grants the minimum permissions needed to view and access objects stored in a Google Cloud Storage (GCS) bucket.

Configure the C3 Agentic AI Platform

After you create a bucket and apply the appropriate IAM policies, apply the following configurations on the C3 Agentic AI Platform.

Enable the file system

Enable the Google Cloud Storage bucket by running the following:

JavaScript
FileSystem.gcs().enable();

For more information on enabling remote file systems on the C3 Agentic AI Platform, see Work With File Systems.

Create the file system mount

Use the following to add a new file system mount path for the Google Cloud Storage bucket:

JavaScript
var mountName = "<mount_name>";
var bucketName = "<gcs_bucket_name>";

FileSystem.gcs().setMount(mountName, "gcs://" + bucketName, ConfigOverride.APP);

Set credentials to access the Google Cloud Storage bucket

The credential steps on this page need the C3.ClusterAdmin role.

The C3 Agentic AI Platform needs a key to access the contents of your Google Cloud Storage bucket. Before you set credentials, generate this key for a principal with the right access to the bucket, such as a Google Cloud Service Account. To create a key for a Service Account, see the Create and manage service account keys topic from the Google Cloud documentation.

After you have a .json file containing the key information, run the following commands to set the credential. The credential authorizes requests to the bucket on behalf of the Service Account principal:

The following code is included for explanatory purposes only. In a production scenario, store keys in an enterprise vault system designed for managing secrets.

JavaScript
var key = <contents_of_json>;
var accessKey = key.private_key_id;
var secretKey = key.private_key;

var credentials = Gcp.inst()
  .defaultCredentials()
  .withAccessKey(key.private_key_id)
  .withSecretKey(key.private_key)
  .withAccountId(key.client_id)
  .withAccountEmail(key.client_email);
GcpBucket.setCredentialsForResourceName(
  bucketName,
  credentials,
  ConfigOverride.APP
);

You can copy existing credentials from an existing Google Cloud Storage bucket to a new bucket using the following code:

JavaScript
var otherBucketName = "<name_of_the_existing_bucket>";
var bucketName = "<gcs_bucket_name>";

var credentials = GcpCredentials.make(
  GcpBucket.forResourceName(otherBucketName).credentials
);

GcpBucket.setCredentialsForResourceName(
  bucketName,
  credentials,
  ConfigOverride.APP
);

To reduce security risk, use a least-privileges approach when establishing IAM roles and setting credentials, especially in production environments.

Validate the connection

After you apply the required configurations, list the files in the bucket to confirm that the platform can reach it:

JavaScript
var mountName = "<mount_name>";

FileSystem.gcs().listFiles(FileSystem.gcs().urlFromMount(mountName));

This request succeeds if the configuration is correct, even if the bucket has no files.

Clear credentials

To clear credentials for a Google Cloud Storage bucket and delete the associated mount path, run the following:

JavaScript
var bucketName = "<gcs_bucket_name>";
var mountName = "<mount_name>";

var bucket = GcpBucket.forResourceName(bucketName);
bucket.clearConfigValue("credentials", ConfigOverride.APP);
bucket.clearSecretValue("credentials", ConfigOverride.APP);
FileSystem.gcs().removeMount(mountName, ConfigOverride.APP);

See also

Was this page helpful?