C3 AI Documentation Home

Manage GCP from User Applications

This topic guides you to configure GCP Pub/Sub within a C3 AI setup, ensuring you have the right permissions. This topic also provides the basic guidelines on the structural requirements of topics and subscriptions.

Prerequisite

To successfully connect your application to Google Cloud Platform (GCP) and utilize the Pub/Sub service, it's crucial to ensure that the GCP credentials you use in the C3 server have the pubsub.admin role. The pubsub.admin role grants comprehensive permissions to manage Pub/Sub resources. This includes creating, updating, and deleting topics and subscriptions, and also managing permissions for other users. With this role, you can effectively oversee the lifecycle of your Pub/Sub resources, ensuring that your application can publish messages to topics and subscribe to them without encountering permission issues.

Understanding topics and subscriptions in Google Cloud Pub/Sub

When working with Google Cloud Pub/Sub, it is essential to understand the relationship between topics and subscriptions. Each subscription is designed to subscribe to exactly one topic, ensuring a clear and organized flow of messages. Additionally, both topics and subscriptions are grouped by Google project, which means that within the same project, their IDs must be unique. This structure helps maintain order and prevents conflicts, allowing for efficient message handling and processing.

GCP Pub/Sub service API operations

The GCP Pub/Sub service API provides a way to manage topics and subscriptions through three primary operations: Create/Update, Get, and Delete.

Create/Update

You can create or update a topic and subscription using the upsertResource() method

Java
GcpTopic.make().withTopicId("test").upsertResource()
GcpSubscription.make().withTopicId("test").withSubscriptionId("test").upsertResource()

Get

To retrieve existing resources, the getResource() method is utilized.

Java
GcpTopic.make().withTopicId("test").getResource()
GcpSubscription.make().withTopicId("test").withSubscriptionId("test").getResource()

Delete

To remove resources, the destroyResource() method is employed. This allows you to delete a topic or subscription.

Java
GcpTopic.make().withTopicId("test").destroyResource()
GcpSubscription.make().withTopicId("test").withSubscriptionId("test").destroyResource()

For all operations, we recommend using the combination of topicId or subscriptionId along with the projectId to accurately identify the resource. Alternatively, you can specify the resource using its resourceName, which must follow this format:

Java
projects/{projectID}/topics/{topicID} // for GCP Topic 
projects/{projectID}/subscriptions/{subscriptionID} // for GCP Subscription

Test sending and receiving messages

This example demonstrates how to verify that messages can be successfully sent to and received from a GCP Pub/Sub topic, ensuring the messaging system works as expected.

Java
   private Array<GcpTopicMessage> buildTopicMessages() {
      ArrayBuilder<GcpTopicMessage> messageArrayBuilder = GcpTopicMessage.arrayBuilder();
      for (int i = 0; i < 5; i++) {
         messageArrayBuilder.add(GcpTopicMessage.make().withId("_" + i).withBody("{}"));
      }
      return messageArrayBuilder.build();
   }

   @Test
   public void testSendAndReceiveMessages() {
      asTestAdmin(() -> {
         Array<GcpTopicMessage> topicMessages = buildTopicMessages();
         Map<String, Error> map = gcpTopic.sendMessages(topicMessages);
         assertEquals(0, map.size());
         assertEventualTrue(null, () -> {
            Array<GcpSubscriptionMessage> gcpSubscriptionMessages = gcpSubscription.receiveMessages();
            return gcpSubscriptionMessages.size() == topicMessages.size();
         }, 15, 15 * 1000);
      });
   }

See also

Was this page helpful?