C3 AI Documentation Home

Manage GCP from User Applications

To use Google Cloud Pub/Sub from a C3 AI application, give your Google Cloud credentials the role they need. You can then create, get, and delete topics and subscriptions, and send and receive messages.

Prerequisite

To use Google Cloud Platform (GCP) Pub/Sub from your application, give the pubsub.admin role to the GCP credentials you use in the C3 server. This role grants full permissions to manage Pub/Sub resources. You can create, update, and delete topics and subscriptions, and manage permissions for other users. With this role, you manage the whole lifecycle of your Pub/Sub resources. Your application can then publish messages to topics and subscribe to them without permission errors.

Understand topics and subscriptions in Google Cloud Pub/Sub

Each Google Cloud Pub/Sub subscription subscribes to exactly one topic, which keeps the flow of messages clear. Google groups both topics and subscriptions by project, so their IDs must be unique within a project. This structure prevents conflicts between resources.

GCP Pub/Sub service API operations

The GCP Pub/Sub service API manages topics and subscriptions through three operations: Create/Update, Get, and Delete.

Create/Update

To create or update a topic and subscription, call the upsertResource() method:

Java
GcpTopic.make().withTopicId("test").upsertResource()
GcpSubscription.make().withTopicId("test").withSubscriptionId("test").upsertResource()

Get

To retrieve existing resources, call the getResource() method:

Java
GcpTopic.make().withTopicId("test").getResource()
GcpSubscription.make().withTopicId("test").withSubscriptionId("test").getResource()

Delete

To delete a topic or subscription, call the destroyResource() method:

Java
GcpTopic.make().withTopicId("test").destroyResource()
GcpSubscription.make().withTopicId("test").withSubscriptionId("test").destroyResource()

For all operations, identify the resource by topicId or subscriptionId together with projectId. You can also specify the resource by its resourceName, which uses this format:

Java
projects/{projectID}/topics/{topicID} // for GCP Topic 
projects/{projectID}/subscriptions/{subscriptionID} // for GCP Subscription

Test sending and receiving messages

The following test verifies that you can send messages to a GCP Pub/Sub topic and receive them from its subscription. Each message carries its content in the data field.

Java
   private Array<GcpTopicMessage> buildTopicMessages() {
      ArrayBuilder<GcpTopicMessage> messageArrayBuilder = GcpTopicMessage.arrayBuilder();
      for (int i = 0; i < 5; i++) {
         messageArrayBuilder.add(GcpTopicMessage.make().withId("_" + i).withData(ByteBuffer.wrap("{}".getBytes())));
      }
      return messageArrayBuilder.build();
   }

   @Test
   public void testSendAndReceiveMessages() {
      asTestAdmin(() -> {
         Array<GcpTopicMessage> topicMessages = buildTopicMessages();
         Map<String, Error> map = gcpTopic.sendMessages(topicMessages);
         assertEquals(0, map.size());
         assertEventualTrue(null, () -> {
            Array<GcpSubscriptionMessage> gcpSubscriptionMessages = gcpSubscription.receiveMessages();
            return gcpSubscriptionMessages.size() == topicMessages.size();
         }, 15, 15 * 1000);
      });
   }

See also

Was this page helpful?