Data Integration Roles and Permissions

Every user who connects a data source, builds a pipeline, or monitors a pipeline run needs a role that allows those actions. A pipeline reads data from an external system, maps it to your data model, and stores it as records in your application. The role to grant depends on where the user works. Users build pipelines in Data Fusion or on the self-service data integration page of a custom application. Users can also run data integration from code. Find the user's situation in the decision table, check what the role allows, then add the user to the UserGroup for that role. Users who load documents and other unstructured content need one of the unstructured data integration roles instead.

Copy link to this section

How roles, UserGroups, and action groups work

Access in the C3 Agentic AI Platform follows a role-based model:

• User: an identity that signs in and performs actions.
• Role: a set of permissions. Each permission allows a method, a Type, or an action group. A Type defines one kind of record or service in the C3 AI Type System. For example, the permission allow:*:selfservicedi-user: allows every method in the selfservicedi-user action group. A role can also hold data permissions, which control which records the role can read or write. A role that nests another role also grants every permission of that role.
• UserGroup: a group of users with exactly one role, and the UserGroup ID equals the role ID. You grant a role to a user by adding the user to the UserGroup with that ID.
• Action group: a named set of methods that a role allows as one unit. Type authors assign a method or a whole Type to an action group with the @action(group+='...') annotation.

Action groups also protect the more privileged data integration and administrative methods:

• provision: protects the DataIntegrator methods that wrap CloudMessageBroker methods. C3.Developer allows provision.
• cluster-admin: protects reading source system credentials together with their secret values. C3.ClusterAdmin allows cluster-admin.
• env-admin: protects managing the allowlist of tables for an external data source (ExternalDsWhitelist). C3.EnvAdmin allows env-admin.

Copy link to this section

Choose a data integration role

Find the row that matches how the user works with data integration, then grant the roles in the second column.

The SelfServiceDI roles are for applications other than C3 AI Studio. A custom application can embed the self-service data integration page so that its users build pipelines there. SelfServiceDI.Role.User gives those users only the data integration actions the page calls, without the full access that C3.Developer grants.

Copy link to this section

Data integration in C3 AI Studio

C3 AI Studio is the environment where you build and manage applications. C3.StudioUser nests C3.StudioSharedUser, the role that holds the permissions common to Studio users and Studio administrators. For data integration, C3.StudioSharedUser:

• Nests C3.DataFusionUser for the Data Fusion generative AI features.
• Nests C3.DataValidationUser for data validation.
• Allows DataInteg.GenAi and GenAiUtil, the methods behind the AI-assisted steps on the Data Fusion canvas.

C3.DataQualityAnalyst gives a Data Fusion user access to the canvas without C3.Developer. The role allows the methods the canvas calls to:

• Register source systems and set their credentials.
• Configure source collections, map transforms, and save and run pipelines.
• Schedule runs with CronJob.
• Monitor runs and work with data validation rules.

C3.DataQualityAnalyst does not cover three tasks:

• Snowflake key-pair authentication: the role does not allow PrivateKeyAuth.createFromPem, the call that saves the private key. A user who sets up a Snowflake key pair needs C3.Developer for that step.
• Unstructured data integration: the role does not allow the Genai Types. Grant one of the roles in Unstructured data integration roles.
• Data Monitoring: C3 AI Studio shows Data Monitoring only to users with C3.Developer, C3.AppAdmin, C3.EnvAdmin, or C3.ClusterAdmin. A user with C3.StudioUser and C3.DataQualityAnalyst sees Data Fusion, File Explorer, and the Batch, MapReduce, and Cron job pages, but not Data Monitoring. To let that user check pipeline runs there, also grant C3.Developer, or check runs on the Data Fusion canvas.

C3.Developer allows every method on the platform (allow:*::*) and the provision action group. The role's data permission grants full data access. Grant C3.Developer only to users who also develop Types or need provision. C3.AppAdmin and C3.EnvAdmin also allow every method, so do not grant them only for data integration.

Copy link to this section

SelfServiceDI.Role.User

Grant SelfServiceDI.Role.User to a user of a custom application who builds pipelines on the self-service data integration page. The role allows allow:*:selfservicedi-user:, every method in the selfservicedi-user action group. The self-service flow assigns that action group to the Types it generates for a pipeline, so the user can work with those Types. The role also nests SelfServiceDI.Role.PageAccess.Home, and the user inherits every permission that role grants.

Copy link to this section

SelfServiceDI.Role.PageAccess.Home

SelfServiceDI.Role.PageAccess.Home holds the permissions that the self-service data integration page calls to configure and run a pipeline. A user receives this role through SelfServiceDI.Role.User. The permissions follow the order in which a user builds a pipeline. A source system is the connection to an external data store, such as a storage bucket or a database, and it holds the credentials. A source collection is one dataset in that source system. For more on these objects, see Data Integration Overview.

Connect source systems:

• Register, configure, inspect, and remove file and SQL source systems (FileSourceSystem, SqlSourceSystem).
• Test connectivity with ping and set credentials for file, SQL, and streaming source systems (CloudMessageSourceSystem), and remove credentials from a SQL source system.
• Look up a source system by name.
• Preview a SQL table, and infer Source Types from a SQL source system. A Source Type describes the fields of the incoming data exactly as they arrive.

Define what to read:

• Create, configure, and remove file and SQL source collections (FileSourceCollection, SqlSourceCollection).
• Read and set configuration values on SourceCollection.Config, and set configuration values on CloudMessageSourceCollection.Config.
• Resolve the inbox and archive URLs of a file source collection.
• Register a streaming source collection and start on-arrival processing, which processes each message as it arrives.
• Infer a Source Type from the data that a source collection reads.

Work with source files:

• Fetch the source files that a file source collection has found.
• Sync file metadata in batches, and remove files together with the data loaded from them.

Connect to SQL databases:

• List table names and describe tables through Jdbc.
• Save credentials for a SQL source system: saving them calls JdbcStore::setCredentials and JdbcStore::setExternal, which a role must allow by name, so the role names both methods.

Build transforms and pipelines:

• Create, fetch, and remove Transform metadata, which maps source fields to the fields of the target entity Type.
• Preview a transform on a small sample of source data (DataIntegrator::transformSources) and save the pipeline (DataIntegrator::upsertPipeline). DataIntegrator is a Type that groups these methods.

Infer schemas and read Type metadata:

• Infer date and time formats, and infer Type metadata from sample JSON.
• Read Type metadata, field and method names, and Type dependencies.
• Read package and Type definitions to render the data model.

Manage files and uploads:

• List, move, and delete files, list file system mounts, find a file from its URL, and write files.
• Upload large files in parts (multipart upload).
• Read line-delimited JSON from files in Amazon S3, Azure, and Google Cloud Storage.

Monitor runs:

• Fetch DataIntegStatus, SourceStatus, and TargetStatus, the status records that show the progress and results of a pipeline run.

The role also allows a few utility methods that the page relies on.

Copy link to this section

C3.DataFusionUser

C3.DataFusionUser gives access to the generative AI features of Data Fusion through its only two permissions:

• Summarize pipeline errors with DataFusionErrorSummarizer.
• Map connector properties with DataFusionConnectorPropertyMapper.

Studio users receive C3.DataFusionUser through C3.StudioSharedUser, which C3.StudioUser nests.

Copy link to this section

Roles for programmatic data integration

Grant C3.Developer to a user or service account that runs data integration from code, such as the C3 AI Console, a notebook, or a scheduled job. C3.Developer allows every method, including the provision action group that protects the DataIntegrator methods that wrap CloudMessageBroker methods.

The platform also defines a role with the ID C3.DataIntegrator. That role grants no permissions and nests no roles, so it does not give a user access to data integration.

Copy link to this section

Assign a data integration role

Grant a role by adding the user to the UserGroup whose ID equals the role ID. Sign in as a user who can manage UserGroup membership in the application, for example a member of C3.UserAdmin, then:

1. Open a Python session connected to the application, for example a Jupyter notebook.

2. Run this code. Replace jane.doe@example.com with the user's ID, and replace SelfServiceDI.Role.User with the role ID from the decision table.

   Python

   Copy code to clipboard

   # Add a user to the self-service data integration role
   c3.User.forId('jane.doe@example.com').addToGroup('SelfServiceDI.Role.User')

3. Confirm the result. The call returns the user. Check that the user's groups field lists the role ID.

For other ways to manage group membership, see Assign Users.

Copy link to this section

Unstructured data integration roles

Unstructured data integration loads documents and other unstructured content into source collections and splits them into passages, short sections of text. A vector retriever searches those passages by similarity so that generative AI features can answer from your content, a pattern called retrieval-augmented generation (RAG). Three roles control access to unstructured data integration.

Copy link to this section

Genai.UnstructuredDI.Common

Genai.UnstructuredDI.Common provides the base permissions for unstructured data integration, most of them read access:

• Read source collections, source files, passages, and their metadata categories and tags.
• Read from retrievers and run similarity search against a vector retriever.
• Read embedder and default configuration, and read basic server and runtime information.

Genai.UnstructuredDI.Common also allows a few write actions: creating persisted records (Data.Persisted::create), writing strings to files (File::writeString), and the Python utility methods in Genai.PyUtil. The role nests GenaiCore.BaseUser.

Copy link to this section

Genai.UnstructuredDI.BasicUser

Genai.UnstructuredDI.BasicUser nests Genai.UnstructuredDI.Common and adds a data permission on Genai.Vector.SourcePassage that applies access control list (ACL) filtering.

Copy link to this section

Genai.UnstructuredDI.AdminUser

Genai.UnstructuredDI.AdminUser provides full management of unstructured data integration:

• Full access to Genai.SourceFile, Genai.SourcePassage, Genai.SourceCollection, and the PgVector retriever.
• Manage source file metadata categories and tags, and preview and apply metadata tagging.

Genai.UnstructuredDI.AdminUser nests Genai.UnstructuredDI.Common and inherits its read permissions. The role's data permissions grant full data access.

Copy link to this section

See also

Data Integration Built-in Roles points to this page from the Security guide.

• Data Integration Built-in Roles
• Permissions in the C3 Agentic AI Platform
• Add Action Groups to a Method
• Assign Users
• Data Fusion Overview
• Data Integration Overview