C3 AI Documentation Home

Prewarming Pool System

Prewarming Pool System

Design document for the Genesis application and workspace prewarming system. This system maintains pools of fully-provisioned applications and workspaces that can be instantly handed over to users, eliminating multi-minute provisioning wait times.

Problem Statement

Creating a Genesis application or workspace involves a lengthy provisioning pipeline:

  1. Application creation (~5-10 min): Provisions a StudioEnv (Kubernetes pod), extracts template files, starts dev/live apps, initializes Code and Agent services.
  2. Workspace creation (~5-10 min): Same pipeline but also includes git clone for git-configured applications.

Users experience this as a cold-start delay every time they create a new application or workspace. The prewarming system eliminates this by maintaining a pool of pre-created resources ready for instant assignment.

Architecture

The system has two modular components, each solving a distinct use case:

ComponentUse CasePool Key
PreprovisionedApplicationUser creates app from a template (blankPkg, genaiPkg, etc.)Per template
PreprovisionedWorkspaceUser creates workspace in a git-configured applicationPer application

A shared Pool Controller (CronJob) runs every minute to reconcile pool sizes and clean up stale entries.

System Context Diagram

Text
                         ┌─────────────────────────────────────────┐
                         │         Pool Controller (CronJob)       │
                         │  genesis-preprovisioning-reconciler      │
                         │  Runs every minute                       │
                         └──────────┬──────────────┬───────────────┘
                                    │              │
                    ┌───────────────▼──┐     ┌─────▼───────────────┐
                    │ Application Pool │     │  Workspace Pool     │
                    │ (per template)   │     │  (per application)  │
                    │                  │     │                     │
                    │ blankPkg  x3     │     │  App-A  x2          │
                    │ genaiPkg  x2     │     │  App-B  x3          │
                    └────────┬─────────┘     └─────────┬───────────┘
                             │                         │
            ┌────────────────▼──────┐    ┌─────────────▼─────────────┐
            │ Template.createApp()  │    │ Workspace.createWithPre() │
            │ tries claim first     │    │ tries claim first         │
            └────────────────┬──────┘    └─────────────┬─────────────┘
                             │                         │
                     ┌───────▼─────────┐       ┌───────▼──────────────┐
                     │  User gets app  │       │  User gets workspace │
                     │  (instant)      │       │  + git checkout      │
                     └─────────────────┘       └──────────────────────┘

Data Model

Entity Types

Genesis.PreprovisionedApplication

Tracks a single pre-provisioned application in the warm pool.

FieldTypeDescription
template!Genesis.TemplateTemplate this was provisioned from
applicationGenesis.ApplicationThe pre-created application
statusenum StatusPROVISIONING, READY, CLAIMING, CLAIMED, ERROR, EXPIRED
claimedByUserUser who claimed (null if unclaimed)
claimedAtdatetimeWhen claimed
claimDurationMsintEnd-to-end duration of the claim operation
createdAt!datetimeWhen provisioning started
readyAtdatetimeWhen reached READY
errorMessagestringError details if failed

DB indexes: [application, status], [status]

Pool Configuration Types

Both pools are configured via separate PoolConfig entity types:

Genesis.PreprovisionedApplication.PoolConfig (per template):

  • template: Which template to prewarm
  • poolSize: Desired number of READY entries
  • maxAgeHours: Expiry threshold (default 48h)
  • enabled: Toggle

Genesis.PreprovisionedWorkspace.PoolConfig (per application):

  • application: Which git-app to prewarm workspaces for
  • poolSize: Desired number of READY entries
  • maxAgeHours: Expiry threshold (default 24h)
  • enabled: Toggle

Status Lifecycle

Text
   ┌──────────────┐
   │ PROVISIONING │ ─── (reconciler promotes) ───► READY ──── (privileged claim) ────► CLAIMING
   └──────────────┘                                  │                                    │
          │                                          │ (maxAge exceeded)        ┌─────────┴─────────┐
          │ (failure or 30m timeout)                 ▼                          │                   │
          ▼                                       EXPIRED                  (success)          (failure)
        ERROR                                                                  │                   │
          │                                                                    ▼                   ▼
          │ (24h)                                                            CLAIMED             ERROR
          ▼
       [removed]

State transitions:

  • PROVISIONING → READY: Reconciler detects underlying workspace is configured.
  • READY → CLAIMING: Genesis.PrivilegedAction.claimWorkspace/claimApplication acquires optimistic lock (race-safe, up to 3 retry attempts on version conflict).
  • CLAIMING → CLAIMED: Ownership transfer, env admin registration, and git checkout all succeeded.
  • CLAIMING → ERROR: Downstream claim work failed; errorMessage records the failure reason. Terminal — requires operator investigation.
  • READY → EXPIRED: Entry exceeds maxAgeHours.
  • PROVISIONING → ERROR: Provisioning failure or 30-minute timeout.

Core Algorithms

1. Reconciliation (every minute)

The Genesis.Preprovisioning.reconcilePools() function:

  1. Log pool status for observability (ready/provisioning/errored counts per pool)
  2. Promote ready entries: Check PROVISIONING entries whose underlying workspace has reached READY or ERROR state, and promote them accordingly. Also timeout entries stuck in PROVISIONING for 30+ minutes with no linked resource.
  3. Reconcile application pools: For each enabled PoolConfig, compute deficit = poolSize - readyCount - provisioningCount. Submit deficit AsyncActions to provisionNew.
  4. Reconcile workspace pools: Same logic per application.
  5. Cleanup expired: Mark stale READY entries as EXPIRED, terminate their environments. Remove ERROR entries older than 24h.

Promotion runs first so that deficit calculations reflect entries that just finished provisioning, preventing over-provisioning.

2. Claim (on user request)

When a user creates an application or workspace, Genesis.PrivilegedAction.claimWorkspace or claimApplication is called as a privileged action (@action(authz='always')):

  1. Retry loop (up to 3 attempts): Re-fetches the oldest READY entry on each iteration for a fresh version.
  2. Atomically update status to CLAIMING via merge() (optimistic locking). If the merge fails (version conflict / race condition), retry with the next available entry.
  3. Single privileged lambda (StudioCluster.executePrivilegedJsLambda): All ownership-transfer writes run in a single Ctx.asRoot() context:
    • Merge Genesis.Application (rename, set admins/users, clear isPreprovisioned, set meta.createdBy), then call Genesis.Application.populateAcl.
    • For each workspace: merge Genesis.Workspace (admins/users, clear isPreprovisioned, set meta.createdBy), call Genesis.Workspace.populateAcl.
    • Transfer StudioEnv admins inline: filter out old createdBy user from admins/users, add claiming user to admins/users/createdBy/updatedBy, call StudioEnv.populateAcl.
    • Cascade to child StudioApps: same inline admin transfer pattern, call StudioApp.populateAcl.
  4. Post-lambda (as calling user): For each workspace found via Genesis.Workspace.fetch by application ID, set hibernationEnabled = true via config, re-fetch workspace, and call doSyncWorkspaceEnvAdmins to register the user in the running environment's C3.EnvAdmin group.
  5. Git checkout (workspace claims): Check out the user's requested branch.
  6. On success: Update status to CLAIMED, record claimDurationMs, return the claimed entity.
  7. On failure: Update status to ERROR, store the error in errorMessage, re-throw so the caller falls back to normal creation.

If no READY entry exists (pool exhausted) or all retry attempts are lost to race conditions, the system seamlessly falls back to normal creation. The user experience is identical either way.

The privileged action eliminates the need for isPreprovisioned == true write data permissions on C3.Code.User — users only need read access to discover preprovisioned entries, and the claim action handles all writes with elevated privileges.

3. Provisioning (non-blocking, background)

Provisioning is split into two phases to avoid thread starvation:

Phase 1 — provisionNew (fast, releases thread immediately):

  1. Creates a PreprovisionedApplication/Workspace entry with status=PROVISIONING.
  2. Creates the actual Application/Workspace with:
    • A placeholder display name (prewarmed <uuid-prefix>)
    • An explicit pkgName of the form pkg<uuid-prefix> (e.g. pkg6306adba). This is the on-disk folder name and is set here — at the moment the folder is created — so it stays stable for the lifetime of the application. pkg<uuid-prefix> is used instead of the display-name-derived prewarmed<uuid> because (a) it is a valid C3 package identifier by construction (starts with a letter, no spaces, no hyphens) and (b) it does not leak the "prewarmed" origin into the package identity, which users see in deployment artifacts and repo layouts.
    • isPreprovisioned: true flag (bypasses user quota checks)
  3. Links the created resource to the entry via merge().
  4. Returns immediately — does NOT wait for the workspace to reach READY.

On failure, sets status=ERROR with the error message.

Phase 2 — promoteReady (in the reconciler, every minute):

  1. Fetches all PROVISIONING entries that have a linked application/workspace.
  2. Checks the workspace's current state:
    • If READY → promotes the entry to READY.
    • If ERROR → marks the entry as ERROR.
    • Otherwise → leaves it in PROVISIONING (will re-check next cycle).
  3. Times out entries stuck in PROVISIONING for 30+ minutes with no linked resource (creation failed silently).

This two-phase design ensures provisionNew never holds an async action thread longer than a few seconds. The underlying configureWorkspace action (which actually provisions the Kubernetes pod and makes the workspace READY) runs independently and gets threads freely.

5. Ownership Transfer

All ownership transfer logic runs inside a single StudioCluster.executePrivilegedJsLambda call, executing in Ctx.asRoot():

For applications:

  • Renames application to user's desired display name (name field only)
  • Does NOT rewrite pkgName — the package name is set at provisioning (pkg<uuid-prefix>) and is the on-disk folder name; rewriting it would desync from disk. See On-disk name stability.
  • Sets the user's description
  • Clears isPreprovisioned flag on the application and its workspaces
  • Sets the claiming user as admin/user on Genesis.Application and Genesis.Workspace
  • Updates meta.createdBy on all affected entities
  • Calls populateAcl on each modified entity to immediately refresh ACL caches
  • Transfers StudioEnv admins inline: filters out old createdBy user from admins/users, adds claiming user to admins/users/createdBy/updatedBy
  • Cascades StudioEnv transfer to all child StudioApps (same inline pattern)
  • Post-lambda: re-enables hibernation on all workspaces, re-fetches each workspace, and calls doSyncWorkspaceEnvAdmins to register the user in the running environment

For workspaces:

  • Renames to user's desired name
  • Clears isPreprovisioned flag
  • Sets the claiming user as admin/user
  • Updates meta.createdBy
  • Calls populateAcl on the workspace
  • Transfers StudioEnv admins inline (same pattern as above)
  • Cascades to child StudioApps
  • Post-lambda: re-enables hibernation, re-fetches workspace, calls doSyncWorkspaceEnvAdmins, then performs git checkout via agent service

CronJobs

The preprovisioning system uses a single CronJob created by Genesis.Preprovisioning.enableReconciler():

CronJob IDPurposeIntervalAction
genesis-preprovisioning-reconcilerPool reconciliation, promotion, cleanupEvery minutereconcilePools

The reconciler uses runAsCreatedBy: true, meaning it executes as the admin user who called enableReconciler(). This is critical because the reconciler needs ClusterAdmin permissions for env.callJson() calls during provisioning.

JavaScript
// Run once after deployment as a ClusterAdmin user:
Genesis.Preprovisioning.enableReconciler()

Integration Points

Modified: Genesis.Template.createApplication()

Text
Before: Always calls Genesis.Application.make({...}).create()
After:  First tries Genesis.PrivilegedAction.claimApplication(templateId, name, desc)
        Falls back to normal creation if claim returns null

Modified: Genesis.Workspace.createWorkspace()

Text
Before: Only Genesis.Workspace.make(spec).create() existed
After:  New static method that tries Genesis.PrivilegedAction.claimWorkspace() for git apps
        Falls back to Genesis.Workspace.make(spec).create()
        Also validates user quota (canCreateWorkspace)

Modified: Genesis.Application.beforeCreate() / Genesis.Workspace.beforeCreate()

Added isPreprovisioned check that bypasses:

  • User quota validation (canCreateApplication, canCreateWorkspace)
  • Name regex validation (placeholder names like "prewarmed abc123" contain spaces)
  • Admin/user assignment (system creates these without a real user context)

The isPreprovisioned field is declared private to prevent external API callers from setting it.

For preprovisioned applications, pkgName is set explicitly by provisionNew to pkg<uuid-prefix> (see Provisioning above) rather than being derived from the placeholder display name. beforeCreate therefore leaves pkgName untouched when it is already set — the derivation-from-name fallback is retained only for the non-preprovisioned path where no explicit pkgName is supplied. See On-disk name stability below.

Modified: Genesis.Workspace.configureWorkspace()

After a preprovisioned workspace finishes configuration, both Genesis-level and Studio-level hibernation are explicitly disabled:

  1. hibernationEnabled = false — prevents the Genesis stopInactiveWorkspaces cron from stopping it.
  2. StudioCluster.Hibernation.setPolicyForSenvs([envId], 'InactivityNoPolicy') — prevents the Studio platform's own inactivity monitor from hibernating the pod.

The isPreprovisioned field is fetched explicitly via this.get('isPreprovisioned').isPreprovisioned rather than accessed directly on this. This is necessary because configureWorkspace is dispatched via AsyncAction, which does not reliably hydrate fields on the this reference — even with @dependency annotations.

isPreprovisioned Flag Lifecycle

The isPreprovisioned flag on both Application and Workspace is transient — it is only true while the resource is owned by the pool system:

PhaseisPreprovisionedhibernationEnabledStudio Hibernation Policy
Provisioningtruetrue (default)Default (active)
Ready (in pool)truefalse (disabled after configure)InactivityNoPolicy (disabled)
Claimed (transferred to user)falsetrue (re-enabled)Default (active)

On claim, the privileged lambda clears the flag and the post-lambda code re-enables Genesis-level hibernation so the resource behaves as a normal user-owned entity — subject to standard quota checks, inactivity timeouts, and all other policies. The Studio-level policy is not explicitly reset on claim; the next resume cycle sets InactivityNoPolicy (see resumeWorkspace), and the Genesis-level stopInactiveWorkspaces takes over as the primary inactivity mechanism for claimed workspaces.

On-disk name stability (pkgName)

Problem. The DB field Genesis.Application.pkgName serves two purposes:

  1. Identity — the platform-visible package name (deployment artifact names, Pkg.Store lookups, on-disk manifest <pkgName>.c3pkg.json).
  2. Location — where the package lives on disk (/usr/workspace/${pkgsPath}/${pkgName}/…) and therefore what value VITE_C3_PKG must have so the UI service can locate ui/react/.

The two are fundamentally the same fact: the on-disk folder name is the package identity in a C3 workspace. They can safely be represented by a single field — provided that field never diverges from what is on disk. Historically, transferOwnership rewrote pkgName on claim (to camelCase(newName)) but could not rename the on-disk folder (platform limitation). While the pod stayed up the drift was invisible; on workspace resume the pod is re-created and getGenesisNodePoolPlanConfig re-templates env vars from the current DB state — the new UI pod boots with VITE_C3_PKG set to the user-facing camelCased name, waitForUiDir polls a path that will never exist, and the pod hangs in appPhase = 'waiting' with appMessage = 'Waiting for UI directory: <path>' indefinitely.

Solution. Keep pkgName as the single source of truth for both roles and make it stable by construction — never rewrite it after the folder has been created on disk.

  1. provisionNew sets pkgName explicitly to pkg<uuid-prefix> (e.g. pkg6306adba). This is a valid C3 package identifier by construction (starts with a letter, no spaces, no hyphens) and does not leak the "prewarmed" origin into the package name that users see in deployment artifacts and repo layouts. The display name (name field) still uses prewarmed <uuid-prefix> for internal observability while the entry is in the pool, but the pkgName and on-disk folder are pkg<uuid-prefix> from the moment they are created.

  2. transferOwnership no longer touches pkgName. It updates the user-facing name and description, clears isPreprovisioned, and swaps admins/users — but the package identity is fixed at provisioning time. The user's chosen display name lives in name; pkgName remains the immutable pkg<uuid-prefix> that matches the on-disk folder.

  3. Genesis.Application#getRootPkgName (currently return pkgName || name) remains the single accessor. Because pkgName is now always populated at create for both preprovisioned and normal paths, the || name fallback is effectively dead code but is left in place for defence-in-depth on legacy rows.

The pod-template pipeline in Genesis.Workspace.js#getGenesisNodePoolPlanConfig is unchanged in shape — the existing ${rootPkgName} template variable now correctly resolves to the on-disk folder name on every code path (initial create, first configuration, resume). Both VITE_C3_PKG (UI container) and MAIN_PKG_NAME (coding-agent container) in config/StudioNodePoolPlanConfig/c3app.json continue to read ${rootPkgName} — no config change is required.

Backfill. Applications claimed under previous code have pkgName that no longer matches the on-disk folder. A one-shot migration is required:

  1. Fetch every Genesis.Application where isPreprovisioned = false and a Genesis.PreprovisionedApplication entry references it with status CLAIMED.
  2. For each such application, resolve the on-disk folder name by inspecting /usr/local/share/c3/fs/genesis/${pkgsPath}/ inside its studio env and locating the single *.c3pkg.json file. The folder name is the correct pkgName.
  3. merge() the resolved value onto the application to overwrite the drifted pkgName.
  4. Applications created under new code have pkgName = pkg<uuid-prefix> from create time and match disk, so backfill is only needed once.

Non-goals.

  • This change does not attempt to rename the folder on disk, and does not require the pkg store to relocate the package.
  • This change does not add a new field on Genesis.Application. The single pkgName field carries both identity and location; keeping them equal is enforced by construction (pkgName is set at create and never rewritten).
  • The user-facing display name (name) continues to change on claim as before — only the package identity is frozen.

Configuration

Default Pool Sizes (seed data)

TemplatePool SizeMax Age
blankPkg348h
genaiPkg248h

Workspace pools are configured dynamically per-application by platform admins.

Observability

Structured Logging

Every reconciliation cycle logs:

Text
Pool status [Application]: template=blankPkg ready=2 provisioning=1 errored=0 desired=3
Pool status [Workspace]: application=abc123 ready=1 provisioning=0 errored=1 desired=2

Key events logged:

  • Pool deficit detection with counts
  • Successful claims (template/app, user, new name, claimDurationMs)
  • Failed claims (race condition fallback)
  • Claim duration per entry
  • Provisioning start/completion/failure
  • Expiry and cleanup actions

Key Metrics to Monitor

MetricWhat it tells you
Claim hit rate% of creations served from pool vs. fallback
Claim duration (claimDurationMs)End-to-end time including cross-env admin sync
Pool fill levelREADY / desired ratio per pool
Provisioning durationTime from PROVISIONING to READY
Error rateFailed provisioning attempts
StalenessAverage time a READY entry sits before claim or expiry

Failure Modes and Mitigations

FailureImpactMitigation
Pool exhaustedUsers fall back to normal creation (no degradation)Alert on low fill rate, increase poolSize
Provisioning failureEntry marked ERROR, cleaned up in 24hMonitor error rate, fix underlying issues
Race condition on claimOne user retries up to 3 times, then falls back to normal creationOptimistic locking via merge(); retry loop fetches next READY entry
Claim downstream failureEntry marked ERROR with errorMessage, user falls back to normal creationMonitor claim error rate; investigate errorMessage
Stale entriesUser gets outdated server version or git statemaxAgeHours forces expiry and refresh
Reconciler downPool drains over timeCronJob auto-restarts; pool drains gracefully
addEnvAdmin fails in claimClaim returns null, falls back to normal creationUser unaffected; investigate remote env health
Runaway provisioningResource exhaustionDeficit calculation caps at poolSize
Thread starvationprovisionNew blocks threads, preventing configureWorkspaceNon-blocking provisionNew + reconciler-driven promotion
Stuck PROVISIONING entryEntry never promoted, wastes pool capacity30-minute timeout in promoteReady marks as ERROR
UI pod stuck "Waiting for UI directory" on resume of a claimed prewarmed workspacePreview never comes up; UI service polls a path that doesn't exist on diskKeep pkgName stable by construction: provisionNew sets pkgName = pkg<uuid-prefix> (matches the folder created on disk) and transferOwnership no longer rewrites it — only the user-facing name changes on claim (see On-disk name stability)

Resource Impact

Each preprovisioned entry consumes:

  • 1 Kubernetes pod (3 CPU, 48GB RAM for SNE + agent/UI/code sidecars)
  • 1 Azure Managed Disk (100 GiB XFS)
  • 1 StudioEnv + StudioApp in the platform DB

Estimated steady-state cost (with default configs):

  • Application pool: 5 pods (3 blankPkg + 2 genaiPkg)
  • Workspace pool: Variable per enabled application

The expiry mechanism (48h for apps, 24h for workspaces) ensures resources are recycled regularly.

File Layout

Text
repo/genesis/src/preprovisioning/
├── Genesis.Preprovisioning.c3typ              # Controller utility type (reconcilePools)
├── Genesis.Preprovisioning.js                 # reconcilePools, cleanup
├── Genesis.PreprovisioningStatus.c3typ        # Shared enum: PROVISIONING, READY, CLAIMING, CLAIMED, ERROR, EXPIRED
├── Genesis.PreprovisionedApplication.c3typ    # Entity type
├── Genesis.PreprovisionedApplication.PoolConfig.c3typ
├── Genesis.PreprovisionedApplication.js       # provisionNew (claim delegated to PrivilegedAction)
├── Genesis.PreprovisionedWorkspace.c3typ      # Entity type
├── Genesis.PreprovisionedWorkspace.PoolConfig.c3typ
└── Genesis.PreprovisionedWorkspace.js         # provisionNew (claim delegated to PrivilegedAction)

repo/genesis/src/
├── Genesis.PrivilegedAction.c3typ             # claimWorkspace, claimApplication declarations
└── Genesis.PrivilegedAction.js                # Claim logic: single privileged lambda with ownership transfer

repo/genesis/data/
├── Genesis.PreprovisionedApplication.PoolConfig/
│   ├── blankPkg.json
│   └── genaiPkg.json
└── Genesis.PreprovisionedWorkspace.PoolConfig/
    └── example.json                           # Disabled template for reference

Modified existing files:

  • repo/genesis/src/Genesis.Template.js - claim before create
  • repo/genesis/src/Genesis.PrivilegedAction.c3typ - declares claimWorkspace and claimApplication with @action(authz='always')
  • repo/genesis/src/Genesis.PrivilegedAction.js - full claim implementation with executePrivilegedJsLambda
  • repo/genesis/src/Genesis.Application.c3typ - isPreprovisioned field
  • repo/genesis/src/Genesis.Application.js - quota bypass
  • repo/genesis/src/workspace/Genesis.Workspace.c3typ - isPreprovisioned field + createWorkspace method + @dependency on configureWorkspace
  • repo/genesis/src/workspace/Genesis.Workspace.js - quota bypass + createWorkspace + Studio hibernation policy for preprovisioned
  • repo/genesis/src/preprovisioning/Genesis.PreprovisionedApplication.js - provisionNew sets pkgName = 'pkg' + <uuid-prefix> explicitly at create (instead of letting beforeCreate derive it from the placeholder display name); transferOwnership no longer rewrites pkgName on claim
  • repo/genesis/metadata/Role/C3.Code.User.json - allow:Genesis.PrivilegedAction::claimWorkspace and claimApplication action permissions

Known Issues and Workarounds

Thread Starvation (Resolved)

Problem: The original provisionNew implementation held an async action thread for up to 20 minutes while calling workspace.waitForReady(). Since configureWorkspace (which makes the workspace READY) also needs an async action thread, this created a deadlock when the thread pool was saturated — provisionNew blocked waiting for READY, but READY could never be reached because configureWorkspace couldn't get a thread.

Resolution: provisionNew was refactored to be non-blocking. It creates the application/workspace, links it to the pool entry, and returns immediately. A new promoteReady() step in the reconciler checks PROVISIONING entries every minute and promotes them once their workspace reaches READY. This eliminates all blocking waits from the async action queue.

Reconciler Must Run as ClusterAdmin

Problem: When the CronJob was deployed via seed data, it ran as provisioner@c3 — a user without permissions to perform operations like OAuthApplication#fetch and C3FileSystem#setMount in provisioned environments. The worker user is also explicitly excluded from receiving C3.EnvAdmin by the platform's StudioEngine.upsertUserToApp() method.

Resolution: The reconciler CronJob is no longer deployed via seed data. Instead, a ClusterAdmin must call Genesis.Preprovisioning.enableReconciler() from the console after deployment. This creates both CronJobs with runAsCreatedBy: true, so all operations run with ClusterAdmin permissions.

Cross-Env addEnvAdmin Authorization (Resolved)

Problem: When a non-admin user (e.g., mytestuser with only C3.Code.User role) claims a preprovisioned application or workspace, the original claim code called ws.addEnvAdmin(user) directly. This makes a cross-env REST call (env.callJson('Lambda', ...)) to the remote StudioEnv to execute User#upsert and user.addToGroup('C3.EnvAdmin'). The call authenticates as the current session user — but that user doesn't exist there yet.

Resolution: The claim flow was moved into Genesis.PrivilegedAction.claimWorkspace / claimApplication with @action(authz='always'). All ownership-transfer writes (Genesis.Application, Genesis.Workspace, StudioEnv, StudioApp merges) run inside a single StudioCluster.executePrivilegedJsLambda call which executes in Ctx.asRoot(), bypassing all data-permission checks. The isPreprovisioned == true write data permission was removed from C3.Code.User. After the privileged lambda, doSyncWorkspaceEnvAdmins runs in the calling user's context (now an admin thanks to populateAcl calls inside the lambda) to register them in the remote environment.

Preprovisioned Workspaces Hibernating (Resolved)

Problem: Preprovisioned workspaces were being hibernated despite the hibernationEnabled = false logic in configureWorkspace. Two root causes:

  1. configureWorkspace accessed this.isPreprovisioned directly, but when invoked asynchronously via AsyncAction, the field was never fetched from the DB — it was always undefined, so the if block was skipped and hibernationEnabled remained at its default true. Note: @dependency(include='isPreprovisioned') annotations do not reliably hydrate fields for AsyncAction-dispatched methods.

  2. Studio's platform-level StudioCluster.Hibernation has its own inactivity policy independent of Genesis. Even if hibernationEnabled were correctly set, Studio would still hibernate idle environments after its own timeout.

Resolution: Changed the field access to an explicit DB fetch via this.get('isPreprovisioned').isPreprovisioned, which guarantees the value is loaded regardless of how the method was invoked. Also added StudioCluster.Hibernation.setPolicyForSenvs([envId], 'InactivityNoPolicy') to disable Studio-level auto-hibernation for pool workspaces.

Future Enhancements

  1. Adaptive pool sizing: Use claim rate history to auto-tune poolSize per template.
  2. Priority queuing: When pool is empty and multiple users request simultaneously, queue them rather than all hitting the slow path.
  3. Warm workspace for non-git apps: Extend PreprovisionedWorkspace to template-based (non-git) applications.
  4. Pool sharing across clusters: For multi-region deployments, share pool status to route users to clusters with available entries.
  5. Hibernation-aware prewarming: When a preprovisioned workspace approaches maxAge, hibernate it rather than terminate (cheaper to resume than re-provision).
  6. Remove STOPPED workspaces from pool: Detect when a preprovisioned workspace's underlying environment enters STOPPED state and remove it from the PreprovisionedWorkspace and PreprovisionedApplication collections (mark as ERROR or EXPIRED), so that stale stopped entries are never claimed.
Was this page helpful?