Setup App for Airgap

The key differences in an air-gapped environment, as they relate to the GenAI app, are:

• Python runtimes cannot be installed directly via Conda or PyPI.
• Access to LLM providers may be unavailable.
• Hugging Face models used in embedders and parsers cannot be downloaded.

Copy link to this section

Export Python Runtimes

Since runtimes cannot be installed via Conda or PyPI in an air-gapped environment, all necessary runtime artifacts, specifically the .conda and .whl files along with their metadata, must be populated in the ArtifactHubService on the air-gapped cluster. The CondaLibraryManager should then be configured to reference this local artifact hub.

Follow the instructions in Export Python Runtime Artifacts for Use in Air-Gapped Environments to export the required artifacts.

In addition to all runtimes defined under genAiBase/metadata/ImplLanguage.Runtime/, the py-sentence_transformers runtime from genaiPlatform must also be exported.

This export process works smoothly for all runtimes not dependent on:

• wheels hosted externally (e.g., GitHub), or
• packages that require building from source.

However, the py-chunker runtime depends on both and requires a more involved process to be made air-gap compatible:

• Build a wheel file for detectron2.
• Publish both the detectron2 and mew3 wheels, along with their metadata, to the ArtifactHubService in a non-air-gapped environment.
• Resolve the runtime, ensuring all URL references are removed.
• Export the runtime artifacts using the standard process.

Important: The CondaLibraryManager#exportRuntimeArtifacts method does not include manually published wheel files and their metadata. These must be manually re-published to the air-gapped ArtifactHubService.

Finally, note that these steps will need the generation of a new, air-gap-compatible runtime artifact.

Copy link to this section

Setup MIS

In the absence of access to external LLM providers, MIS must be configured to host appropriate models locally. Follow the steps outlined in the resources below:

• Create and Configure the C3 AI Model Inference Service
• Create and Deploy a VllmPipe
• Manage Routes of the C3 AI Model Inference Service to Change or Upgrade LLMs
• Monitor and Scale the C3 AI Model Inference Service
• Use C3 AI Model Inference Service for LLM Text Generation and Inference Requests

The only additional consideration is that the runtimes required by MIS, py-vllm050post1 and py-vllm061post2, must also be exported following the same process described above.

Copy link to this section

Use Locally Cached Models

To correctly populate the model files required for chunking and indexing, refer to the documentation for the Genai.App.Airgap.Config type.

Copy link to this section

Incompatible Features

One current limitation of MIS is that it does not support chat completion or image payloads (reference). As a result, Dynamic Agent and multimodal parsing using any of Genai.SourceFile.Chunker.Mew3, Genai.SourceFile.Chunker.MultimodalPdf, or Genai.SourceFile.Chunker.MsftX are not air-gap compatible unless a whitelisted external LLM provider is available.

While document chunking with images will not fail, the resulting chunks will simply omit the image content.

Copy link to this section

Troubleshooting

Missing NLTK tagger files. Some files might conditionally require the averaged_perceptron_tagger_eng for chunking via the older langchain-based chunkers. To work around the issue,

• On a local machine or in any non-air-gapped environment, install nltk==3.9.1 and run nltk.download(averaged_perceptron_tagger_eng).
• Move the zip file to the air-gapped cluster and run the following in the app console:

function setupMissingNltkTagger() {
  Genai.PyUtil.downloadAndUnzip('<FileUrl of uploaded zip file>', null, null, true, '/home/c3/nltk_data/taggers/');
}
lambda = C3.Lambda.fromJsFunc(setupMissingNltkTagger);
C3.app()
  .nodePool('task')
  .nodes()
  .each((n) => n.callJson('Lambda', 'apply', lambda.toJson()));

If chunking is routed to an App.NodePool other than the default task, change the above accordingly.

Few-shot embedder does not initialize. Work around this issue by force-reinitializing in the app console:

GenaiCore.Embedder.Engine.list().each((e) => e.terminate());

cfg = Genai.App.AirGapConfig.getConfig();
engineName = 'Genai.Agent.Tool.Util.FewShotRetriever_embedder';
embedder = GenaiCore.Embedder.Hf.make({
  id: engineName,
  modelName: cfg.fewShotEmbedderModelName,
  modelDirectory: cfg.fewShotEmbedderPath,
}).merge({ mergeInclude: 'modelName, modelDirectory', returnInclude: 'this' });
embedderEngine = GenaiCore.Embedder.Engine.createFromEmbedder(embedder, { name: engineName });
GenaiCore.DynamicPy.Engine.waitForEngineRunning(embedderEngine, { maxWaitSeconds: 120 });