Image Chunker and Verbalization

When processing image files, the goal is to index the verbalized content of the images to facilitate efficient querying. This process is known as image chunking. Currently supported image formats are JPEG (.jpeg, .jpg) and PNG (.png).

Copy link to this section

Setting up the Image Chunker

• The Image Chunker is enabled by default, using the GPT-4 model as the default verbalizer.
  • The available models are any model available on your cluster.
  • The API keys must be set for the model you want to use.
• The major configuration is:
  • LLM Config: Genai.SourceFile.Chunker.Image.Config#llmConfigName (specifies which LLM to use, such as Gemini or GPT-4)

Modify the LLM Config to specify the language model to use for verbalization.

Copy link to this section

Steps to Change the Model Configuration

1. Find the completion client configurations you have available:

GenaiCore.Llm.Completion.Client.UiHelper.listCompletionClients();

2. Return the current LLM configuration you have set on Genai.SourceFile.Chunker.Image.Config#llmConfigName.

   JavaScript

   Copy code to clipboard

   Genai.SourceFile.Chunker.Image.Config.setConfigValue('llmConfigName', '<model_name>');
   

The model_name is the model name config of the LLM completion client, not the actual model you want to use.

Copy link to this section

Using the Image Chunker from the static console

• Execute the following to command to do the chunking and retrieve the image verbalization.

  JavaScript

  Copy code to clipboard

  var filePath = '<filePath>';
  var sourceFile = c3.Genai.SourceFile.make({ originalFile: C3.File.make(filePath) }).upsert();
  
  //Optionally, the maxLength allowed for chunking after verbalization can be overridden as follows,
  var imageCfg = Genai.SourceFile.Chunker.Image.Config.getConfig();
  imageCfg.setConfigValue('maxLength', 1000); //the default value is 1000
  //
  
  var passagesWithGptVerbalization = Genai.SourceFile.Chunker.Image.chunkFile(
    sourceFile,
    Genai.SourceFile.Chunker.UniversalChunker.Spec.make({ chunker: Genai.SourceFile.Chunker.Image }),
    true
  ).passages[0]['imageVerbalization'];