Changelog for genaiPlatform Version 2.1

Copy link to this section

[2.1.0]

Copy link to this section

Added

• GenaiCore.Llm.Embedding.Client
  • Added GenaiCore.Llm.Embedding.Client abstract interface for a text and/or image embedding client for an LLM service.
  • Added GenaiCore.Embedder.Llm to facilitate integration with existing GenaiCore.Embedder interface.
• GenaiCore.Embedder.Hf#uploadModelFiles
  • Added support for persisting Huggingface embedder files in C3 to freeze and avoid future downloads from Huggingface.
• GenaiCore.Llm.Completion#completionBatch
  • Added support for synchronous thread-parallelized batch completion.
• GenaiCore.Llm.Completion#submitCompletionJob
  • Added support for asynchronous batch completion, through GenaiCore.Llm.BatchRequestJob.
• GenaiCore.Llm.Processor
  • Added GenaiCore.Llm.Processor abstract interface for LLM request/response pre/post-processing.
  • Added GenaiCore.Llm.Processor.Lambda to support pre/post-processing using lambda.
• GenaiCore.Llm.AzureAi.Model
  • Added GenaiCore.Llm.AzureAi.Model to support LLMs from Azure Ai Model Catalog.
• Added support for safety_settings (only applicable to Vertex Ai models) as an option for GenaiCore.Llm.Completion#completion.
• Added optional tracing to all core components, enabled through Observability.Tracer.Config#enableTracing:
  • GenaiCore.Llm.Completion#completion
  • GenaiCore.Llm.Embedding#embedding
  • GenaiCore.Llm.Processor#preprocess and GenaiCore.Llm.Processor#postprocess
  • GenaiCore.Embedder#encodeBatch
  • GenaiCore.VectorStore#similaritySearch
  • GenaiCore.SessionStore#updateState and GenaiCore.SessionStore#getState
  • GenaiCore.Tool#call
• GenaiCore.Tool
  • Added GenaiCore.Tool abstract interface for representing a tool in an agentic flow.
  • Added GenaiCore.Tool.C3Method to support tools made from existing C3 methods, both static and member.
  • Added GenaiCore.Tool.NativePy to support tools with native Python implementations.

Copy link to this section

Updated

• py-llm and related runtimes upgraded to update the version of litellm from 1.46.1 to 1.63.11 and to include py-opentelemetry
  • py-litellm
  • py-vertexai_litellm
  • py-bedrock_litellm
• GenaiCore.BaseUser permissions expanded to be more thorough.

Copy link to this section

Fixed

• Fixed error in GenaiCore.SessionStore.Kv when the session state exceeds the kv store (e.g. Cassandra) limit for a single entry, by using filesystem-based storage when storage size is exceeded.
• Fixed concurrency issue when writing/reading different sessions in the same GenaiCore.SessionStore.Kv.

Copy link to this section

Removed

• In-memory caching for GenaiCore.Llm.Auth