LLM Service Package Documentation
Point any C3 or Nexus consumer at the llmService gateway and call models through one OpenAI-compatible endpoint and virtual key, instead of wiring each application to a provider directly. Administrators register the available models and provider credentials once; consumers then pick a model and call it. Usage, spend, and request traces are visible in one place, and new provider models become available without a platform upgrade. Use it whenever an application or agent must call an LLM or an embedding model.
Dependencies
| Dependent Package | Purpose of this Package |
|---|---|
nexusBase | Base CloudService, app-start hooks, and shared utilities. |
Key types in this package
| Type | What you use it for |
|---|---|
| LlmService | Entry point for the centralized LLM service. |
| LiteLlmProxy.CloudService | The OpenAI-compatible gateway your consumers call. |
| LiteLlmProxy.ModelRegistry | Register and list the models the gateway exposes. |
| LiteLlmProxy.CredentialRegistry | Register the provider credentials that back the virtual keys. |
| LaminarCloudService | Observability service for traces, metrics, and logs of LLM calls. |
Key files in this package
The package is structured in the following manner:
Text
.
└── llmService/
├── config
├── metadata
├── resource
├── seed
├── test
└── src/
└── litellm| Folder name | Content |
|---|---|
config | Package configuration files. |
metadata | Seed roles and package metadata. |
resource | Runtime resources such as packaged Helm charts. |
seed | Seed data for initializing the service (models, providers). |
test | Tests per runtime for the proxy and observability types. |
src/litellm | LiteLLM proxy CloudService, model registry, credential registry, and clients. |