Stack

Scope: namespaced · Optional umbrella · Licensed (managed-deployment feature + a stacks cap)

Stack is an optional umbrella resource that creates and owns a curated bundle of the granular resources, auto-wires their references, and aggregates progress — an ARM-deployment-style one-shot provisioning experience.

It does not replace the per-component resources. The granular CRDs remain a first-class way to compose the platform by hand; Stack simply sits on top for teams that want a single object to manage a whole environment.

When to use it

• Use Stack when you want to provision and track a full environment (secrets
  • datastores + gateway + observability + UI) as one unit, with the operator wiring the cross-references for you and rolling status up into one place.
• Use the granular resources when you want fine-grained control, are composing resources across namespaces by hand, or are sharing datastores between several independently-managed workloads.

What it does

• Creates + owns the bundled components (the operator sets owner references so the whole bundle is garbage-collected with the Stack).
• Auto-wires the references between them (Gateway→Postgres, Gateway→Observability, ChatUI→Gateway, workloads→datastores) — this relies on the auto-wiring capability.
• Aggregates progress into status.components[] and status.endpoints, so a single kubectl get stack shows the whole environment's readiness and URLs.

Drift policy

By default the Stack continuously enforces the curated spec on its children: any manual edit you make to a generated component CR (for example bumping a datastore's storage size or a workload's replica count) is reverted on the next reconcile. spec.driftPolicy lets you relax this:

driftPolicy	Behaviour
Enforce (default)	The curated spec is reasserted on every reconcile — manual child edits are reverted (full drift correction).
Adopt	The curated spec is applied on creation and whenever the Stack's own spec changes (an ARM-style "redeploy"). Between Stack edits, manual per-component edits are preserved.

Adopt mirrors how an Azure Resource Manager deployment behaves: the Stack seeds and wires the resources, but afterwards you can tune each component directly. Editing the Stack itself reasserts every child and overwrites your drift, so treat a Stack edit as a redeploy.

The owner reference is kept under both policies, so deleting the Stack always cascades to (and tears down) every component it created.

apiVersion: core.navique.com/v1alpha1
kind: Stack
metadata:
  name: forge
spec:
  driftPolicy: Adopt   # let users tune individual components after creation
  # ...

Datastore sizing

Each managed datastore under spec.datastores takes an optional resources shorthand that sizes the underlying container's CPU and memory. It is honored for the Postgres, ClickHouse, Redis, and MongoDB datastores (managed mode only), and is fanned into the emitted PostgresCluster / ClickHouseCluster / RedisInstance / MongoCluster child as its managed.resources.

Field	Maps to
cpu	CPU request
memory	memory request
memoryLimit	memory limit

The shorthand deliberately has no CPU limit (CPU limits throttle rather than protect). If you need one, use the standalone datastore CRD directly, which takes a full managed.resources.

apiVersion: core.navique.com/v1alpha1
kind: Stack
metadata:
  name: forge
spec:
  datastores:
    postgres:
      storageSize: 20Gi
      resources: { cpu: 500m, memory: 1Gi, memoryLimit: 2Gi }
    clickhouse:
      storageSize: 50Gi
      resources: { cpu: 500m, memory: 2Gi, memoryLimit: 3Gi }
    redis:
      storageSize: 5Gi
      resources: { cpu: 100m, memory: 256Mi, memoryLimit: 512Mi }
    mongo:
      storageSize: 20Gi
      resources: { cpu: 250m, memory: 1Gi, memoryLimit: 2Gi }

When resources is omitted the upstream chart defaults apply (ClickHouse keeps its operator-tuned 3Gi memory limit to avoid OOMs under Langfuse load).

Choosing the gateway implementation

spec.gateway.type selects the gateway product: litellm (default) or waeg (the Wäg gateway, via the waeg-operator). See Gateway → Wäg gateway for what differs.

With type: waeg the Stack:

• requires datastores.clickhouse.enabled in the default split storage mode — admission rejects the Stack otherwise, because that is where Wäg's analytics live. Set gateway.waeg.storageMode: single to keep them in the Stack's Postgres instead, which drops the requirement entirely — see Wäg analytics storage;
• auto-wires the Stack's own ClickHouse (and Redis, when enabled) as the gateway's second storage plane — in split mode only;
• declares the waeg database on that ClickHouse so the gateway can start at all: Wäg creates only its tables, never its database (ClickHouseCluster → databases[]);
• names the gateway's logical Postgres database waeg rather than litellm, so the two implementations never share a schema;
• attaches no Langfuse ref — Wäg cannot export traces declaratively, so the Stack does not wire one only for the Gateway to refuse it.

spec.gateway.waeg passes the Wäg-only knobs through: topology, jobWorkers, worker, storageMode, openfga, autoscaling, bootstrapAdmin, dataEncryptionKeySecretRef, modelAccess, scim and jwt. The Enterprise licence is declared once, on spec.gateway.licenseSecretRef, and the gateway's organization on spec.gateway.organization — which a type: waeg gateway needs before a ChatUI can attach to it.

spec:
  datastores:
    clickhouse: { enabled: true, backup: { enabled: true, provider: s3, s3: { destinationPath: s3://b } } }
    redis:      { enabled: true }
  gateway:
    enabled: true
    type: waeg
    licenseSecretRef: { name: waeg-enterprise-license, key: license }
    waeg:
      topology: Split
      worker: { enabled: true, replicas: 2 }
      scim:
        enabled: true
        tokenSecretRef: { name: waeg-scim-token, key: scim-token }
        defaultRole: viewer
      jwt:
        appClaim: azp
        requireRegisteredApplication: true

Fields below that are LiteLLM-specific (rolePermissions, healthCheck, storePromptsInLogs, guardrails, per-model limits — and the claim-mapping half of jwtAuth) are reported on the Gateway child's FeaturesSupported condition when type: waeg — they are never silently dropped.

Wäg analytics storage

spec.gateway.waeg.storageMode selects where the gateway's analytics (request logs, usage, spend) live. It is the Stack passthrough of Gateway.spec.waeg.storageMode.

Mode	Analytics live in	datastores.clickhouse.enabled
split (default)	the Stack's ClickHouse	required
single	the Stack's Postgres	not needed — no ClickHouse is provisioned

Upstream Wäg's own default is single, and it is the recommended starting point; this operator keeps split as its default for continuity and because it holds up at volume. With single the Stack provisions no ClickHouse at all and the admission rule that demanded one no longer applies. Leaving the default in place without a ClickHouse is rejected with:

gateway.type=waeg with the default storageMode 'split' requires
datastores.clickhouse.enabled; set gateway.waeg.storageMode: single to keep
analytics in Postgres instead

Switching modes does not migrate analytics history

The two planes are separate stores. Changing storageMode on a running Stack points the gateway at the other one; the analytics already collected stay where they were, and neither the operator nor Wäg copies them across.

apiVersion: core.navique.com/v1alpha1
kind: Stack
metadata:
  name: forge
  namespace: forge
spec:
  datastores:
    postgres:
      enabled: true
      storageSize: 20Gi
      backup: { enabled: true, provider: s3, s3: { destinationPath: s3://forge-pg } }
    redis: { enabled: true }
    # no clickhouse block at all — storageMode: single needs none
  gateway:
    enabled: true
    type: waeg
    organization: { name: forge }
    waeg:
      storageMode: single

In the default split mode the Stack additionally declares the waeg database on its managed ClickHouse, because Wäg creates only its tables and never its database — see ClickHouseCluster → databases[].

Gateway organization

spec.gateway.organization creates the gateway's top-level organization — the object budgets and rate caps roll up to. It is passed straight through to the child Gateway's spec.organization.

Field	Description
name (required)	The organization name
maxBudget	Spend cap for the organization
budgetDuration	Period the cap resets over (e.g. 30d). LiteLLM only — Wäg budget policies have no period field
rpmLimit	Requests-per-minute cap
tpmLimit	Tokens-per-minute cap

On a type: litellm gateway an organization stays optional — teams, keys and models all work without one.

A Wäg gateway needs one before a ChatUI can attach

With type: waeg this field is required to auto-wire a ChatUI. Wäg has no standalone virtual key: a key belongs to an application, and applications are org-rooted — so with no organization there is nothing to mint the ChatUI's credential under. Before this field existed, a Stack with gateway.type: waeg and chatUI.enabled: true left the ChatUI stuck on

Gateway "…" is type=waeg and has no spec.organization: a Wäg virtual key belongs
to an application, and applications are org-rooted

with nothing settable from a Stack to satisfy it — the granular Gateway had spec.organization, the Stack had no passthrough. Set it here and the ChatUI is registered as a WaegApplication under that organization, with its WaegVirtualKey minted against it.

The organization is also what spec.gateway.waeg.modelAccess grants hang off: the model ACL is declared against it, so the grants are ignored without one.

spec:
  datastores:
    clickhouse: { enabled: true, backup: { enabled: true, provider: s3, s3: { destinationPath: s3://forge-ch } } }
    redis:      { enabled: true }
  gateway:
    enabled: true
    type: waeg
    licenseSecretRef: { name: waeg-enterprise-license, key: license }
    organization:
      name: navique-ag
      maxBudget: 2000
      rpmLimit: 1000
      tpmLimit: 500000
    waeg:
      modelAccess:                  # org-rooted: needs the organization above
        openByDefault: false
        fallbackMode: deny
        grants:
          - models: [premium, economy]
  chatUI:
    enabled: true                   # auto-wired as an application under navique-ag

See ChatUI connects as an application for the objects the operator emits, and What is not applied for the organization fields Wäg cannot honour.

Gateway Enterprise licence

spec.gateway.licenseSecretRef supplies the gateway's own Enterprise licence, passed through to the Gateway's instance.licenseSecretRef. For type: litellm that is the LiteLLM Enterprise key; for type: waeg it is the Wäg Enterprise licence, and it is the difference between a working Enterprise gateway and one that answers 402 license_required.

Every Wäg Enterprise module is gated on it — SSO, SCIM, audit, CMEK, FIPS and console branding — and enforcement is unconditional, so without this field a Stack-managed Wäg gateway has no reachable Enterprise feature at all.

spec:
  gateway:
    type: waeg
    licenseSecretRef: { name: waeg-enterprise-license, key: license }

The Secret key defaults to license. Let the Stack's SecretsManagement backend materialize the Secret rather than creating it by hand.

Branding follows the licence

Once a Wäg gateway is licensed, the operator applies the built-in Navique theme pack to its console automatically (an owned <gateway>-branding ConfigMap). The Stack has no opt-out field for it — to set your own pack, or to keep Wäg's own livery, use the granular Gateway with waeg.brandingConfigMapRef / waeg.defaultBranding: false.

Gateway sizing

spec.gateway.resources sizes the LiteLLM proxy container. Unlike the datastore shorthand it is a full Kubernetes ResourceRequirements (so you can set a CPU limit), passed straight through to the emitted Gateway's instance.resources → the LiteLLMInstance's spec.resources. Omit it to leave the litellm-operator's own default.

spec:
  gateway:
    resources:
      requests: { cpu: "500m", memory: 512Mi }
      limits:   { cpu: "2", memory: 2Gi }

Pinning the gateway proxy build

spec.gateway.image pins the LiteLLM proxy image, passed through to the emitted Gateway's instance.image → the LiteLLMInstance's spec.image.

spec:
  gateway:
    image:
      repository: ghcr.io/berriai/litellm
      tag: v1.94.0-dev.2

Leave it unset unless you are qualifying a specific build. With no pin the litellm-operator applies its own default tag — the one validated against its database-migration entrypoint — so an arbitrary tag can break the migration and leave the gateway down.

Do not try to patch the child Gateway by hand instead: with the default drift policy Enforce the Stack re-applies its children on every reconcile and your edit is reverted within seconds. Declare the pin here.

Gateway JWT auth & RBAC

spec.gateway.jwtAuth and spec.gateway.rolePermissions enable JWT-based API authentication and per-role model access on the Stack's Gateway — passed straight through to the Gateway's instance.jwtAuth / instance.rolePermissions. See JWT API authentication & RBAC for the fields. Requires a LiteLLM Enterprise license.

spec:
  gateway:
    jwtAuth:
      enabled: true
      userRolesJWTField: roles
      userAllowedRoles: ["basic_user"]
      enforceRBAC: true
    rolePermissions:
      internal_user: { models: ["anthropic-claude"] }

Wäg console branding

spec.gateway.waeg.brandingConfigMapRef and defaultBranding pass through to the Gateway's console branding: the Navique theme by default, your own theme pack with the licensed custom-branding feature, or Wäg's own look with defaultBranding: false.

On a Wäg gateway

spec.gateway.jwtAuth also turns on JWT auth for type: waeg — enabled, publicKeyURL (→ Wäg's jwksUrl), issuer, audience and userIDJWTField (→ subjectClaim) all apply. The Wäg-only knobs live in spec.gateway.waeg.jwt, and spec.gateway.rolePermissions plus the claim mappings (userRolesJWTField, userRoleJWTField, userAllowedRoles, enforceRBAC, userIDUpsert, teamIDsJWTField, adminJWTScope) are reported as unsupported instead: Wäg authorizes through OpenFGA and registered applications. See Data-plane JWT authentication.

spec:
  gateway:
    type: waeg
    licenseSecretRef: { name: waeg-enterprise-license, key: license }
    jwtAuth:
      enabled: true
      publicKeyURL: https://login.microsoftonline.com/<tenant>/discovery/v2.0/keys
      issuer: https://login.microsoftonline.com/<tenant>/v2.0
      audience: <client-id>
      userIDJWTField: sub
    waeg:
      jwt:
        appClaim: azp
        requireRegisteredApplication: true

Hardening switches can stop the gateway booting

waeg.jwt.allowMasterKey: false or allowVirtualKeys: false require jwtAuth.enabled: true, a publicKeyURL, an audience and insecureSkipVerify: false — otherwise Wäg refuses to start. allowVirtualKeys: false also cuts off every application key, the Stack's ChatUI included.

Wäg SCIM provisioning

spec.gateway.waeg.scim turns on Wäg's SCIM v2 endpoints on the Stack's gateway, so your IdP provisions and deprovisions console users directly. It is independent of SSO — it works with no interactive login wired — and needs spec.gateway.licenseSecretRef.

spec:
  gateway:
    type: waeg
    licenseSecretRef: { name: waeg-enterprise-license, key: license }
    waeg:
      scim:
        enabled: true                                              # default
        tokenSecretRef: { name: waeg-scim-token, key: scim-token }  # required
        defaultRole: viewer
        orgSource: waeg
        roleMap: { "AI Platform Admins": admin }

The bearer token is projected onto the gateway pods as WAEG_EE_SCIM_TOKEN, so rotating the Secret is the whole rotation procedure. The endpoints live under /waeg/admin/v1/ee/scim/v2. See SCIM provisioning for every field.

ChatUI datastores: MongoDB & Meilisearch

MongoDB (the conversation store) and Meilisearch (search) back the ChatUI and are configured under datastores.mongo / datastores.meilisearch — the same mode: managed | external switch as the other datastores. They are only provisioned when the ChatUI is enabled, and default to managed.

• managed (default): the Stack provisions and owns a MongoCluster / MeilisearchInstance and wires the ChatUI to it. Size via storageSize (Meilisearch also honors storageClass / resources).
• external: no datastore CR is provisioned; the ChatUI is wired to a user-managed instance. Mongo needs connectionSecretRef (Secret key MONGO_URI, override via .key). Meilisearch needs host plus a connectionSecretRef holding the master key (Secret key MEILI_MASTER_KEY).

spec:
  datastores:
    mongo:
      mode: managed
      storageSize: 10Gi
    meilisearch:
      mode: external
      host: https://search.example.com
      connectionSecretRef: { name: meili-master-key }   # key MEILI_MASTER_KEY

MCP servers (web search & tools)

chatUI.mcp wires MCPServer servers into the Stack's LibreChat UI. Two ways, combinable:

• catalog — bundled servers the Stack provisions and owns as MCPServer children (one per key) and wires into the ChatUI. First entry: websearch (enterprise web search). Requires the bundled-mcp-catalog license; an unlicensed entry is Refused and skipped (the UI still comes up).
• refs — wire pre-existing MCPServer CRs you create yourself (catalog or external — e.g. a self-hosted in-cluster MCP server), in any namespace. The Stack references them without owning their lifecycle.

spec:
  chatUI:
    enabled: true
    mcp:
      catalog: [ websearch ]                 # Stack creates + wires the MCPServer
      refs:
        - { name: my-internal-mcp }          # an MCPServer you manage

Referencing any MCP server auto-enables LibreChat's Agents endpoint. The Stack's MCPServer children appear in its progress tree and are torn down with the Stack. See MCPServer for the server definition and the automatic SSRF allowlist.

Guardrails (data-path)

gateway.guardrails wires Guardrail data-path guardrails into the Stack's Gateway. Two ways, combinable:

• catalog — bundled engines the Stack provisions and owns as Guardrail children (one per key) and wires into the Gateway. First entry: pseudonymizer (PII pseudonymization). Requires the guardrail (Enterprise) license; an unlicensed entry is Refused and skipped (the Gateway still comes up, unguarded).
• refs — wire pre-existing Guardrail CRs you create yourself (catalog or external HTTP guardrail), in any namespace. The Stack references them without owning their lifecycle.

spec:
  gateway:
    enabled: true
    guardrails:
      catalog: [ pseudonymizer ]             # Stack creates + wires the Guardrail
      refs:
        - { name: my-guardrail }             # a Guardrail you manage

Both catalog and external guardrails are licensed and routed through the operator-injected license-gate proxy (fails closed if the license lapses). The Stack's Guardrail children appear in its progress tree and are torn down with the Stack. See Guardrail for the full definition.

Debugging a guardrail inside a Stack

Set logLevel (and optionally blockedReason) on the Stack, not on the child:

spec:
  gateway:
    guardrails:
      catalog: [ pseudonymizer ]
      logLevel: debug                        # one proxy log record per request
      blockedReason: "blocked by the Navique license gate"

A Stack rewrites each child's curated spec, so under the default driftPolicy: Enforce a kubectl patch of the child Guardrail's logLevel is reverted — and since the Stack watches its Guardrail children, the patch triggers the very reconcile that undoes it. (With driftPolicy: Adopt a manual patch survives, but only until the next edit to the Stack, and it switches off drift correction for every child.) These fields apply to the catalog children the Stack owns; guardrails wired via refs are yours, so set logLevel on those CRs directly.

The proxy's decision counters need no opt-in at all — see Troubleshooting a failed guardrail call.

Backups & deletion protection

Because a Stack is the path non-technical users take, it treats stateful data defensively.

Backups are mandatory for managed stateful datastores. A Stack must set an enabled backup on every managed datastores.postgres, datastores.clickhouse, and datastores.mongo (the same set that is auto-Locked) — applying one without it is rejected (Postgres at admission via CEL; ClickHouse/Mongo at reconcile), so nobody stands a datastore up with no backups. Redis (cache) and Meilisearch (a rebuildable index) are exempt. The block is the same provider-neutral shape everywhere (s3/azure/gcs + schedule + retention) and is fanned into the managed child; Postgres runs CNPG-native PITR, ClickHouse a clickhouse-backup CronJob, MongoDB a mongodump+rclone CronJob. External datastores need no backup here — you manage that yourself. Set datastores.requireBackup: false to skip the requirement (advanced users / CI).

Managed datastores are auto-Locked. When the Stack provisions a managed PostgresCluster, ClickHouseCluster, or MongoCluster (which holds the ChatUI conversation history), it also creates a Lock on it, so a mistaken kubectl delete is blocked. Redis (a cache) and Meilisearch (a rebuildable search index) are never locked.

• This is deletion protection at the data layer: deleting the Stack stays blocked until you remove those Locks — the datastores (and their PVCs) are never destroyed by accident. The Stack reports the blocking Lock(s) in its status.
• The Stack creates each Lock once and never recreates one you remove — so to delete a protected Stack you simply kubectl delete lock <name> first, then delete the Stack. The operator won't race you by putting the Lock back.
• Set datastores.protectData: false to skip creating the Locks (existing Locks are left untouched).

spec:
  datastores:
    postgres:
      storageSize: 20Gi
      backup:
        enabled: true
        provider: s3
        s3:
          destinationPath: s3://forge-backups/pg
          credentialsSecretRef: { name: pg-backup }
    clickhouse: { storageSize: 20Gi }
    redis: { storageSize: 5Gi }
    # protectData: false   # opt out of the auto-Locks (not recommended)

Namespace separation

By default every component lands in the Stack's own namespace. Each component block optionally takes a namespace to place it elsewhere:

spec:
  gateway:       { namespace: forge-gateway }
  chatUI:        { namespace: forge-ui }
  observability: { type: langfuse }   # stays in the Stack's namespace

When a component declares a namespace, the Stack:

• creates the namespace if it does not exist (and leaves it in place on delete);
• places a SecretsManagement in every namespace that hosts a workload, so each component's same-namespace secretsRef resolves locally;
• auto-wires the cross-namespace references (e.g. a Gateway in one namespace to the Postgres/Langfuse in another) — you still set no refs by hand.

Because Kubernetes owner references cannot cross namespaces, a cross-namespace child is tracked by the core.navique.com/owned-by label and removed by the Stack's finalizer on delete (same-namespace children keep the owner-reference cascade). The auto-created namespaces themselves are not deleted.

SSO / OIDC login

spec.sso configures single sign-on across the whole Stack from one identity provider. The shared issuer/provider/scopes apply to every enabled component, while each component gets its own OAuth client (callback URLs differ) via a per-component client ref. A component receives SSO only when it is enabled and its client ref is set, so you can roll SSO out selectively.

Field	Description
issuerURL	Shared OIDC issuer / discovery base URL
providerName	Shared display label
scopes	Shared requested scopes
provider / tenantID / authorizationEndpoint / tokenEndpoint / userinfoEndpoint	Gateway-only provider flavor + endpoints (see the Gateway SSO notes)
gateway / chatUI / observability	Per-component OAuth client refs ({ name, clientIDKey?, clientSecretKey? })

spec:
  sso:
    issuerURL: https://idp.example.com
    providerName: "Acme SSO"
    provider: generic-oidc
    authorizationEndpoint: https://idp.example.com/authorize
    tokenEndpoint: https://idp.example.com/token
    userinfoEndpoint: https://idp.example.com/userinfo
    gateway:       { name: gateway-oidc }
    chatUI:        { name: chatui-oidc }
    observability: { name: langfuse-oidc }

This is what powers self-service SSO from the management console's deployment wizard. Per-component details (which env each app receives, licensing) are on the Gateway, ChatUI, and Observability pages.

Mesh

spec.mesh.mode opts the Stack's component namespaces into the cluster ServiceMesh (mTLS + isolation). It defaults to auto (join when a Ready ServiceMesh exists, no-op otherwise); enabled requires one; disabled never joins. The Stack only labels its namespaces — the ServiceMesh singleton does the enrollment — so a mesh enabled later joins existing Stacks with no redeploy.

Licensing

Stack requires the managed-deployment feature and a stacks instance cap in the License. Without them, compose the platform with the granular resources, which are always available (subject to their own caps in the Community edition).

See Editions & Licensing.