Secrets Management

The rule: no human-supplied secret is ever inline in a custom resource, a chart value, or a template. Credentials reach the platform through one of three paths, selected and configured by the SecretsManagement resource.

The three sources of secrets

1. External Secrets (ESO) — pulls from an external secret store (Azure Key Vault by default, but any ESO provider) into namespaced Kubernetes Secrets.
2. Sealed Secrets — encrypted-at-rest manifests decrypted in-cluster.
3. Operator-generated — credentials the operator itself mints (LiteLLM virtual keys, Langfuse callback keys, datastore passwords). Stored as owned Kubernetes Secrets with owner references for garbage collection; you may convert them to ESO/Sealed manually.

SecretsManagement is required and selects backend (1) or (2). Workloads bind a same-namespace SecretsManagement via secretsRef and reference the materialized Secret names.

ESO providers

The operator provisions a namespaced SecretStore (not a ClusterSecretStore), keeping each namespace's credentials isolated.

Azure Key Vault (the production default)

• Workload Identity (authType: WorkloadIdentity) — production on AKS. Each consuming workload's ServiceAccount needs the azure.workload.identity/use: "true" label and a federated identity credential in Azure. The operator sets the pod/SA labels via the chart values it renders.
• Service Principal (authType: ServicePrincipal) — works anywhere, including non-AKS clusters and Kind (which has no OIDC federation for Workload Identity). Supply tenantID + authSecretRef.{clientID, clientSecret}.

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata: { name: azure-keyvault, namespace: forge-gateway }
spec:
  provider:
    azurekv:
      authType: WorkloadIdentity
      vaultUrl: https://kvforgepltprod.vault.azure.net/
      serviceAccountRef: { name: forge-sa }

The federated credential is a mandatory manual step

The operator sets up the Kubernetes side (the labelled ServiceAccount, the SecretStore, the ExternalSecrets). It does not call Azure, so it cannot create the federated identity credential that lets that ServiceAccount exchange its token for the platform managed identity. You must create one per (namespace, serviceAccount) — the AKS OIDC issuer requires an exact-match subject and does not support wildcards:

az identity federated-credential create \
  --name "eso-<namespace>-<serviceAccount>" \
  --identity-name "<managed-identity>" \
  --resource-group "<resource-group>" \
  --issuer "$(az aks show -g <resource-group> -n <cluster> --query oidcIssuerProfile.issuerUrl -o tsv)" \
  --subject "system:serviceaccount:<namespace>:<serviceAccount>" \
  --audiences "api://AzureADTokenExchange"

Until it exists, ESO's token exchange fails with Entra AADSTS70021: No matching federated identity record found …. The operator watches ESO's SecretStore status and mirrors this onto the SecretsManagement FederatedCredentialReady condition:

• Waiting — ESO has not validated the store yet.
• False / MissingFederatedCredential — the AADSTS70021 error was seen; the message names the exact subject to create a credential for.
• True — ESO authenticated; the credential is in place.

Ready still reflects that the backend was wired; FederatedCredentialReady is the specific signal for the Azure-side trust. (With authType: ServicePrincipal no federated credential is needed and this condition is not set.)

HashiCorp Vault

Lets the entire secrets path run in-cluster (no cloud Key Vault, Service Principal, or federated identity). For KV-v2, each keys value is the secret path and the Kubernetes key name is the property within it.

Any provider (raw passthrough)

provider: raw injects the ESO spec.provider block verbatim, so every ESO provider is supported — AWS, GCP, IBM, Akeyless, 1Password, Kubernetes, and any provider ESO adds in the future — with no operator changes. The externalSecrets[].data form exposes the full ESO remoteRef (remoteKey / property / version).

See the SecretsManagement reference for the exact field shapes and examples of each provider.

Azure Key Vault key matrix

When using Azure Key Vault, these are the secret names the operator expects (carried from the platform's previous chart values). SecretsManagement.eso.externalSecrets[].keys maps Kubernetes Secret keys → Key Vault secret names.

Gateway (forge-gateway)

Kubernetes Secret / key	Key Vault secret	Used for
model-credentials / OPENAI_API_KEY	gateway-foundry-api-key	Upstream model auth
model-credentials / ANTHROPIC_API_KEY	gateway-foundry-api-key	Upstream model auth
litellm-db-credentials / DATABASE_URL	(assembled from PostgresCluster)	LiteLLM DB
entra-sso-credentials / client-id,client-secret	gateway-entra-sso-client-id / -secret	UI SSO (when enableEntraSSO)
litellm-license / license-key	gateway-litellm-license-key	LiteLLM enterprise

ChatUI (forge-ui)

Kubernetes Secret / key	Key Vault secret
librechat-credentials-env / CREDS_KEY	chatui-creds-key
librechat-credentials-env / CREDS_IV	chatui-creds-iv
librechat-credentials-env / JWT_SECRET	chatui-jwt-secret
librechat-credentials-env / JWT_REFRESH_SECRET	chatui-jwt-refresh-secret
librechat-credentials-env / LITELLM_API_KEY	chatui-litellm-api-key (or an auto-minted virtual key when licensed)
forge-ui-basic-auth / htpasswd	chatui-basic-auth-htpasswd

MongoDB and Meilisearch credentials are not sourced from Key Vault for ChatUI. They are owned by the datastore resources: the Meilisearch master key by the MeilisearchInstance, and the Mongo SCRAM password + per-database MONGO_URI by the MongoCluster.

Langfuse (forge-langfuse)

nextauth-secret + salt are auto-generated and rotated by the langfuse-operator into a <instance>-generated-secrets Secret — do not source these from Key Vault unless overriding. Datastore credentials come from the referenced datastore resources' materialized Secrets (plus Key Vault for external datastores).

Operator-generated secrets

• LiteLLM virtual keys (ChatUI auto-wiring) and master/salt keys (autoGenerate: true).
• CloudNativePG role passwords (surfaced as a per-database credentials Secret).
• The MongoCluster SCRAM password and per-database MONGO_URI Secrets.
• The MeilisearchInstance master-key Secret.

All operator-owned Secrets get owner references for garbage collection and a managed-by=navique-ai-core-operator label.

Rotation

With the credential-rotation feature and a per-resource rotation block (default interval 90 days), the operator rotates the credentials it owns — LiteLLM virtual keys, and generated ClickHouse / Redis / Mongo / Meilisearch passwords — re-stamping a core.navique.com/rotated-at annotation.

Out of scope (rotated by their own systems): Key Vault secrets (rotate in KV; ESO re-syncs) and CNPG-managed role passwords (rotate via CNPG). The operator never rotates credentials it does not own. Without the feature, the rotation block is ignored.

Migrating off plaintext

Rotate the previously-committed secrets

The platform's earlier Helm charts committed live secrets to git (model API keys, Entra SSO client secret, LiteLLM license, database URLs, Langfuse secrets). These are compromised by definition and must be rotated as part of migration — do not copy any of them into the operator. Rotate every one in Key Vault, source them via ESO, and scrub git history if feasible.

Don'ts

• No secret values in samples — use placeholders and a note to populate the vault.
• No secret logging. Redact connection strings in events and conditions.
• No secrets baked into the operator image (the license public key is fine; never the private key).