> ## Documentation Index > Fetch the complete documentation index at: https://docs.firstflow.app/llms.txt > Use this file to discover all available pages before exploring further. # AI Providers > How FirstFlow resolves the LLM credentials that power widget composition, classification, and flow generation per-organization keys, Cloudflare Gateway routing, and environment fallback. Several FirstFlow features call an LLM at runtime: guide widget composition, branch/classify decisions, flow generation, and the internal analytics evaluators. None of them embed a key. Instead each call asks a resolver for credentials, and the resolver walks a fixed precedence chain a per-node override, then your organization's configured provider, then a Cloudflare AI Gateway, then a server environment variable. This page explains that chain: where keys live, how they are encrypted, which features consume which provider, and what happens when nothing is configured. Configuring a provider is a dashboard action; understanding the resolution order is what lets you reason about why a feature picked the model it did. ## How resolution works A provider is an organization-scoped credential row, not a global setting. FirstFlow recognizes exactly three provider names `openai`, `anthropic`, and `cloudflare-gateway` and a feature never reaches for a raw key directly. It calls `AiProviderResolverService`, which resolves the right client for the organization that owns the agent in play. The order is deterministic. The resolver prefers a directly configured provider; failing that it routes through a Cloudflare AI Gateway you configured (the gateway holds the upstream key and FirstFlow authenticates to *it*); failing that and only for the Anthropic family it falls back to the server's `ANTHROPIC_API_KEY` environment variable. Branch/classify nodes add one more rung at the very top: a per-node API key, kept for explicit overrides and legacy nodes. ```mermaid theme={null} flowchart TD A["Feature needs an LLM
(resolves by agentId → organizationId)"] --> B{"Per-node
API key?"} B -- "yes (classify nodes only)" --> KEY["Use that key"] B -- "no" --> C{"Org has a direct
provider configured?"} C -- "yes" --> D["Use openai / anthropic key"] C -- "no" --> E{"Org has a
Cloudflare AI Gateway?"} E -- "yes" --> F["Route via gateway baseURL
(cf-aig-authorization header)"] E -- "no" --> G{"ANTHROPIC_API_KEY
in env?"} G -- "yes (anthropic family only)" --> H["Use env key"] G -- "no" --> I["Return null → feature
degrades gracefully"] classDef ok fill:#dcfce7,stroke:#16a34a,color:#14532d; classDef route fill:#fef9c3,stroke:#ca8a04,color:#713f12; classDef stop fill:#fee2e2,stroke:#dc2626,color:#7f1d1d; class KEY,D,H ok; class F route; class I stop; ``` Resolution is keyed to the **organization**, derived from the agent. A feature passes an `agentId`; the resolver looks up the agent's `organizationId` and reads that organization's enabled provider rows. One organization's keys never serve another's traffic. When the chain finds nothing, the resolver returns `null` rather than throwing. The calling feature treats that as "no AI available" and degrades the rest of the platform keeps running. That is why AI features quietly do nothing on a fresh self-host with no key set, instead of erroring. ## How credentials are stored The secret half of a provider the API key is encrypted with **AES-256-GCM** before it touches Postgres, in the `encryptedAuth` column. The non-secret half (a Cloudflare account id, gateway id, and manual model list) is stored as plaintext JSON in `config`. The encryption key comes from `INTEGRATION_ENCRYPTION_KEY` (a 64-hex-character string). If that variable is missing or malformed, FirstFlow auto-generates an ephemeral key and logs a warning fine for local development, but stored secrets become unreadable across restarts, so set a stable key in production. Decrypted credentials never leave the server. The public HTTP shape (`AiProviderPublic`) exposes `hasAuth: boolean` instead of the key, and the resolver's decrypted view is marked internal-only. When you read a provider back from the API you get its metadata and whether a key is present never the key itself. ## Configuring a provider Open **Settings → AI Providers** in the dashboard, choose a provider, and paste its key. FirstFlow validates the credential live before saving: it calls the provider's real models endpoint (`https://api.anthropic.com/v1/models` for Anthropic, `https://api.openai.com/v1/models` for OpenAI) and rejects the save with a `400` if the key is unusable. A successful save returns the safe public view. Under the hood this is a thin REST surface, all scoped to your organization via your dashboard session: ```http theme={null} GET /ai-providers → list configured providers POST /ai-providers → add a provider (validates the key) PUT /ai-providers/:id → update key / config / default / enabled DELETE /ai-providers/:id → remove a provider GET /ai-providers/:provider/models → live model list (cached 24h) ``` Responses use the standard `{ data }` envelope. Adding a second provider of the same name returns `409 Conflict` one row per provider name per organization. Pick **Anthropic**, **OpenAI**, or **Cloudflare AI Gateway** and paste the key. The create call shape is the same for the direct providers only `auth.apiKey` matters: ```json theme={null} POST /ai-providers { "provider": "anthropic", "displayName": "Production Anthropic", "auth": { "apiKey": "sk-ant-…" }, "isDefault": true } ``` The save fails fast with `400` if the key cannot list models, so a stored provider is always one that worked at least once. For OpenAI and Anthropic the model list is pulled live from the provider's API and cached for 24 hours per organization, so dropdowns reflect what your key can actually reach. OpenAI ids containing `dall-e` are typed as `image`; everything else is `text`. On the relevant editors (for example a classify node, covered below) choose the provider family and one of its live models. Composition and generation use Anthropic automatically see [What each feature uses](#what-each-feature-uses). ### Cloudflare AI Gateway The gateway is a routing front-end, not a model vendor. It tunnels to sub-providers, so its configuration carries an account id, a gateway id, and a **manually maintained** model list Cloudflare cannot enumerate models over its API, so `GET /ai-providers/cloudflare-gateway/models` returns exactly the list you saved. Every model id in that list must be prefixed with its sub-provider, and the save is rejected otherwise. The resolver builds the upstream base URL from your account and gateway ids and authenticates with a `cf-aig-authorization: Bearer ` header the actual OpenAI/Anthropic key lives inside Cloudflare, not in FirstFlow. ```json theme={null} POST /ai-providers { "provider": "cloudflare-gateway", "auth": { "apiKey": "" }, "config": { "accountId": "", "gatewayId": "", "models": [ { "modelId": "anthropic/claude-3-5-sonnet", "modelName": "Sonnet 3.5", "modelType": "text" }, { "modelId": "openai/gpt-4o", "modelName": "GPT-4o", "modelType": "text" } ] } } ``` An unprefixed id like `gpt-4o` is rejected with a message telling you to write `openai/gpt-4o`. Because the gateway can route either family, configuring it alone makes both the OpenAI and Anthropic families available to features that pick a family the resolver strips the prefix when it needs a bare model id. ## What each feature uses The features split into two groups: ones that always use Anthropic, and ones where you pick the family per node. The provider/model column below is the runtime default, not a hard limit. | Feature | Provider | Notes | | ------------------------------------------------------- | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | Guide widget composition | Anthropic `claude-sonnet-4-6` | Renders a `WidgetTree` from the flow walker's instruction; validated against the widget schema in [`@firstflow/widget-kit`](/widget-kit/overview). | | Flow generation & URL import | Anthropic | Prompt-to-flow and "import from URL" in the [authoring](/features/authoring) tools. Throttled per organization. | | Branch / classify nodes | `openai` or `anthropic`, your model | AI-resolved branching for decision/classify nodes see [Flows & nodes](/concepts/flows-and-nodes). | | Analytics evaluators, topic/intent/sentiment, slot-fill | `openai` or `anthropic` | Single-shot prompt-to-JSON. Resolves org creds; Anthropic-only env fallback. | Composition and generation go through the Anthropic-compatible client the resolver returns, so a Cloudflare Gateway with an `anthropic/…` model satisfies them too. The feature asks the resolver for an Anthropic client; whether that client points at `api.anthropic.com` or at your gateway is transparent to the feature. ### Branch and classify nodes A classify node resolves a branch by calling an LLM with the user's projected context and the branch labels, then matching the model's answer back to a branch id. Its config is stored per node in the agent's `sdkSettings` and carries a provider family (`openai` or `anthropic`), a model, free-text `instructions`, and tuning knobs: * `confidenceFloor` (default `0.5`) answers below this route to the fallback branch. * `resolutionMode` `blocking` waits for the decision; `optimistic` proceeds and corrects. * `ttlMs` (default `300000`) how long a resolved decision is reused. * `fallbackBranchId` where unmatched / low-confidence / errored decisions land. * `signalMask` which context signals (message, metrics, behaviors, session, memory) are projected to the model. A fresh, unconfigured node does not default blindly to OpenAI. FirstFlow inspects which providers your organization has configured and defaults the node to a family it can actually resolve, picking the first live model for that family. This prevents the silent failure where a node opens on a provider the runtime cannot satisfy and then routes *every* classification to the fallback branch. If the node's model fails at resolve time bad model, provider outage, an 8-second timeout the decision falls back to `fallbackBranchId` with `confidence: 0` rather than erroring the flow. The credential precedence for these nodes is the full chain with the per-node key on top: an explicit per-node key wins, otherwise the resolver uses the organization's provider or gateway. There is no env fallback for a node that names `openai`; the env fallback only ever covers the Anthropic family. ## Self-hosting and environment fallback Dashboard-managed providers are the primary path and work identically on a self-host. The environment variables are the floor beneath them useful for a single-tenant deployment where you would rather set one key than click through the UI. Set `ANTHROPIC_API_KEY` to give every organization an Anthropic fallback for composition, generation, and the analytics evaluators when they have no provider row. `OPENAI_API_KEY` exists for parity, but note the resolver's env fallback is **Anthropic-only**: a classify node configured for `openai` with no organization provider will not pick up `OPENAI_API_KEY` configure an OpenAI provider in the dashboard instead. Set a stable `INTEGRATION_ENCRYPTION_KEY` so encrypted keys survive restarts. ```bash theme={null} ANTHROPIC_API_KEY=sk-ant-… # env fallback for the Anthropic family OPENAI_API_KEY=sk-… # parity; not used by the env fallback path INTEGRATION_ENCRYPTION_KEY=$(openssl rand -hex 32) # 64 hex chars; encrypts stored keys ``` With none of these set and no provider configured, AI features return nothing and the platform keeps serving non-AI traffic. The full variable reference lives in [Configuration](/self-hosting/configuration). ## Next Choose a model per node in [Flows & nodes](/concepts/flows-and-nodes), see what composition produces in [Widgets & blocks](/concepts/widgets-and-blocks), or set the underlying keys when [self-hosting](/self-hosting/configuration).