> For the complete documentation index, see [llms.txt](https://docs.zerve.ai/self-managed/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zerve.ai/self-managed/configuration/llm-providers.md).

# LLM Providers

Zerve's AI features — the agent (chat, code generation, planning), superagents, AI blocks in notebooks — are served by an LLM provider of your choice.

Supported providers:

* Anthropic
* OpenAI
* Google Gemini
* Amazon Bedrock
* OpenRouter

Any OpenAI-compatible gateway is also supported, for installs that cannot reach provider APIs directly. See [LLM gateway](#llm-gateway).

Credentials can be supplied at two levels: install-wide, in the Helm chart, and per organization, in the application UI. An organization's own key always takes precedence over the install-wide one for that provider.

## Install-level keys

Provider keys are read by the `canvas` service and apply to every organization on the install:

```yaml
# my-values.yaml
canvas:
  workloads:
    common:
      secrets:
        ZERVE_AGENT_ANTHROPIC_API_KEY:
          secretName: zerve-llm-keys
          key: anthropic_api_key                  # optional, defaults to ZERVE_AGENT_ANTHROPIC_API_KEY
      env:
        DEFAULT_AGENT_AI_PROVIDER: "anthropic"    # optional, this is the default
```

| Provider      | Variable                         |
| ------------- | -------------------------------- |
| Anthropic     | `ZERVE_AGENT_ANTHROPIC_API_KEY`  |
| OpenAI        | `ZERVE_AGENT_OPENAI_API_KEY`     |
| Google Gemini | `ZERVE_AGENT_GEMINI_API_KEY`     |
| OpenRouter    | `ZERVE_AGENT_OPENROUTER_API_KEY` |

Amazon Bedrock takes no key — see [Amazon Bedrock](#amazon-bedrock).

Configure at least the provider named by `DEFAULT_AGENT_AI_PROVIDER` (`anthropic`, `openai`, `bedrock`, `openrouter` or `gemini`). It is the provider Zerve uses for an organization that has not approved one of its own. Keys for several providers can be set at once; users then select between the ones their organization approves.

Set these on `workloads.common` rather than a single workload: the API, WebSocket and worker workloads all call the provider.

## Organization-level keys

Organization administrators can supply their own keys in the application, under **Organization Settings → Model Settings → AI API keys**. Keys entered there are stored in [Vault](/self-managed/configuration/vault.md) and override the install-level key for the same provider.

## Amazon Bedrock

Bedrock uses AWS credentials rather than an API key. It is available only to organizations whose [execution environment](/self-managed/configuration/code-execution.md) is backed by AWS credentials — static keys or an assumed role — because Zerve invokes Bedrock with those credentials, in the region configured alongside them. Models that require a cross-region inference profile are resolved automatically from that region.

Grant `bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream` and `bedrock:ListFoundationModels` to the identity Zerve assumes, and enable access to the models you intend to use in the Bedrock console.

## LLM gateway

Installs without direct access to provider APIs can route all agent traffic through a single OpenAI-compatible endpoint:

```yaml
# my-values.yaml
canvas:
  workloads:
    common:
      secrets:
        LLMGW_API_KEY:
          secretName: zerve-llm-gateway
          key: api_key                            # optional, defaults to LLMGW_API_KEY
      env:
        LLMGW_API_BASE: "https://llm-gateway.acme.internal"
```

Both values are required; the service refuses to start an LLM call if `LLMGW_API_BASE` is set without `LLMGW_API_KEY`. Zerve posts to `{LLMGW_API_BASE}/v1/chat/completions` with `Authorization: Bearer {LLMGW_API_KEY}`, sends the model as `<provider>/<model>` (for example `anthropic/claude-sonnet-4-6`), and expects streaming and tool calls in the OpenAI format. Your gateway is responsible for holding the provider credentials and for mapping those model names onto its own upstreams.

With a gateway configured, agent traffic no longer uses install-level or organization-level provider keys.

## Network egress

Unless you route everything through an [LLM gateway](#llm-gateway), Zerve calls the provider APIs directly, so a cluster that restricts outbound traffic has to allow that egress explicitly — otherwise the AI features fail at the first call with a connection timeout.

The canvas service API, WebSocket and worker workloads need outbound HTTPS to whichever of the following you use:

| Provider      | Endpoint                                                                   |
| ------------- | -------------------------------------------------------------------------- |
| Anthropic     | `api.anthropic.com`                                                        |
| OpenAI        | `api.openai.com`                                                           |
| Google Gemini | `generativelanguage.googleapis.com`                                        |
| OpenRouter    | `openrouter.ai`                                                            |
| Bedrock       | `bedrock-runtime.<region>.amazonaws.com`, `bedrock.<region>.amazonaws.com` |

Your compute backend needs the same egress if users create AI blocks.

With a gateway, none of the above applies: the only destination to allow is `LLMGW_API_BASE`, and the gateway holds the provider credentials and makes the outbound calls itself. That is the usual arrangement for an air-gapped install, where the provider endpoints are unreachable by design.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.zerve.ai/self-managed/configuration/llm-providers.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.