# OpenAI-compatible endpoints

> Run Pullfrog against any endpoint that speaks the OpenAI Chat Completions API, such as LiteLLM, Cloudflare AI Gateway or vLLM.

Route Pullfrog's model calls to any endpoint that speaks the OpenAI Chat Completions API: a [LiteLLM](https://docs.litellm.ai/docs/simple_proxy) proxy, [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/), [Alibaba DashScope](https://www.alibabacloud.com/help/en/model-studio/), or a self-hosted [vLLM](https://docs.vllm.ai/) server. Use it for a model the catalog does not carry, several providers behind one gateway, or your own inference.

Three cases have a shorter route:

- A provider listed on [models.dev](https://models.dev) runs from its own API key as a [custom provider](https://pullfrog.com/docs/providers#custom-providers), with no base URL or token limits to declare.
- Azure OpenAI has [its own setup](https://pullfrog.com/docs/azure).
- A gateway that serves a catalogued provider's models under their own IDs takes that provider's [base URL](https://pullfrog.com/docs/gateways), which keeps its model list and Smart ladder.

## Setup

### Store the five values

In the Pullfrog console, click **Set up** on the **Custom (OAI)** tile of the **Providers** card and fill in the five [required values](https://pullfrog.com/docs/openai-compatible#variables). Pullfrog injects them into every run, and a repository can override the values stored for its organization.

To keep them in the workflow instead, put `OPENAI_COMPATIBLE_API_KEY` in a GitHub Actions secret and the rest in the `env:` block:

`.github/workflows/pullfrog.yml`:

```yaml
env:
  OPENAI_COMPATIBLE_BASE_URL: https://gateway.ai.cloudflare.com/v1/<acct>/<gateway>/compat
  OPENAI_COMPATIBLE_API_KEY: ${{ secrets.OPENAI_COMPATIBLE_API_KEY }}
  OPENAI_COMPATIBLE_MODEL: workers-ai/@cf/meta/llama-3.1-8b-instruct
  OPENAI_COMPATIBLE_CONTEXT: "128000"
  OPENAI_COMPATIBLE_MAX_OUTPUT: "16384"
```

### Pick the model

Open the **Model** card's editor, choose **Manual** and pick **OpenAI-compatible** under **Custom** in the model menu.

## Variables

| Name | Secret | Purpose |
| --- | --- | --- |
| `OPENAI_COMPATIBLE_BASE_URL` | No | Base URL of the endpoint, the path that serves `/chat/completions`. |
| `OPENAI_COMPATIBLE_API_KEY` | Yes | API key or bearer token for the endpoint. |
| `OPENAI_COMPATIBLE_MODEL` | No | Exact model ID the endpoint serves. Pullfrog runs it as written and never upgrades it. |
| `OPENAI_COMPATIBLE_CONTEXT` | No | Context window of that model, in tokens. |
| `OPENAI_COMPATIBLE_MAX_OUTPUT` | No | Maximum completion tokens that model accepts. |
| `OPENAI_COMPATIBLE_HEADERS` | Yes | Optional. Extra request headers, one `Name: Value` per line. |
| `OPENAI_COMPATIBLE_IMAGE_INPUT` | No | Optional. `"false"` for a model that only accepts text. |

The first five are required, and a run without them fails before the agent starts. The two limits must be positive numbers, so `128k` is rejected. A value set in the workflow wins over a stored one.

Put the key only in `OPENAI_COMPATIBLE_API_KEY`, never inside the base URL. A base URL set in the workflow is not masked in logs.

> On an account that pays through [Pullfrog Router](https://pullfrog.com/docs/router), store the values in the Pullfrog console. Pullfrog decides how to fund a run before the runner starts, when GitHub Actions secrets are not visible to it, so a run whose values exist only there falls back to the Router's default model.

### Token limits

Your endpoint owns the model catalog, so Pullfrog cannot look up the model's limits and needs you to declare them. Undeclared limits would break two things:

- **Completions are capped at 32000 tokens**, and a model with a smaller cap rejects every request. GPT-4o and GPT-4o mini stop at 16384, and many open models at 4096 or 8192.
- **Auto-compaction turns off.** Compaction keys off the context limit, so a long session grows until the endpoint refuses it.

Set `OPENAI_COMPATIBLE_CONTEXT` to the model's real window. A low value compacts earlier than needed; a high value delays compaction until the endpoint refuses the request.

### Custom headers

An endpoint can require more than a bearer token, such as the service token of an access proxy in front of a self-hosted server. Set `OPENAI_COMPATIBLE_HEADERS` to one `Name: Value` per line, under **Send custom headers** in the dialog or in your workflow:

`.github/workflows/pullfrog.yml`:

```yaml
env:
  OPENAI_COMPATIBLE_HEADERS: |
    CF-Access-Client-Id: ${{ secrets.CF_ACCESS_CLIENT_ID }}
    CF-Access-Client-Secret: ${{ secrets.CF_ACCESS_CLIENT_SECRET }}
```

Pullfrog sends them on every request to this endpoint and no other, and masks them in logs. A named endpoint takes its own `OPENAI_COMPATIBLE_<NAME>_HEADERS` and never borrows the unnamed endpoint's. Custom headers need Pullfrog 0.1.122 or later.

### Text-only models

An agent sometimes reads an image, such as a screenshot of a page it rendered. A model that only accepts text rejects that request, and the run fails.

Set `OPENAI_COMPATIBLE_IMAGE_INPUT` to `"false"` for such a model, with [**Add secret**](https://pullfrog.com/docs/api-keys#pullfrog-secrets) in the console or in your workflow:

`.github/workflows/pullfrog.yml`:

```yaml
env:
  OPENAI_COMPATIBLE_IMAGE_INPUT: "false"
```

The agent is then told it can't read the image, and works without it. A named endpoint takes its own `OPENAI_COMPATIBLE_<NAME>_IMAGE_INPUT`. The setting needs Pullfrog 0.1.138 or later.

## Several endpoints

Give an endpoint a name to keep more than one. Each named endpoint holds its own five values and appears in the model menu under **Custom**, so every repository can pick one, and switching between them never means entering a key again.

### Name the endpoint

In the **Configure Custom (OAI)** dialog, type a name under **Add a named endpoint** and click **Add**. Names use lowercase letters, digits and single hyphens, such as `vllm-a`, up to 32 characters.

### Store its five values

Fill in the endpoint's base URL, key, model ID and token limits, and save.

### Pick it

Choose it in a repository's model menu, or name it for a single run with `@pullfrog --model=openai-compatible/vllm-a`.

A named endpoint's variables carry the name, uppercased and with hyphens as underscores, after `OPENAI_COMPATIBLE_`. To keep `vllm-a` in the workflow instead:

`.github/workflows/pullfrog.yml`:

```yaml
env:
  OPENAI_COMPATIBLE_VLLM_A_BASE_URL: https://vllm-a.internal.example.com/v1
  OPENAI_COMPATIBLE_VLLM_A_API_KEY: ${{ secrets.VLLM_A_KEY }}
  OPENAI_COMPATIBLE_VLLM_A_MODEL: qwen3-coder
  OPENAI_COMPATIBLE_VLLM_A_CONTEXT: "128000"
  OPENAI_COMPATIBLE_VLLM_A_MAX_OUTPUT: "16384"
```

Named and unnamed endpoints never mix:

- A run on a named endpoint uses that endpoint's values and nothing else. A value it lacks is reported by its own name, never borrowed from the unnamed endpoint.
- The unnamed endpoint keeps the bare `OPENAI_COMPATIBLE_*` names and the **OpenAI-compatible** menu entry, so existing setups need no change.
- Named endpoints need Pullfrog 0.1.105 or later. An older runner reads `openai-compatible/vllm-a` as the model ID `vllm-a` on the unnamed endpoint.

## Smart model selection

Under [Smart](https://pullfrog.com/docs/models#smart), a custom endpoint has no ladder. Smart runs the models your guidance mentions, or a catalogued provider's ladder through your gateway.

### Models the endpoint serves

Pick **Custom (OAI)** as the Smart provider. With no guidance, every run takes the endpoint's own model, `OPENAI_COMPATIBLE_MODEL`.

To use other models the endpoint serves, mention them by ID in the guidance on the **Model** row, in your own words:

```text
Anything under src/billing/ and any large pull request goes to qwen3-coder-plus.
Everything else can run on qwen3-coder-flash.
```

A run takes a model the guidance mentions when the guidance gives it that work, and the endpoint's own model otherwise.

- They all run on the unnamed endpoint, with its key and its two token limits, so set the limits to fit the smallest model you mention.
- A [named endpoint](https://pullfrog.com/docs/openai-compatible#several-endpoints) can be mentioned by its name instead, and runs on its own values.
- Pullfrog never checks an ID against the endpoint, so an ID it doesn't serve fails the run that picks it.

### Through a provider's tile

A gateway that serves a provider's models under their own IDs, such as OpenAI's or DeepSeek's, can also run that provider's models on Smart, from the provider's own tile:

1. Click **Set up** on the provider's tile of the **Providers** card. Put the gateway's key in the API key field, then expand **Set up a proxy or gateway** and enter the gateway's base URL, such as the path that ends in `/v1`.
2. Open the **Model** card's editor, choose **Smart** and pick that provider.

The gateway has two requirements:

- It must serve the request the provider sends, which [Gateways and proxies](https://pullfrog.com/docs/gateways) lists for each one. OpenAI and xAI send the Responses API (`POST /v1/responses`), not only `/chat/completions`.
- It must accept the provider's own model IDs, which the model menu lists. Pullfrog moves a model to a newer version when one ships, so the gateway has to serve the new ID too.

## Model IDs

The model menu's **OpenAI-compatible** entry runs the unnamed endpoint; `OPENAI_COMPATIBLE_MODEL` names the actual model, in your gateway's own format:

| Gateway | Model ID format | Example |
| --- | --- | --- |
| [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) | `<provider>/<model>` | `workers-ai/@cf/meta/llama-3.1-8b-instruct` |
| DashScope | Bare model ID | `qwen3-coder-plus` |
| LiteLLM proxy | The `model_name` from the proxy's `model_list` | `claude-opus-internal` |

## LiteLLM

A [LiteLLM proxy](https://docs.litellm.ai/docs/simple_proxy) needs no special configuration. Point the base URL at the proxy's `/v1` path and use a [virtual key](https://docs.litellm.ai/docs/proxy/virtual_keys):

`.github/workflows/pullfrog.yml`:

```yaml
env:
  OPENAI_COMPATIBLE_BASE_URL: https://litellm.internal.example.com/v1
  OPENAI_COMPATIBLE_API_KEY: ${{ secrets.LITELLM_VIRTUAL_KEY }}
  OPENAI_COMPATIBLE_MODEL: claude-opus-internal # model_name from model_list
  OPENAI_COMPATIBLE_CONTEXT: "200000"
  OPENAI_COMPATIBLE_MAX_OUTPUT: "32000"
```

LiteLLM's own routing, failover and spend tracking apply as configured. The same setup covers models the proxy serves from self-hosted vLLM or Ollama.

## Routing

Pullfrog runs the model on OpenCode, through an [`@ai-sdk/openai-compatible`](https://www.npmjs.com/package/@ai-sdk/openai-compatible) provider pointed at `OPENAI_COMPATIBLE_BASE_URL`. Requests go to `<base URL>/chat/completions` with your key as the bearer token.

## Troubleshooting

| Symptom | Fix |
| --- | --- |
| "`OPENAI_COMPATIBLE_MODEL` must be set when the model is `openai-compatible/byok`" | The repository uses the OpenAI-compatible entry, but no model ID is stored or set in the workflow. |
| "OpenAI-compatible endpoint is missing required settings" | The message names each missing or non-numeric value. |
| "Line 2 of `OPENAI_COMPATIBLE_HEADERS` is not a header" | That line needs a `Name: Value` shape, in ASCII. A blank line is fine. |
| A `400` right after the agent reads an image, such as `messages.content.type is invalid` | The model only accepts text. Set `OPENAI_COMPATIBLE_IMAGE_INPUT: "false"`. |
| `max_tokens is too large` | The value of `OPENAI_COMPATIBLE_MAX_OUTPUT` is above what the model accepts. Lower it to the model's real limit. |
| HTTP `404` from the endpoint | Pullfrog appends `/chat/completions` to the base URL, so the base URL is usually the path that ends in `/v1`. |
