OpenAI-compatible endpoints

Route Pullfrog's model calls to any endpoint that speaks the OpenAI Chat Completions API: a LiteLLM proxy, Cloudflare AI Gateway, Alibaba DashScope, or a self-hosted vLLM server. Use it for a model the catalog does not carry, several providers behind one gateway, or your own inference.

Three cases have a shorter route:

  • A provider listed on models.dev runs from its own API key as a custom provider, with no base URL or token limits to declare.
  • Azure OpenAI has its own setup.
  • A gateway that serves a catalogued provider's models under their own IDs takes that provider's base URL, which keeps its model list and Smart ladder.

Setup

Store the five values

In the Pullfrog console, click Set up on the Custom (OAI) tile of the Providers card and fill in the five required values. Pullfrog injects them into every run, and a repository can override the values stored for its organization.

To keep them in the workflow instead, put OPENAI_COMPATIBLE_API_KEY in a GitHub Actions secret and the rest in the env: block:

.github/workflows/pullfrog.yml
env:
  OPENAI_COMPATIBLE_BASE_URL: https://gateway.ai.cloudflare.com/v1/<acct>/<gateway>/compat
  OPENAI_COMPATIBLE_API_KEY: ${{ secrets.OPENAI_COMPATIBLE_API_KEY }}
  OPENAI_COMPATIBLE_MODEL: workers-ai/@cf/meta/llama-3.1-8b-instruct
  OPENAI_COMPATIBLE_CONTEXT: "128000"
  OPENAI_COMPATIBLE_MAX_OUTPUT: "16384"

Pick the model

Open the Model card's editor, choose Manual and pick OpenAI-compatible under Custom in the model menu.

Variables

NameSecretPurpose
OPENAI_COMPATIBLE_BASE_URLNoBase URL of the endpoint, the path that serves /chat/completions.
OPENAI_COMPATIBLE_API_KEYYesAPI key or bearer token for the endpoint.
OPENAI_COMPATIBLE_MODELNoExact model ID the endpoint serves. Pullfrog runs it as written and never upgrades it.
OPENAI_COMPATIBLE_CONTEXTNoContext window of that model, in tokens.
OPENAI_COMPATIBLE_MAX_OUTPUTNoMaximum completion tokens that model accepts.
OPENAI_COMPATIBLE_HEADERSYesOptional. Extra request headers, one Name: Value per line.
OPENAI_COMPATIBLE_IMAGE_INPUTNoOptional. "false" for a model that only accepts text.

The first five are required, and a run without them fails before the agent starts. The two limits must be positive numbers, so 128k is rejected. A value set in the workflow wins over a stored one.

Put the key only in OPENAI_COMPATIBLE_API_KEY, never inside the base URL. A base URL set in the workflow is not masked in logs.

Token limits

Your endpoint owns the model catalog, so Pullfrog cannot look up the model's limits and needs you to declare them. Undeclared limits would break two things:

  • Completions are capped at 32000 tokens, and a model with a smaller cap rejects every request. GPT-4o and GPT-4o mini stop at 16384, and many open models at 4096 or 8192.
  • Auto-compaction turns off. Compaction keys off the context limit, so a long session grows until the endpoint refuses it.

Set OPENAI_COMPATIBLE_CONTEXT to the model's real window. A low value compacts earlier than needed; a high value delays compaction until the endpoint refuses the request.

Custom headers

An endpoint can require more than a bearer token, such as the service token of an access proxy in front of a self-hosted server. Set OPENAI_COMPATIBLE_HEADERS to one Name: Value per line, under Send custom headers in the dialog or in your workflow:

.github/workflows/pullfrog.yml
env:
  OPENAI_COMPATIBLE_HEADERS: |
    CF-Access-Client-Id: ${{ secrets.CF_ACCESS_CLIENT_ID }}
    CF-Access-Client-Secret: ${{ secrets.CF_ACCESS_CLIENT_SECRET }}

Pullfrog sends them on every request to this endpoint and no other, and masks them in logs. A named endpoint takes its own OPENAI_COMPATIBLE_<NAME>_HEADERS and never borrows the unnamed endpoint's. Custom headers need Pullfrog 0.1.122 or later.

Text-only models

An agent sometimes reads an image, such as a screenshot of a page it rendered. A model that only accepts text rejects that request, and the run fails.

Set OPENAI_COMPATIBLE_IMAGE_INPUT to "false" for such a model, with Add secret in the console or in your workflow:

.github/workflows/pullfrog.yml
env:
  OPENAI_COMPATIBLE_IMAGE_INPUT: "false"

The agent is then told it can't read the image, and works without it. A named endpoint takes its own OPENAI_COMPATIBLE_<NAME>_IMAGE_INPUT. The setting needs Pullfrog 0.1.138 or later.

Several endpoints

Give an endpoint a name to keep more than one. Each named endpoint holds its own five values and appears in the model menu under Custom, so every repository can pick one, and switching between them never means entering a key again.

Name the endpoint

In the Configure Custom (OAI) dialog, type a name under Add a named endpoint and click Add. Names use lowercase letters, digits and single hyphens, such as vllm-a, up to 32 characters.

Store its five values

Fill in the endpoint's base URL, key, model ID and token limits, and save.

Pick it

Choose it in a repository's model menu, or name it for a single run with @pullfrog --model=openai-compatible/vllm-a.

A named endpoint's variables carry the name, uppercased and with hyphens as underscores, after OPENAI_COMPATIBLE_. To keep vllm-a in the workflow instead:

.github/workflows/pullfrog.yml
env:
  OPENAI_COMPATIBLE_VLLM_A_BASE_URL: https://vllm-a.internal.example.com/v1
  OPENAI_COMPATIBLE_VLLM_A_API_KEY: ${{ secrets.VLLM_A_KEY }}
  OPENAI_COMPATIBLE_VLLM_A_MODEL: qwen3-coder
  OPENAI_COMPATIBLE_VLLM_A_CONTEXT: "128000"
  OPENAI_COMPATIBLE_VLLM_A_MAX_OUTPUT: "16384"

Named and unnamed endpoints never mix:

  • A run on a named endpoint uses that endpoint's values and nothing else. A value it lacks is reported by its own name, never borrowed from the unnamed endpoint.
  • The unnamed endpoint keeps the bare OPENAI_COMPATIBLE_* names and the OpenAI-compatible menu entry, so existing setups need no change.
  • Named endpoints need Pullfrog 0.1.105 or later. An older runner reads openai-compatible/vllm-a as the model ID vllm-a on the unnamed endpoint.

Smart model selection

Under Smart, a custom endpoint has no ladder. Smart runs the models your guidance mentions, or a catalogued provider's ladder through your gateway.

Models the endpoint serves

Pick Custom (OAI) as the Smart provider. With no guidance, every run takes the endpoint's own model, OPENAI_COMPATIBLE_MODEL.

To use other models the endpoint serves, mention them by ID in the guidance on the Model row, in your own words:

Anything under src/billing/ and any large pull request goes to qwen3-coder-plus.
Everything else can run on qwen3-coder-flash.

A run takes a model the guidance mentions when the guidance gives it that work, and the endpoint's own model otherwise.

  • They all run on the unnamed endpoint, with its key and its two token limits, so set the limits to fit the smallest model you mention.
  • A named endpoint can be mentioned by its name instead, and runs on its own values.
  • Pullfrog never checks an ID against the endpoint, so an ID it doesn't serve fails the run that picks it.

Through a provider's tile

A gateway that serves a provider's models under their own IDs, such as OpenAI's or DeepSeek's, can also run that provider's models on Smart, from the provider's own tile:

  1. Click Set up on the provider's tile of the Providers card. Put the gateway's key in the API key field, then expand Set up a proxy or gateway and enter the gateway's base URL, such as the path that ends in /v1.
  2. Open the Model card's editor, choose Smart and pick that provider.

The gateway has two requirements:

  • It must serve the request the provider sends, which Gateways and proxies lists for each one. OpenAI and xAI send the Responses API (POST /v1/responses), not only /chat/completions.
  • It must accept the provider's own model IDs, which the model menu lists. Pullfrog moves a model to a newer version when one ships, so the gateway has to serve the new ID too.

Model IDs

The model menu's OpenAI-compatible entry runs the unnamed endpoint; OPENAI_COMPATIBLE_MODEL names the actual model, in your gateway's own format:

GatewayModel ID formatExample
Cloudflare AI Gateway<provider>/<model>workers-ai/@cf/meta/llama-3.1-8b-instruct
DashScopeBare model IDqwen3-coder-plus
LiteLLM proxyThe model_name from the proxy's model_listclaude-opus-internal

LiteLLM

A LiteLLM proxy needs no special configuration. Point the base URL at the proxy's /v1 path and use a virtual key:

.github/workflows/pullfrog.yml
env:
  OPENAI_COMPATIBLE_BASE_URL: https://litellm.internal.example.com/v1
  OPENAI_COMPATIBLE_API_KEY: ${{ secrets.LITELLM_VIRTUAL_KEY }}
  OPENAI_COMPATIBLE_MODEL: claude-opus-internal # model_name from model_list
  OPENAI_COMPATIBLE_CONTEXT: "200000"
  OPENAI_COMPATIBLE_MAX_OUTPUT: "32000"

LiteLLM's own routing, failover and spend tracking apply as configured. The same setup covers models the proxy serves from self-hosted vLLM or Ollama.

Routing

Pullfrog runs the model on OpenCode, through an @ai-sdk/openai-compatible provider pointed at OPENAI_COMPATIBLE_BASE_URL. Requests go to <base URL>/chat/completions with your key as the bearer token.

Troubleshooting

SymptomFix
"OPENAI_COMPATIBLE_MODEL must be set when the model is openai-compatible/byok"The repository uses the OpenAI-compatible entry, but no model ID is stored or set in the workflow.
"OpenAI-compatible endpoint is missing required settings"The message names each missing or non-numeric value.
"Line 2 of OPENAI_COMPATIBLE_HEADERS is not a header"That line needs a Name: Value shape, in ASCII. A blank line is fine.
A 400 right after the agent reads an image, such as messages.content.type is invalidThe model only accepts text. Set OPENAI_COMPATIBLE_IMAGE_INPUT: "false".
max_tokens is too largeThe value of OPENAI_COMPATIBLE_MAX_OUTPUT is above what the model accepts. Lower it to the model's real limit.
HTTP 404 from the endpointPullfrog appends /chat/completions to the base URL, so the base URL is usually the path that ends in /v1.
© 2026 Pullfrog, Inc.

On this page