Codex not working with OSS running on VLLM

What version of Codex is running?

codex-cli 0.21.0

Which model were you using?

oss-gpt-120B

What platform is your computer?

Darwin 24.5.0 arm64 arm

What steps can reproduce the bug?

Setup config.toml with

[model_providers.oss]
base_url = "https://vllm.example.com/v1"
env_key = "OPENAI_API_KEY"

[profiles.genai]
model_provider="oss"
model = "openai/gpt-oss-120b"
name = "Vllm"

What is the expected behavior?

I expect prompt "2+2" should respond with "4"

What do you see instead?

I get the output "🖐 error sending request for url (http://localhost:11434/v1/chat/completions)"; however, I set the base_url to not be localhost?

Additional information

Whats the proper way to integrate with VLLM?

I've confirmed the base_url in config.toml is not used for oss. As a workaround for now, you can go with the env variable "CODEX_OSS_BASE_URL" and it works without any issues: CODEX_OSS_BASE_URL="http://{your-host}/v1" codex

A good workaround I have found for now is just start a local TCP forwarder: 127.0.0.1:11434 > xxx.xxx.xxx.xxx:PORT

Fix #2257 by allowing users to configure oss provider in config.toml

I think this is still an issue. Tool calling with codex 0.36.0 does not work with the gpt-oss-20b model served with vllm using their container:

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e "HUGGING_FACE_HUB_TOKEN=$(cat ~/huggingface.token.txt)" \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  -p 8000:8000 \
  --ipc=host \
  --name vllm_gpt-oss-20b_$(date +%Y-%m-%d_%H-%M-%S) \
  vllm/vllm-openai:latest \
    --model openai/gpt-oss-20b \
    --max-model-len 10k \
    --gpu-memory-utilization 0.95

I can get codex to call the model with "Hello" and other prompts, but as soon as the prompt triggers the usage of tools, codex just does nothing.

But when I serve the gpt-oss-20b model with updated llama.cpp, everything works great.

So is the problem codex or vLLM?

@bartlettroscoe

@bartlettroscoe Its still and issue, just a separate one found here.

I think it is VLLM. In VLLM's guide for deploying GPT OSS it shows that Response API with Streaming does not support function calling. So if Codex, is calling that api path, then I think that would be the problem

Fix openai#2257 by allowing users to configure oss provider in config…

Navigation Menu

Repository navigation

Codex not working with OSS running on VLLM #2257

What version of Codex is running?

Which model were you using?

What platform is your computer?

What steps can reproduce the bug?

What is the expected behavior?

What do you see instead?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Codex not working with OSS running on VLLM #2257

Description

What version of Codex is running?

Which model were you using?

What platform is your computer?

What steps can reproduce the bug?

What is the expected behavior?

What do you see instead?

Additional information

Activity

seratch commented on Aug 14, 2025

BlackFrameAI commented on Aug 17, 2025

bartlettroscoe commented on Sep 20, 2025

jayteaftw commented on Sep 22, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions