Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ The following model configurations are automatically available when `OPENAI_API_

| Alias | Model | Use Case | Inference Parameters |
|-------|-------|----------|---------------------|
| `openai-text` | `gpt-4.1` | General text generation | `temperature=0.85, top_p=0.95` |
| `openai-reasoning` | `gpt-5` | Reasoning and analysis tasks | `extra_body={"reasoning_effort": "medium"}` |
| `openai-vision` | `gpt-5` | Vision and image understanding | `extra_body={"reasoning_effort": "medium"}` |
| `openai-text` | `gpt-5.6-luna` | General text generation | `temperature=0.85, top_p=0.95, extra_body={"reasoning_effort": "none"}` |
| `openai-reasoning` | `gpt-5.6-luna` | Reasoning and analysis tasks | `extra_body={"reasoning_effort": "medium"}` |
| `openai-vision` | `gpt-5.6-luna` | Vision and image understanding | `extra_body={"reasoning_effort": "medium"}` |
| `openai-embedding` | `text-embedding-3-large` | Text embeddings | `encoding_format="float"` |

### OpenRouter Models
Expand Down Expand Up @@ -93,9 +93,23 @@ When the Data Designer library or the CLI is initialized, default model configur
- **Model Providers**: `~/.data-designer/model_providers.yaml`

<Note title="Existing configurations are preserved">
Upgrades do not overwrite existing configuration files. If `nvidia-text` or `nvidia-reasoning` still references a retired model, update `~/.data-designer/model_configs.yaml` manually or run `data-designer config reset` to restore the current defaults. Resetting deletes all custom configurations.
Upgrades do not overwrite existing configuration files. If `nvidia-text`, `nvidia-reasoning`, or an `openai-*` alias still references a retired model (for example `gpt-5` or `gpt-4.1`), update `~/.data-designer/model_configs.yaml` manually or run `data-designer config reset` to restore the current defaults. Resetting deletes all custom configurations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small YAML example might make this migration easier. If someone changes only the model name, the saved temperature and top_p values remain, but reasoning_effort="none" is missing, which returned HTTP 400 in my check. Showing the inference parameters alongside the model would help users update without resetting their custom configs.

</Note>

If you only change the `model` of an existing alias, the saved inference parameters stay as they were. `gpt-5.6-luna` also needs `reasoning_effort` set explicitly for the text alias (omitting it returned HTTP 400), so update the parameters together with the model. For example, an `openai-text` alias that used to point at `gpt-4.1`:

```yaml
model_configs:
- alias: openai-text
model: gpt-5.6-luna
provider: openai
inference_parameters:
temperature: 0.85
top_p: 0.95
extra_body:
reasoning_effort: none
```

<Tip>
While these files provide a convenient way to specify settings for your model providers and configuration you use most often, they can always be set programmatically in your SDG workflow.
</Tip>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,7 @@ class NordColor(Enum):
}
NEMOTRON_3_NANO_OMNI_30B_A3B_REASONING_INFERENCE_PARAMS = {"temperature": 0.60, "top_p": 0.95}
GPT5_INFERENCE_PARAMS = {"extra_body": {"reasoning_effort": "medium"}}
GPT5_TEXT_INFERENCE_PARAMS = DEFAULT_TEXT_INFERENCE_PARAMS | {"extra_body": {"reasoning_effort": "none"}}

PREDEFINED_PROVIDERS_MODEL_MAP = {
NVIDIA_PROVIDER_NAME: {
Expand All @@ -370,9 +371,9 @@ class NordColor(Enum):
},
},
OPENAI_PROVIDER_NAME: {
"text": {"model": "gpt-4.1", "inference_parameters": DEFAULT_TEXT_INFERENCE_PARAMS},
"reasoning": {"model": "gpt-5", "inference_parameters": GPT5_INFERENCE_PARAMS},
"vision": {"model": "gpt-5", "inference_parameters": GPT5_INFERENCE_PARAMS},
"text": {"model": "gpt-5.6-luna", "inference_parameters": GPT5_TEXT_INFERENCE_PARAMS},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There’s another compatibility detail with openai-text: calls that set max_tokens worked with GPT-4.1, but Luna rejects that parameter and asks for max_completion_tokens, even with reasoning disabled. It looks like DD forwards it unchanged. Translating the limit for the affected OpenAI models, with a test for an explicit token limit, would help preserve those calls.

"reasoning": {"model": "gpt-5.6-luna", "inference_parameters": GPT5_INFERENCE_PARAMS},
"vision": {"model": "gpt-5.6-luna", "inference_parameters": GPT5_INFERENCE_PARAMS},
Comment on lines +375 to +376

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m seeing a compatibility issue with MCP tools here: Luna returns HTTP 400 for function tools with reasoning_effort="medium" through Chat Completions. The same exchange works with GPT-5, and with Luna when reasoning is disabled. Would it make sense to revisit the interim configuration in #1129 so these aliases keep working with tools while Responses support is pending?

"embedding": {"model": "text-embedding-3-large", "inference_parameters": DEFAULT_EMBEDDING_INFERENCE_PARAMS},
},
OPENROUTER_PROVIDER_NAME: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,13 +78,18 @@ def test_get_builtin_model_configs():
assert builtin_model_configs[3].model == "nvidia/nemotron-3-embed-1b"
assert builtin_model_configs[3].provider == "nvidia"
assert builtin_model_configs[4].alias == "openai-text"
assert builtin_model_configs[4].model == "gpt-4.1"
assert builtin_model_configs[4].model == "gpt-5.6-luna"
assert builtin_model_configs[4].provider == "openai"
assert builtin_model_configs[4].inference_parameters == ChatCompletionInferenceParams(
temperature=0.85,
top_p=0.95,
extra_body={"reasoning_effort": "none"},
)
assert builtin_model_configs[5].alias == "openai-reasoning"
assert builtin_model_configs[5].model == "gpt-5"
assert builtin_model_configs[5].model == "gpt-5.6-luna"
assert builtin_model_configs[5].provider == "openai"
assert builtin_model_configs[6].alias == "openai-vision"
assert builtin_model_configs[6].model == "gpt-5"
assert builtin_model_configs[6].model == "gpt-5.6-luna"
assert builtin_model_configs[6].provider == "openai"
assert builtin_model_configs[7].alias == "openai-embedding"
assert builtin_model_configs[7].model == "text-embedding-3-large"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from __future__ import annotations

from typing import Any
from urllib.parse import urlparse

from data_designer.config.utils.media_helpers import audio_format_from_mime_type
from data_designer.engine.models.clients.adapters.http_model_client import (
Expand All @@ -30,6 +31,8 @@
TransportKwargs,
)

_OPENAI_HOSTNAME = "api.openai.com"


class OpenAICompatibleClient(HttpModelClient):
"""Native HTTP adapter for OpenAI-compatible provider APIs.
Expand Down Expand Up @@ -57,6 +60,21 @@ def supports_embeddings(self) -> bool:
def supports_image_generation(self) -> bool:
return True

def _adapt_chat_body(self, body: dict[str, Any]) -> dict[str, Any]:
"""Rename ``max_tokens`` to ``max_completion_tokens`` for the hosted OpenAI API.

Newer OpenAI models (for example ``gpt-5.6-luna``) reject ``max_tokens`` on Chat
Completions. Other OpenAI-compatible servers (vLLM, NVIDIA, ...) still expect it,
so the rename only applies to ``api.openai.com``. An explicit ``max_completion_tokens``
(for example from ``extra_body``) wins.
"""
if "max_tokens" not in body or urlparse(self._endpoint).hostname != _OPENAI_HOSTNAME:
return body
adapted = dict(body)
max_tokens = adapted.pop("max_tokens")
adapted.setdefault("max_completion_tokens", max_tokens)
return adapted

# -------------------------------------------------------------------
# Chat completion
# -------------------------------------------------------------------
Expand All @@ -68,7 +86,7 @@ def completion(self, request: ChatCompletionRequest) -> ChatCompletionResponse:
provider_name=self.provider_name,
model_name=request.model,
)
payload = {"model": request.model, "messages": messages, **transport.body}
payload = {"model": request.model, "messages": messages, **self._adapt_chat_body(transport.body)}
response_json = self._post_sync(self._ROUTE_CHAT, payload, transport.headers, request.model, transport.timeout)
return parse_chat_completion_response(response_json)

Expand All @@ -79,7 +97,7 @@ async def acompletion(self, request: ChatCompletionRequest) -> ChatCompletionRes
provider_name=self.provider_name,
model_name=request.model,
)
payload = {"model": request.model, "messages": messages, **transport.body}
payload = {"model": request.model, "messages": messages, **self._adapt_chat_body(transport.body)}
response_json = await self._apost(
self._ROUTE_CHAT, payload, transport.headers, request.model, transport.timeout
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,12 @@ def _make_client(
sync_client: MagicMock | None = None,
async_client: MagicMock | None = None,
api_key: str | None = "sk-test-key",
endpoint: str = ENDPOINT,
) -> OpenAICompatibleClient:
concurrency_mode = ClientConcurrencyMode.ASYNC if async_client is not None else ClientConcurrencyMode.SYNC
return OpenAICompatibleClient(
provider_name=PROVIDER,
endpoint=ENDPOINT,
endpoint=endpoint,
api_key=api_key,
concurrency_mode=concurrency_mode,
sync_client=sync_client,
Expand Down Expand Up @@ -156,6 +157,60 @@ def test_completion_posts_to_chat_completions_route() -> None:
assert call_args.kwargs["headers"]["X-Trace"] == "1"


def test_completion_translates_max_tokens_for_openai_api() -> None:
sync_mock = make_mock_sync_client(_chat_response())
client = _make_client(sync_client=sync_mock, endpoint="https://api.openai.com/v1")

client.completion(ChatCompletionRequest(model=MODEL, messages=[{"role": "user", "content": "Hi"}], max_tokens=256))

payload = sync_mock.post.call_args.kwargs["json"]
assert payload["max_completion_tokens"] == 256
assert "max_tokens" not in payload


def test_completion_keeps_explicit_max_completion_tokens_for_openai_api() -> None:
sync_mock = make_mock_sync_client(_chat_response())
client = _make_client(sync_client=sync_mock, endpoint="https://api.openai.com/v1")

client.completion(
ChatCompletionRequest(
model=MODEL,
messages=[{"role": "user", "content": "Hi"}],
max_tokens=256,
extra_body={"max_completion_tokens": 512},
)
)

payload = sync_mock.post.call_args.kwargs["json"]
assert payload["max_completion_tokens"] == 512
assert "max_tokens" not in payload


def test_completion_keeps_max_tokens_for_other_openai_compatible_endpoints() -> None:
sync_mock = make_mock_sync_client(_chat_response())
client = _make_client(sync_client=sync_mock)

client.completion(ChatCompletionRequest(model=MODEL, messages=[{"role": "user", "content": "Hi"}], max_tokens=256))

payload = sync_mock.post.call_args.kwargs["json"]
assert payload["max_tokens"] == 256
assert "max_completion_tokens" not in payload


@pytest.mark.asyncio
async def test_acompletion_translates_max_tokens_for_openai_api() -> None:
async_mock = make_mock_async_client(_chat_response())
client = _make_client(async_client=async_mock, endpoint="https://api.openai.com/v1")

await client.acompletion(
ChatCompletionRequest(model=MODEL, messages=[{"role": "user", "content": "Hi"}], max_tokens=256)
)

payload = async_mock.post.call_args.kwargs["json"]
assert payload["max_completion_tokens"] == 256
assert "max_tokens" not in payload


@pytest.mark.parametrize(
"tool_choice",
[
Expand Down
Loading