DataDesignerProfilingError on pandas 3.1.x when previewing a single record with an LLM column #1144

Priority Level: Medium (annoying but has a workaround: pandas<3.1)

Describe the bug

With pandas 3.1.x, DataDesigner.preview() (and any profiling pass) fails with DataDesignerProfilingError when the dataset has a single record and contains an LLM column. The declared range is pandas>=2.3.3,<4, so a fresh install picks up pandas 3.1.x (3.1.0 was released 2026-10-22 and 3.1.1 on 2026-11-04) and hits this.

data_designer.interface.errors.DataDesignerProfilingError: Error profiling preview dataset: float() argument must be a string or a real number, not 'NAType'

Root cause, from the traceback and a check in plain pandas:

1. calculate_output_token_stats (data_designer/engine/analysis/utils/column_statistics_calculations.py) does df[col].apply(lambda v: count_text_tokens(str(v))), then .mean(), .median() and .std().
2. The LLM output column is Arrow-backed. In pandas 3.0.6, Series.apply on it returns NumPy int64. In pandas 3.1.x it returns int64[pyarrow].
3. .std() (ddof=1) of one value is np.float64(nan) for NumPy but <NA> for the pyarrow dtype.
4. The llm_column_ensure_python_floats validator (data_designer/config/analysis/column_statistics.py) calls float(<NA>) through prepare_number_for_reporting, which raises TypeError.

Plain pandas, no Data Designer:

import pandas as pd, pyarrow as pa
s = pd.Series([{"answer": "hi"}], dtype=pd.ArrowDtype(pa.struct([("answer", pa.string())])))
t = s.apply(lambda v: len(str(v)))
print(t.dtype, repr(t.mean()), repr(t.std()))
# pandas 3.0.6: int64 np.float64(16.0) np.float64(nan)
# pandas 3.1.1: int64[pyarrow] 16.0 <NA>

With two or more records the standard deviation is a real number, so profiling passes. Only the single-record case fails.

Steps/Code to reproduce bug

Self-contained, no API key (a local stub serves the chat completion). Run it with pandas 3.1.1, then with 3.0.6:

"""preview() of one record with an LLM structured column fails to profile on pandas 3.1.x."""
import json
import tempfile
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer

import data_designer.config as dd
import pandas as pd
from data_designer.interface import DataDesigner
from pydantic import BaseModel


class StubOpenAI(BaseHTTPRequestHandler):
    def do_POST(self):
        self.rfile.read(int(self.headers.get("content-length", 0)))
        body = json.dumps({
            "id": "x", "object": "chat.completion", "created": 0, "model": "m",
            "choices": [{"index": 0, "finish_reason": "stop",
                         "message": {"role": "assistant", "content": '{"answer": "hi"}'}}],
            "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},
        }).encode()
        self.send_response(200)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass


class Out(BaseModel):
    answer: str


server = HTTPServer(("127.0.0.1", 0), StubOpenAI)
threading.Thread(target=server.serve_forever, daemon=True).start()

builder = dd.DataDesignerConfigBuilder(model_configs=[dd.ModelConfig(
    alias="m", model="m", provider="p", skip_health_check=True,
    inference_parameters=dd.ChatCompletionInferenceParams(max_parallel_requests=1))])
builder.with_seed_dataset(dd.DataFrameSeedSource(df=pd.DataFrame({"topic": ["a"]})))
builder.add_column(dd.LLMStructuredColumnConfig(
    name="out", model_alias="m", prompt="Say hi about {{ topic }}", output_format=Out))

designer = DataDesigner(
    artifact_path=tempfile.mkdtemp(), auto_configure_logging=False,
    model_providers=[dd.ModelProvider(
        name="p", endpoint=f"http://127.0.0.1:{server.server_port}/v1",
        provider_type="openai", api_key="k")])
result = designer.preview(builder, num_records=1)
print("OK", len(result.dataset), "row")

Output on pandas 3.1.1 (Data Designer 0.10.0):

data_designer.interface.errors.DataDesignerProfilingError: Error profiling preview dataset: float() argument must be a string or a real number, not 'NAType'

Output on pandas 3.0.6: OK 1 row.

Expected behavior

preview() and create() succeed for one record on any pandas version the package allows. A single-record standard deviation should be reported as missing or nan, as it is on pandas 3.0.

Agent Diagnostic / Prior Investigation

An AI coding agent (Claude Code) did this investigation and I reviewed the result. What it did:

• Searched this repository's issues and PRs for "pandas 3.1", "NAType", "profiler pandas" and "DataDesignerProfilingError". Nothing covers this problem.
• Confirmed the failure on the released 0.10.0 and on 1.0.0rc2 with pandas 3.1.1. The 1.0.0rc2 failure was seen in a downstream test suite and its cause was not examined.
• Compared the code on main (00e20c0, 2026-11-09) with v0.10.0. main is 10 commits ahead and none of them touch engine/analysis, config/analysis or the pandas constraints, and the lines quoted above are identical on main. I did not run main itself, so this is a source comparison, not a run.
• Reduced the failure from a downstream test to the script above, then to the plain-pandas snippet.
• Tried one candidate fix. Replacing the series in calculate_output_token_stats with .apply(...).astype("float64") makes the script above pass on pandas 3.1.1. I only ran it as a local patch, not as a PR.

Additional context

• Environment: Python 3.12.11, data-designer and data-designer-engine 0.10.0, pandas 3.1.1, pyarrow 24.0.0, numpy 2.5.4.
• Workaround: pandas<3.1. A downstream project (interplay-adversarial/interplay) pinned pandas>=2.2,<3.1 for this reason, because its judge call previews exactly one record.
• Possible fixes, either would do: coerce to NumPy float64 before computing statistics (as tested above), or make prepare_number_for_reporting and the validator treat pd.NA as MissingValue.CALCULATION_FAILED. The second also covers other statistics that go through the same validator.
• I'm happy to send a PR once a maintainer has triaged this issue.