Skip to content

Make applyChatTemplate a buffer-based C ABI for objcxx interop#7

Merged
FuJacob merged 1 commit into
mainfrom
chat-template-c-abi
May 30, 2026
Merged

Make applyChatTemplate a buffer-based C ABI for objcxx interop#7
FuJacob merged 1 commit into
mainfrom
chat-template-c-abi

Conversation

@FuJacob
Copy link
Copy Markdown
Owner

@FuJacob FuJacob commented May 30, 2026

Summary

The applyChatTemplate shipped in #6 took a std::vector<ChatMessage> (a struct with std::string members) and returned std::string. That compiles under this package's test target (Cxx interop) but does not bridge into the Cotabby app target, which uses objcxx interop — a returned std::string has no usable Swift String initializer there, so the app literally cannot call it.

This replaces it with a detokenize-style C ABI that crosses the boundary cleanly:

int applyChatTemplate(const char* system_text, const char* user_text,
                      bool add_assistant, char* buffer, int buffer_size)

system + user are passed directly (autocomplete is always exactly those two turns), matching how the app already calls tokenize/detokenize with const char* + a caller buffer. The ChatMessage struct is removed.

Return contract:

  • > 0 — bytes written (<= buffer_size), the formatted prompt.
  • 0 — no model, no chat template, or render failure → caller falls back to the raw prompt path.
  • < 0-(required size); buffer too small, caller resizes and retries.

Validation

  • swift build — clean.
  • swift testExecuted 15 tests, 0 failures (3 model-dependent tests skipped without COTABBY_TEST_MODEL_PATH). The no-model guard test and the end-to-end chat-template assertion were updated to the new signature.

Risk / rollout notes

…nterop

The previous applyChatTemplate took a std::vector<ChatMessage> (a struct with
std::string members) and returned std::string. That compiles under the Cxx
interop mode this package's test target uses, but does NOT bridge into the
Cotabby app target, which uses objcxx interop — a returned std::string has no
usable Swift String initializer there, so the app could never call it.

Replace it with a detokenize-style C ABI that crosses the boundary cleanly:

    int applyChatTemplate(const char* system_text, const char* user_text,
                          bool add_assistant, char* buffer, int buffer_size)

system + user are passed directly (autocomplete is always exactly those two
turns), matching how the app already calls tokenize/detokenize with const char*
and a caller buffer. Return contract: >0 bytes written, 0 = no model / no
template / render failure (caller falls back to raw), <0 = -(required size) so
the caller can resize and retry. Drops the ChatMessage struct entirely.

Tests updated to the new signature; swift test green (15 tests, 0 failures,
3 model-dependent skipped).
@FuJacob FuJacob merged commit 403917c into main May 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant