MIGRATION.md

Migration Guide: Updating Message Handling

This guide outlines the necessary changes to migrate your code from the previous message handling structure (using the Message class) to the new structure introduced to support multi-modal content and tool calls, based on the abstract ChatMessage class.

Core Changes

1. Message Class Removed: The old Message class has been removed.
2. ChatMessage Hierarchy: A new abstract base class ChatMessage has been introduced. Specific message types now extend this class:
   • SystemMessage
   • UserMessage
   • AssistantMessage
   • ToolMessage
3. Multi-Modal Content: Messages (primarily UserMessage, AssistantMessage, ToolMessage) now use a List<ContentChunk> for their content field to support text, images, etc. The ContentChunk interface has implementations like TextChunk, ImageURLChunk, etc.
   • getTextContent() helper methods were introduced to prevent having to loop over ContentChunks to find TextChunks
4. Tool Calls: AssistantMessage now includes a List<ToolCall> toolCalls field, and a ToolMessage message type was introduced to support function calling/tools.
5. Streaming Deltas: The DeltaChoice class (used in streaming responses via MessageChunk) now contains a DeltaMessage object (accessible via getDelta()). This DeltaMessage object, in turn, holds the partial delta fields: role, content (as List<ContentChunk>), and toolCalls.

Migration Steps

1. Update ChatCompletionRequest Message List

The messages field in ChatCompletionRequest now expects List<ChatMessage> instead of List<Message>.

Before:

import nl.dannyj.mistral.models.completion.Message;

// ...
List<Message> messages = ...;
        ChatCompletionRequest request = ChatCompletionRequest.builder()
                .messages(messages)
                // ...
                .build();

After:

import nl.dannyj.mistral.models.completion.ChatMessage;

// ...
List<ChatMessage> messages = ...; // See MessageListBuilder changes below
        ChatCompletionRequest request = ChatCompletionRequest.builder()
                .messages(messages)
                // ...
                .build();

2. Update MessageListBuilder Usage

The MessageListBuilder now works with ChatMessage and its subclasses.

Before:

import nl.dannyj.mistral.models.completion.Message;
import nl.dannyj.mistral.builders.MessageListBuilder;

// ...
List<Message> messages = new MessageListBuilder()
        .system("System prompt")
        .user("User query")
        .assistant("Assistant response")
        .build();

After:

import nl.dannyj.mistral.models.completion.ChatMessage;
import nl.dannyj.mistral.builders.MessageListBuilder;

// ...
List<ChatMessage> messages = new MessageListBuilder()
        .system("System prompt") // Creates SystemMessage
        .user("User query")     // Creates UserMessage
        .assistant("Assistant response") // Creates AssistantMessage (text only)
        // .assistant(listOfToolCalls) // Use this overload for tool calls
        // .tool("Tool result", "tool_call_id_123") // Creates ToolMessage
        .build();

// Alternatively, add pre-constructed messages:
// messages = new MessageListBuilder()
//     .message(new SystemMessage("System prompt"))
//     .message(new UserMessage("User query"))
//     .message(AssistantMessage.fromText("Assistant response")) // Factory method
//     .build();

3. Update Handling of Non-Streaming Responses (ChatCompletionResponse)

The message field within the Choice object (obtained from ChatCompletionResponse.getChoices()) is now specifically an AssistantMessage.

Before:

ChatCompletionResponse response = client.createChatCompletion(request);
Message responseMsg = response.getChoices().get(0).getMessage();
String content = responseMsg.getContent();
System.out.println(content);

After:

import nl.dannyj.mistral.models.completion.message.AssistantMessage;
import nl.dannyj.mistral.models.completion.content.ContentChunk;
import nl.dannyj.mistral.models.completion.content.TextChunk;

// ...
ChatCompletionResponse response = client.createChatCompletion(request);
AssistantMessage responseMsg = response.getChoices().get(0).getMessage();

// Extract text content (handle potential multi-modal content)
String textContent = "";
if (responseMsg.getContent() != null) {
    for (ContentChunk chunk : responseMsg.getContent()) {
        if (chunk instanceof TextChunk textChunk) {
            textContent += textChunk.getText(); // Append text from TextChunks
        }
        // Handle other chunk types (ImageURLChunk, etc.) if necessary
    }
}
System.out.println(textContent);

// Check for tool calls if applicable
if (responseMsg.getToolCalls() != null) {
    // Process tool calls
}

4. Update Handling of Streaming Responses (ChatCompletionChunkCallback)

The onChunkReceived method receives MessageChunk. The structure within MessageChunk.getChoices().get(0) (which is a DeltaChoice) has changed. Access role, content, and toolCalls directly from the DeltaChoice object.

Before:

client.createChatCompletionStream(request, new ChatCompletionChunkCallback() {
    @Override
    public void onChunkReceived(MessageChunk chunk) {
        // Old structure assumed delta was nested in a Message object
        String contentDelta = chunk.getChoices().get(0).getMessage().getContent();
        if (contentDelta != null) {
            System.out.print(contentDelta);
        }
    }
    // ... onComplete, onError
});

After:

import nl.dannyj.mistral.models.completion.DeltaChoice;
import nl.dannyj.mistral.models.completion.message.DeltaMessage; // <-- Add this import
import nl.dannyj.mistral.models.completion.message.MessageChunk; // <-- Updated import path
import nl.dannyj.mistral.models.completion.content.ContentChunk;
import nl.dannyj.mistral.models.completion.content.TextChunk;
// ...
client.createChatCompletionStream(request, new ChatCompletionChunkCallback() {
    @Override
    public void onChunkReceived(MessageChunk chunk) {
        if (chunk.getChoices() == null || chunk.getChoices().isEmpty()) return;

        DeltaChoice deltaChoice = chunk.getChoices().get(0);
        DeltaMessage delta = deltaChoice.getDelta(); // <-- Get the DeltaMessage

        // Check for text content delta using the convenience method
        String textDelta = delta.getTextContent();

        if (textDelta != null) {
            System.out.print(textDelta);
        }

    }
    // ... onComplete, onError
});

5. Update Handling of Choice Object in ChatCompletionResponse

The Choice class, which is part of the ChatCompletionResponse.getChoices(), has the following breaking changes:

finishReason Type Change

• The finishReason field is now of type FinishReason (an enum) instead of a raw String.
• Before: String finishReason = choice.getFinishReason();
• After: FinishReason finishReason = choice.getFinishReason();
• To get the underlying string value (e.g., "stop", "length"), you can call finishReason.getReason().

Example: Handling finishReason (within the context of iterating choices from ChatCompletionResponse)

Before(inside loop processing choice):

String reason = choice.getFinishReason();
    if ("stop".equals(reason)) {
        // ...
        }

After(inside loop processing choice):

    import nl.dannyj.mistral.models.completion.FinishReason; // Ensure import
// ...
FinishReason reason = choice.getFinishReason();
if (reason == FinishReason.STOP) {
    // ...
}
// Or, to compare with the string value:
// if ("stop".equals(reason.getReason())) { ... }

logProbs Field Removal

• The logProbs field (previously a String) has been removed from the Choice class as it is no longer returned by the API.