Skip to content

add AsyncToolset#5127

Merged
longcw merged 21 commits intomainfrom
longc/async-toolset
Apr 1, 2026
Merged

add AsyncToolset#5127
longcw merged 21 commits intomainfrom
longc/async-toolset

Conversation

@longcw
Copy link
Copy Markdown
Contributor

@longcw longcw commented Mar 17, 2026

AsyncToolset lets tools run in the background while the voice agent keeps talking. Example

class TravelToolset(AsyncToolset):
    @function_tool
    async def book_flight(self, ctx: AsyncRunContext, origin: str, dest: str) -> str:
        await ctx.update(f"Searching flights from {origin} to {dest}...")
        # agent replies to user that it's searching the flight

        flights = await search_flights(origin, dest)
        await ctx.update(f"Found {len(flights)} options, booking the best one...")
        # agent will update to user when it's idle

        booking = await confirm_booking(flights[0])
        return f"Booked! Confirmation: {booking.id}"
        # final result will also be updated when both user and agent is idle

agent = Agent(tools=[TravelToolset(id="travel")], instructions="...")

How it works:

  • Progress updates: Long-running tools can push intermediate status messages (e.g. "Found 3 flights, confirming the best one...") into the conversation via ctx.update(). The tool updates and results are injected as tool outputs immediately when updates are created.

  • Delivery timing: Reply is generated when the agent is idle. Multiple updates from concurrent tools are merged into a single reply to avoid back-to-back replies. If the agent already replied after update injection (e.g. the user asked about it and the LLM answered from context), the tool reply is skipped.

  • Duplicate prevention: To avoid calling the same async tool multiple times for the same goal, by default it rejects the duplicate and lets the LLM re-call with explicit confirmation, cancel the existing task, or do nothing.

@longcw longcw changed the title add async toolset add AsyncToolset Mar 18, 2026
@longcw longcw marked this pull request as ready for review March 25, 2026 10:29
@chenghao-mou chenghao-mou requested a review from a team March 25, 2026 10:30
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Member

@theomonnom theomonnom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! pending and update could be merged

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@longcw longcw merged commit 51f5cb5 into main Apr 1, 2026
20 of 22 checks passed
@longcw longcw deleted the longc/async-toolset branch April 1, 2026 13:56
osimhi213 added a commit to de-id/livekit-agents that referenced this pull request Apr 3, 2026
* upstream/main: (26 commits)
  fix(cartesia): handle flush_done message in TTS _recv_task (livekit#5321)
  docs: add example agent replies to AsyncToolset (livekit#5313)
  add tag field to evaluation OTEL log records (livekit#5315)
  (phonic) Add `min_words_to_interrupt` to Phonic plugin options (livekit#5304)
  use delta aggregation temporality for otel metrics (livekit#5314)
  fix(openai realtime): support per-response tool_choice in realtime sessions (livekit#5211)
  add 7-day uv cooldown (livekit#5290)
  fix is_context_type for generic RunContext types (livekit#5307)
  evals: custom judges, tag metadata, and OTEL improvements (livekit#5306)
  feat: OTEL metrics for latencies, usage, and connection timing (livekit#4891)
  add session_end_timeout and gracefully cancel entrypoint on shutdown (livekit#4580)
  fix(core): reduce TTS output buffering latency (livekit#5292)
  (phonic) Update languages fields (livekit#5285)
  fix(cli): prevent api_key/api_secret from leaking in tracebacks (livekit#5300)
  fix(core): fix BackgroundAudioPlayer.play() hanging indefinitely (livekit#5299)
  add AsyncToolset (livekit#5127)
  append generate_reply instructions as system msg and convert it to user msg if unsupported (livekit#5287)
  fix(core): reset user state to listening when audio is disabled (livekit#5198)
  feat(mistralai): add ref_audio support to Voxtral TTS for zero-shot voice cloning (livekit#5278)
  (gemini-3.1-flash-live-preview): add warning for generate_reply (livekit#5286)
  ...
russellmartin-livekit pushed a commit that referenced this pull request Apr 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants