Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
da3b47a
refactor(media): share session image rendering across native and ACP …
Sun-GLiang Oct 22, 2026
51bb72b
fix(ci): sync audited dependencies and media inventories
Sun-GLiang Oct 22, 2026
e820fd6
fix(ci): include SVG validator in Windows package trigger
Sun-GLiang Oct 22, 2026
a4fe7a1
fix(media): preserve raster-only avatar validation
Sun-GLiang Oct 22, 2026
fab4d12
fix(acp): cancel media waits and deduplicate snapshot budgets
Sun-GLiang Oct 23, 2026
b0171ee
fix(ci): update audited dompurify and ip-address dependencies
Sun-GLiang Oct 23, 2026
9a5fe57
fix(ci): sync artifact budget and reconcile current main
Sun-GLiang Oct 23, 2026
831c6d7
merge: sync main and advance media compatibility epoch
Sun-GLiang Oct 24, 2026
ebcf72b
fix(media): preview XML SVG and preserve queued ACP output
Sun-GLiang Oct 24, 2026
d7bcc65
fix(media): bound ACP draining and SVG style parsing
Sun-GLiang Oct 25, 2026
0445512
fix(media): bound SVG sanitization and preserve ACP content
Sun-GLiang Oct 25, 2026
79516ab
refactor(media): split SVG previews into an independent change
Sun-GLiang Oct 25, 2026
ea548a8
merge: sync main before independent ACP media acceptance
Sun-GLiang Oct 25, 2026
f97aec6
fix(acp): isolate media captures and bound queued evidence
Sun-GLiang Oct 25, 2026
20fc4f6
fix(media): persist image reception and recover interrupted publications
Sun-GLiang Oct 25, 2026
f38b93c
refactor(media): share image presentation and separate admission policy
Sun-GLiang Oct 25, 2026
b4f4b0d
merge: share media foundations without ordering the feature PRs
Sun-GLiang Oct 25, 2026
7a3265c
test(media): distinguish background reception from lazy preview reads
Sun-GLiang Oct 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import type { ResolveImageDelivery } from '@maka/core/image-delivery';
/** One attachment port: the Composer stages through it and the transcript reads image bytes through it. */
export interface ComposerStagingServices extends ComposerAttachmentService {
readBytes(sessionId: string, artifactId: string): Promise<ArtifactBinaryReadResult>;
openSavedImage?: (sessionId: string, artifactId: string) => Promise<{ ok: boolean }>;
resolveImageDelivery?: ResolveImageDelivery;
}
const context = createServicesContext<ComposerStagingServices>('ComposerStagingServicesProvider');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import { useComposerStagingServices } from '../staging-services.js';

/** Transcript attachment reads come from the injected attachment port, never from the caller. */
export function StagedQuoteChatView(props: Omit<ComponentProps<typeof PlanChatView>,
'handleRef' | 'pendingQuotes' | 'onQuoteAnnotationSubmit' | 'onReadAttachmentBytes' | 'onResolveImageDelivery'>) {
'handleRef' | 'pendingQuotes' | 'onQuoteAnnotationSubmit' | 'onReadAttachmentBytes' | 'onResolveImageDelivery' | 'onOpenSavedImage'>) {
const staging = useComposerStaging();
const { readBytes, resolveImageDelivery } = useComposerStagingServices();
return <PlanChatView {...props} {...staging.chatViewQuoteProps} onReadAttachmentBytes={readBytes} onResolveImageDelivery={resolveImageDelivery} />;
const { readBytes, resolveImageDelivery, openSavedImage } = useComposerStagingServices();
return <PlanChatView {...props} {...staging.chatViewQuoteProps} onOpenSavedImage={openSavedImage} onReadAttachmentBytes={readBytes} onResolveImageDelivery={resolveImageDelivery} />;
}
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ export function QuoteCompanionPanel(props: {
onPromptAccepted?: (panelId: string, prompt: string) => void;
onActivityStateChange?: (panelId: string, active: boolean) => void;
}) {
const { attachments } = useWorkbarServices();
const { attachments, artifacts } = useWorkbarServices();
const mentions = useComposerMentionsContext();
const locale = useUiLocale();
const toast = useToast();
Expand Down Expand Up @@ -404,7 +404,7 @@ export function QuoteCompanionPanel(props: {
liveTurns={companion.liveTurns}
activeTurn={companion.activeTurn}
activeSession={companion.companionSession}
onReadAttachmentBytes={attachments.readBytes} onResolveImageDelivery={attachments.resolveImageDelivery}
onOpenSavedImage={artifacts.openPath} onReadAttachmentBytes={attachments.readBytes} onResolveImageDelivery={attachments.resolveImageDelivery}
deriveTurnPresentation={deriveTurnPresentation}
onEditUserMessage={(turnId) => {
const message = companion.messages.find(
Expand Down
1 change: 1 addition & 0 deletions apps/desktop/src/renderer/features/workhub/ports.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ export interface WorkHubServices extends WorkHubWorkspaceServices {
model: string;
}): Promise<void>;
readonly attachments: ComposerAttachmentService;
openSavedImage?: (sessionId: string, artifactId: string) => Promise<{ ok: boolean }>;
readAttachmentBytes(sessionId: string, artifactId: string): Promise<ArtifactBinaryReadResult>;
resolveImageDelivery?: ResolveImageDelivery;
prepareAttachments(sessionId: string, items: Array<{ approvalId: string; name: string; mimeType?: string } | { file: File }>): Promise<AttachmentRef[]>;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ export function WorkHubRoot() {
<div ref={history} className="workHubHistory" aria-hidden={!showConversation} inert={!showConversation}>
<WorkHubConversation
workLinks={links}
onReadAttachmentBytes={services.readAttachmentBytes} onResolveImageDelivery={services.resolveImageDelivery}
onOpenSavedImage={services.openSavedImage} onReadAttachmentBytes={services.readAttachmentBytes} onResolveImageDelivery={services.resolveImageDelivery}
onOpenWork={(id) => call(services.presentation.openSession(id))}
scrollBehavior="auto"
onNew={() => composer.current?.focus()}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import type { MakaBridge } from '../../../preload/bridge-contract.js';
import type { ComposerStagingServices } from '../../features/conversation/index.js';

export function createDesktopComposerStagingServices(
bridge: Pick<MakaBridge, 'attachments'> = window.maka,
bridge: Pick<MakaBridge, 'attachments' | 'app'> = window.maka,
): ComposerStagingServices {
return bridge.attachments;
return { ...bridge.attachments, openSavedImage: (sessionId, artifactId) => bridge.app.openArtifactPath(sessionId, artifactId) };
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ export function createDesktopWorkHubServices(
| 'runtimeHostProfiles'
| 'settings'
| 'attachments'
| 'app'
> = window.maka,
): WorkHubServices {
return {
Expand Down Expand Up @@ -93,6 +94,7 @@ export function createDesktopWorkHubServices(
bridge.connections.setDefaultModel({ slug: llmConnectionSlug, model }),
attachments: bridge.attachments,
readAttachmentBytes: bridge.attachments.readBytes,
openSavedImage: (sessionId, artifactId) => bridge.app.openArtifactPath(sessionId, artifactId),
resolveImageDelivery: bridge.attachments.resolveImageDelivery,
prepareAttachments: async (sessionId, items) => {
const result = await bridge.workHub.prepareAttachments(sessionId, items);
Expand Down
44 changes: 44 additions & 0 deletions docs/architecture/session-media-preview.zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# ACP 会话媒体发布

ACP 的 image、resource_link 和 embedded resource 通过 `PluginExecutorContext.publishMedia` 交给 Host,复用 `ChatImageDeliveryService` 的读取边界、联网策略、校验、队列、配额和 Artifact 保存。媒体格式由 Host 校验器决定;本变更沿用 PNG/JPEG/GIF/WebP 支持,SVG 格式与预览由独立变更承接。图片输出只携带当前 Session 的 Artifact 引用,不把 base64 写进消息事件。每次路径或 URL 发布独立捕获,避免同一轮覆盖源文件后复用旧图;已保存结果始终按 Artifact 引用回放,内联内容仍按字节去重。

正文结构化图片转为 `maka://runtime/attachments/<artifactId>` Markdown 引用。工具结果新增有序 `content.parts`,支持文本、已保存图片和 diff;旧 text/file_diff/image 保持可读。旧客户端无法解码新结果,兼容 epoch 从 229 升至 230,合并前需对照 main 确认。

通用复制层收集并重写混合内容的图片引用;ACP 会话仍受既有 provider-owned conversation cloning 限制,本变更不开放外部执行器会话复制。Goal 评估递归保留混合结果中的文字和 diff,省略图片,并沿用脱敏和长度限制。CLI 按顺序输出描述;没有媒体发布能力或发布失败时返回有界文字提示。

通知接收时完成准入并绑定当前轮次,串行处理以保留到达顺序。取消或断连中断媒体等待,但收尾已收到的文字和工具结果;prompt 返回后仍检测断连直至队列排空,迟到通知和媒体完成不会写回已结束轮次。慢图片归档会延迟后续文字,应在端到端验收中观察。

Host 最多 2 个并行捕获、128 个内存任务、512 个持久接收任务;解码中的内联字节最多 8 MiB,SQLite 暂存字节最多 32 MiB。有效原图接收上限为 8 MiB,预览仍限 2 MiB 和既有像素约束;超预览大小的已保存图片提供打开原件入口。ACP 每轮序列化媒体预算为 12 MiB,以容纳 8 MiB 原图的 base64 扩展。ACP 按唯一媒体计数和累计字节限制每轮保留内容,预算按裁剪后媒体对象的序列化 UTF-8 字节计费,audio 计入预算;未使用的媒体元数据、非文本 thought 和不展示的 user message 通知在入队前丢弃。重复工具快照复用媒体对象;纯文本和纯 diff 沿用旧规则,混合结果最多 64 项,超出明确提示省略。

验证覆盖媒体投影、真实 MIME、配额和网络失败、源文件删除后的归档回放、会话复制,以及取消和断连时的顺序与收尾。SVG 文件和代码块预览独立验收。

## 接收、保存与恢复

正文和 ACP 结构化媒体使用同一 Host 接收服务。已提交正文中的授权本地/远程图片主动登记持久任务,接收不再依赖图片是否滚入视口;预览字节仍按需读取。任务与 pending 状态原子写入 Artifact SQLite;已收到的字节先以有界 BLOB 暂存及 SHA-256 校验,再进入既有 fsync 文件和元数据提交流程。最终消息仍只存引用,最终图片仍存 Artifact 文件。

临时读取失败自动重试至多三次,次数与下次时间持久化;权限、格式、尺寸等永久失败退出队列,只保留可见失败原因。配额失败的有效暂存字节可供手动重试,但在新工作需要容量时按旧任务顺序回收;失败状态仍保留。接收缓冲区不是超出归档配额的永久文件仓库。删除会话同时清理任务和暂存字节。

消息正文结束与图片接收完成分别展示,不能用正文完成代表所有附件已保存。待接收时展示接收状态;已保存但超过预览上限时展示原件入口;真正缺失、被拒绝或超出归档配额的来源仍显示失败及可用的重试操作,不伪装成功。

启动恢复在中断运行封口前先完成已暂存字节的归档。ACP 图片的 Artifact 元数据保留准确 publicationRunId;如果已归档图片尚未进入该运行的 canonical 正文或工具结果,Host 写入一条明确标注的恢复图片正文,且对模型历史隐藏。确定性事件 ID 和 Artifact 创建时间保证再次崩溃后可幂等回放,不伪造原工具完成状态或顺序。已结束运行不重新打开;取消后晚到且未进入历史的图片不通过此路径追加。对于正文提交后尚未来得及登记任务的中断运行,启动补扫已提交正文恢复登记。

接收意图不等于已经收到字节:在首次源读取前消失的文件/URL无法凭空恢复;超过配额、磁盘写入失败和非法图片也不能承诺保存成功。已收到字节的进程崩溃恢复与源消失、已归档但尚未发布的消息恢复分别测试。
Loading
Loading