From da60005a806e602eb86db5b4519fb6dcfd2f4798 Mon Sep 17 00:00:00 2001 From: fan Date: Tue, 10 Nov 2026 17:44:13 +0800 Subject: [PATCH 1/2] Retire official avatars from the plugin skills ChatCut no longer offers official avatars, and the hosted MCP has no manage_avatar catalog or import action, so the old digital-human skill sent agents to calls that now fail. Every host now uses the character flow from ChatCut's current agent skill: manage_character for the person and submit_avatar_video with characterId. - Claude Code and Antigravity share the Codex copy instead of keeping their own; Grok (and Cursor through it), Kimi and Qwen Work keep their host frontmatter. Antigravity drops its official-avatar reference file. - Widget Forms maps character choices to characterId with the character's main image, in place of avatar identity and catalog preview handling. - README no longer marks characters and products as Desktop only. Co-Authored-By: Claude Opus 5.5 --- README.md | 4 +- antigravity/skills/digital-human/SKILL.md | 278 +------- ...atar-generation-safety-and-verification.md | 250 ------- antigravity/skills/widget-forms/SKILL.md | 9 +- claude/skills/digital-human/SKILL.md | 530 +------------- codex/skills/digital-human/SKILL.md | 675 ++++++------------ codex/skills/widget-forms/SKILL.md | 25 +- grok/skills/digital-human/SKILL.md | 12 +- grok/skills/widget-forms/SKILL.md | 12 +- kimi/skills/digital-human/SKILL.md | 644 +++++------------ qwen/skills/digital-human/SKILL.md | 569 +++++---------- qwen/skills/widget-forms/SKILL.md | 2 +- 12 files changed, 617 insertions(+), 2393 deletions(-) mode change 100644 => 120000 antigravity/skills/digital-human/SKILL.md delete mode 100644 antigravity/skills/digital-human/references/avatar-generation-safety-and-verification.md mode change 100644 => 120000 claude/skills/digital-human/SKILL.md diff --git a/README.md b/README.md index 5625fd3..da44d37 100644 --- a/README.md +++ b/README.md @@ -129,8 +129,8 @@ ChatCut edits your own footage and generates new media, and the result is an edi **Saved across projects** -- Characters: people you feature again, with photos, a description and a voice, for consistent generated shots and presenter videos *(Desktop)*. -- Products: photos, selling points, price and specs for product ads, so scripts stick to what you saved *(Desktop)*. +- Characters: people you feature again, with photos, a description and a voice, for consistent generated shots and presenter videos. +- Products: photos, selling points, price and specs for product ads, so scripts stick to what you saved. - Memory of your preferences, and Workflow Skills that save a working editing process for reuse. **Writing and planning** *(Desktop)* diff --git a/antigravity/skills/digital-human/SKILL.md b/antigravity/skills/digital-human/SKILL.md deleted file mode 100644 index ea251bd..0000000 --- a/antigravity/skills/digital-human/SKILL.md +++ /dev/null @@ -1,277 +0,0 @@ ---- -name: digital-human -description: Create and manage script- or audio-driven AI avatar videos from an official or saved presenter, an imported portrait, or a representative still prepared from video. Use for 数字人、数字分身、虚拟人、照片开口说话、人像口播、改稿不重拍, AI avatar, AI avatar video, avatar video, talking avatar, talking photo, photo avatar, video avatar, AI presenter, virtual presenter, virtual spokesperson, digital human, or a person's digital twin, including choosing or creating the avatar and deciding its voice, script, aspect ratio, and output quality. Do not use for a static profile-picture avatar, game or 3D character creation, an industrial digital twin, ordinary B-roll, generic image-to-video, video translation, talking-head editing, lip-sync dubbing alone, or TTS-only requests. -user-invocable: true ---- - -Before using this skill, read [the required continuation](references/avatar-generation-safety-and-verification.md) in full. It is part of this skill and contains mandatory workflow, tool-contract, and safety instructions; do not act from this entry file alone. - - -# Digital Human - -If `manage_custom_voice`, `manage_avatar`, or `submit_avatar_video` are not loaded yet, load them with `ToolSearch` before calling them, so you use their real parameters instead of guessing. - -Create a reusable presenter identity or generate a lip-synced presenter video -without exposing the underlying provider. Treat avatar selection, speech, script, -format, consent, generation, and verification as one stateful conversation. - -## When to Use - -Use this skill when the user wants to: - -- Make an official or saved AI avatar, talking avatar, or virtual presenter speak. -- Turn a consented portrait into a reusable digital-human identity. -- Use a representative still from an imported video as a photo avatar. -- Create a synthetic AI presenter from a description, when live capabilities allow it. -- Draft or revise the narration for a digital-human video. -- Regenerate, reorder, remove, retry, or accept segments from an avatar-video batch. - -Treat **AI avatar** as the broad default English concept. **Digital twin** means a -reusable likeness of a specific real person; **photo avatar** or **talking photo** -means a still-image source; **AI presenter**, **virtual presenter**, and **virtual -spokesperson** describe the presenter's role. **Digital human** is valid but is often -used for broader enterprise or interactive experiences. - -Route adjacent requests elsewhere: - -- Speech, voice audition, or voice cloning without avatar video: use the Voice Skill. -- Ordinary footage cleanup or presenter editing: use the Talking Head Guide. -- Generic image-to-video or video generation without a speaking avatar: use the - Video Generation Skill. -- Translation or dubbing of an existing video: use the dedicated translation or - dubbing workflow when available. - -## Required Inputs - -Resolve these slots before submission, but do not ask for them in a fixed order: - -1. **Target project** — use the active project when it is unambiguous. -2. **Avatar identity and ready revision** — an official avatar, a saved avatar, or - a newly created photo or synthetic avatar. -3. **Speech source** — one exact available voice, one cloned voice, or one imported - audio asset. A text-driven source also requires the final script. -4. **Final script** — preserve the user's wording and punctuation. Show any AI-written - draft in full and let the user edit or approve it before generation. -5. **Aspect ratio** — `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. Explicit user - choice wins; otherwise omit it so the backend uses `auto` and preserves the - selected avatar's original framing. Do not substitute the current project ratio. -6. **Resolution** — `1080p` by default; use `720p` when requested or when live - capabilities require it, and explain any fallback. -7. **Optional controls** — motion guidance only when the live capability response - says it is supported. For an official catalog avatar, use its returned - `hasBackground` value: preserve the original scene when true and request a - transparent background when false. For a saved/custom avatar, preserve its original - background by default. Request a transparent background only when the user explicitly - asks to remove the background or use transparency. Do not ask the user to choose a - background mode; briefly state the resolved default before submission. An explicit - user request to preserve or remove the background always wins. - -For avatar creation also resolve: - -- A non-empty name. Keep numeric validation limits out of the question label; - only explain a limit if the submitted name actually fails validation. -- One final avatar-creation source: an imported JPEG/PNG portrait, a - representative JPEG/PNG still prepared from an imported video, or a prompt - for a synthetic identity when supported. -- Explicit consent when a real person's likeness is involved. - -Do not pass a video directly to photo-avatar creation unless the live capability -contract explicitly supports it. For an imported video, use the host's prepared -representative still or help select a clear, unobstructed, front-facing frame, then -use that image asset as the source. - -When a user attaches or references a video containing a person and says "this -avatar", "this person", "use her/him", `这个形象`, `这个人`, `用她`, `用他`, or an -equivalent phrase, resolve the person in that exact attachment as the intended -likeness source. Do not treat the attachment as the requested speech source merely -because it has audio or a transcript. If the user also supplies a new topic or an -approved script, that new topic or script wins; do not read or reuse the video's -spoken content unless the user explicitly asks for it. - -The attached video is a prospective avatar source, not a ready `identityId`. Enter -the video-to-avatar creation flow: prepare a clean representative still, obtain the -required likeness consent, create the identity, and then reuse that identity for the -requested generation. Do not show official or saved avatar choices while this -prospective source remains valid. Offer other identities only when the user declines -to use the person in the video or no eligible face frame can be prepared. - -If "this" could reasonably mean the person's likeness, the video's existing pixels, -or its spoken content, ask exactly one targeted clarification before choosing a -workflow: "Do you want to use the person in this video as the avatar and have them -speak your new script?" Do not replace this with a generic "What do you want to do -with this video?" question. - -When the user is about to upload or pick a video for avatar creation, tell them up -front that the chosen frame must show the face clearly with nothing covering it — no -burned-in subtitles, captions, watermarks, stickers, or hands over the face — and -prefer a frame with a neutral, front-facing pose. Never upload an inspection contact -sheet or any preview that carries an overlaid time label as the avatar source; use a -clean frame with no added markings. - -A video source also carries the person's voice. After the avatar is created from a -video, offer to clone that voice from the same video so the avatar speaks with it: - -1. Run the Voice Skill's cloning flow (`manage_custom_voice action=clone` with the - video's asset and explicit voice-cloning consent — likeness consent alone does - not cover the voice). -2. When the cloned voice is ready, call - `manage_avatar action=set-default-voice identityId= voiceRef=` - so it becomes this avatar's saved default voice. -3. Later generations with this avatar should use the returned `defaultVoiceRef` by - default, while still letting the user pick another voice. - -This is an offer, not an automatic step: skip it when the user declines or the video -has unusable audio (music, multiple speakers, heavy noise). - -## Workflow - -This is an intent-driven constraint resolver, not a linear wizard. Do not walk the -user through every section or ask for fields in a preset order. - -On every turn: - -1. Extract all facts already supplied by the user, attachments, active project, - selected media, saved avatar state, and prior answers in this request. -2. Resolve deictic phrases such as "this", `这个`, `这个形象`, and `这个人` against - the attached or selected media and the user's action words before inferring a - generic media workflow. -3. Infer the user's immediate requested action: explore, choose, create, revise, - generate, inspect progress, retry, reorder, remove, accept, place, or export. -4. Compute only the blockers for that action. Defaults and live saved bindings count - as resolved unless they conflict with an explicit request. -5. If nothing blocks the action, execute it immediately. -6. Otherwise ask about the single most important missing or ambiguous constraint, - preferably with structured choices, then recompute from the new state. - -For example, a user who names a ready avatar, supplies approved text, and accepts its -saved voice should not be asked to choose them again. A user inspecting a running -batch does not need to resolve a new script or aspect ratio. Consent is requested only -for the creation or cloning action that needs it. - -### Load live capabilities and state when relevant - -Use `ToolSearch` to load `manage_avatar` and `submit_avatar_video` if they are not -already visible. Call `manage_avatar` for live capabilities before -offering creation or advanced options. - -If the user has neither selected an exact identity nor supplied a prospective -likeness source, list the current official and saved identities before recommending -one. Do not infer availability from chat history, examples, or provider knowledge. -For official avatars, use the live catalog's `tags` as factual selection filters and -its `description` as recommendation guidance. Match those fields against the user's -requested presenter and use case, and never infer a missing age, gender, ethnicity, -role, setting, or capability from the preview's appearance. -For the empty saved-avatar recommendation state, call the official catalog with -`limit: 5`; do not page through the catalog or turn a full catalog page into one -recommendation form. - -An avatar attached from ChatCut's Digital Humans library or native visual picker is -already an exact selection. Reuse its attached ChatCut `avatarId` as `identityId` -across later confirmation and retry turns. An official attached identity is directly -usable by `submit_avatar_video`; it does not need to be imported or rediscovered in -the first catalog page. Before resolving speech for an attached identity, call -`manage_avatar` with `action: get` and that exact ID unless a live result in the -current turn already supplies its `defaultVoiceRef`. Never reject or replace an -attached avatar just because it is absent from the five-item recommendation page. - -Only use identities and revisions reported as ready. Pending, failed, deleting, -deleted, or orphaned identities are not valid generation inputs. - -If the digital-human tools are unavailable, explain that this generation capability -is not connected. Do not silently substitute generic video generation or call the -backend HTTP routes directly. - -### Resolve the avatar when it is missing or invalid - -Respect an explicit user choice. Otherwise: - -- A person-containing video explicitly or provisionally resolved as the likeness - source: continue the video-to-avatar creation flow. Do not enter the generic avatar - picker and do not ask the user to choose among saved or official identities. -- No ready saved avatar: show at most five suitable official avatars and include the - reserved **create** card in the same choice surface. This is a strict empty-state - contract: never render a sixth official recommendation, and never omit the - creation option merely because official avatars are available. -- Ready saved avatars exist: show them first (up to five total cards), then the - reserved create card. -- Exactly one ready saved avatar: offer it first and ask whether to use it. -- Multiple ready saved avatars: prefer higher live usage count; when counts tie, prefer - the newest creation. If usage metadata is absent, rank by newest creation only; - never invent a frequency. - -In embedded ChatCut, avatar identity selection is media-preview-only and is a strict exception to the generic -single-branch `` rule. Always use visual cards through the Widget Forms -Skill, including when exactly one ready saved avatar exists. Never ask a text-only -category question such as "use the saved avatar" versus "browse official avatars", -and never use ``, ``, or label-only identity cards for avatar -selection. For an avatar card in ChatCut's native widget — official or the user's own -saved avatar — write only its exact `identityId` as `avatar-id`; the host resolves the -current name, preview video or fallback image, aspect ratio, and submitted value from the live avatar -identity even when the tool payload does not expose a preview URL. Never copy `value`, -`name`, `media`, or `aspect-ratio` onto an `avatar-id` option. If the host cannot -resolve preview media, omit that identity rather than falling back to text. Prefer -the live preview video when available and use the preview image only as its poster -or fallback. When -the user has ready saved avatars, place their cards before official recommendations. -Ask one most important missing question at a time and do not repeat answered questions. - -The raw tags and visual-card requirements below describe embedded ChatCut only. In Antigravity, the `widget-forms` host adapter takes priority: use concise text choices with stable `identityId` values mapped internally to returned names and descriptions, and direct the user to ChatCut's Digital Humans library for previews. Never emit raw ChatCut tags or promise chat avatar video. Preserve the separate `create_avatar` value; it returns to this workflow for a conversation attachment plus `asset-import`, rather than automatically opening a native dialog. This adapter changes presentation only; identity readiness, required inputs and consent checks remain mandatory. -For a saved identity whose `list` row has no preview URLs, call `manage_avatar -action=get` for that exact candidate before rendering it; use the selected ready -revision's returned preview media and do not invent or copy media from another avatar. - -One reserved label-only option value opens a native editor surface instead of -submitting an answer. Write it with no `media`: - -- `create_avatar` — opens the native creation dialog where the user uploads a photo - and creates a personal avatar. Keep its card label a short "create your own - avatar" phrase (e.g. 创建专属数字人); do not mention uploading a photo — the - dialog explains that itself. - -Clicking that card does not produce a form answer. When the user finishes in the -native dialog, their chosen or newly created avatar is sent back automatically as -this question's answer with the avatar attached; treat that reply as the avatar -selection and do not re-ask. - -The selection shape is therefore one `` containing up to five avatar -preview cards (saved first, then official) followed by the creation card. Localize -the visible question and card label to the conversation language: - -```text - - - - - - -``` - -Creating an identity is separate from generating a video: - -1. Resolve the source or prompt and name. -2. Obtain required consent. -3. Create the identity through `manage_avatar`. -4. Wait until it is ready or terminal. Where `track_progress` offers - `target=avatar`, call `action=wait` with the `identityIds` and do what its - result says; otherwise re-read it with `manage_avatar action=get refresh=true` - in a later turn. -5. Save it under **My avatars**. Do not auto-generate a video merely because avatar - creation succeeded. - -When asking for a portrait in an embedded ChatCut Widget, always mark the upload as an avatar source so -the host uses the same validation, JPEG normalization, private transport storage, and -upload-readiness behavior as the native Digital Humans dialog: - -```text - - - - -``` - -Do not use an unmarked generic `` upload for avatar creation. -Published plugin forms do not provide this native upload control. Ask the user to -attach the portrait through their host, then use the Asset Import Skill and pass the -returned ChatCut image asset id to `manage_avatar`. diff --git a/antigravity/skills/digital-human/SKILL.md b/antigravity/skills/digital-human/SKILL.md new file mode 120000 index 0000000..79588af --- /dev/null +++ b/antigravity/skills/digital-human/SKILL.md @@ -0,0 +1 @@ +../../../codex/skills/digital-human/SKILL.md \ No newline at end of file diff --git a/antigravity/skills/digital-human/references/avatar-generation-safety-and-verification.md b/antigravity/skills/digital-human/references/avatar-generation-safety-and-verification.md deleted file mode 100644 index 8fd0b74..0000000 --- a/antigravity/skills/digital-human/references/avatar-generation-safety-and-verification.md +++ /dev/null @@ -1,250 +0,0 @@ - -### Resolve speech only when the requested action needs it - -For an official avatar, never silently use its paired voice and never immediately -open the broader voice catalog. When the live result provides `defaultVoiceRef` and -the user has not already made this choice, first ask one blocking text-only decision -in the conversation language: whether to use this avatar's official built-in voice -or choose another voice. In embedded ChatCut, render that decision with localized -``, then stop and wait. -If the user accepts, resolve speech with that exact `defaultVoiceRef` and do not call -`manage_avatar action=voices`. Call `action=voices` only after the user chooses -another voice, or when the live identity has no `defaultVoiceRef`. Do not ask for -language or script and then pre-emptively call `action=voices` in the same turn. - -For a saved or newly created avatar, reuse its saved voice binding when present and -confirm it; otherwise offer available voices. When the user settles on a voice they -want this avatar to keep, save it with `manage_avatar action=set-default-voice`; -`list`/`get` then return it as the avatar's `defaultVoiceRef`. - -For voice discovery, audition, or cloning, follow the Voice Skill and reuse its live -voice list, consent gate, and generated voice asset. Do not duplicate voice-cloning -logic here. - -In Antigravity, use `widget-forms` text voice choices and audition through the clean ChatCut editor link, keeping each exact `voiceRef` mapped internally and retaining the localized `clone_voice` action. Ask script confirmation as a separate text field when needed. The following native widget presentation rules apply only inside embedded ChatCut. - -When an exact voice still needs to be selected, never present voice names as prose -beside a free-text field. Use the exact `voiceRef` returned by `manage_avatar` as the -submitted value. Voice selection is audio-preview-only and is a strict exception to -the generic `` rule: - -- Never render ``, ``, prose voice names, or label-only voice - options. A voice without a returned playable `previewUrl` must be omitted from the - recommendation surface rather than presented as an unplayable choice. -- When the user wants to audition voices, prefer up to five suitable available - official, preset, or custom voices that actually include `previewUrl`. Render them as one - required ``; `hasPreview: true` - without `previewUrl` is not playable and must not be presented as an audition - card. Always append one no-media `` action, localized to the conversation language, as the - final card. This clone option is required on every recommended-voice surface. -- An official avatar's paired default voice may still be used without listing the - broader official voice catalog. Do not show that default voice as an audition - card unless an actual audio `previewUrl` was returned for it. Never reuse the - avatar image or video URL as a voice preview. -- Copy each returned `previewUrl` exactly. In particular, keep - `/voice-samples/...` root-relative; do not expand it to `chatcut.com` or invent - another host. -- When script text is also being confirmed, put the voice selector and script field - in the same widget. Do not make the user type a voice name into the script field. - -```text - - - - - - - -``` - -If the source began as a user video, ask whether the user wants that video's voice. -If yes, follow the Voice Skill's explicit voice-cloning authorization flow before -using it. An attachment alone is not permission to clone. If no, continue with the -normal voice picker. - -An existing project audio asset may drive the avatar directly. When audio is the -speech source, do not invent or require text unless the user also wants a transcript -or script review. - -### Resolve a script only for text-driven generation - -The script may be pasted by the user, derived from selected project material, or -written collaboratively. When drafting: - -1. Ask only for missing intent such as audience, goal, tone, facts, and duration. -2. Produce the complete proposed script. -3. Let the user edit it or approve it explicitly. -4. Submit exactly the confirmed content. - -Each explicit text segment must contain 1–5000 characters. Omit explicit segments -for an ordinary script so the backend generates one continuous video up to 5000 -characters. Use explicit segments only when the user requests meaningful paragraph -or shot boundaries. Longer text is split automatically and may create at most 10 -segments. Never summarize, translate, drop, duplicate, or reorder content to fit a -limit. Preserve wording and -punctuation; non-semantic line whitespace may be normalized automatically. - -Text-driven avatar generation is limited by the 5000-character segment boundary, -not by an estimated audio duration. ChatCut TTS and cloned voices are rendered to -audio before avatar-video submission. Long audio-backed text is prepared as smaller -ordered segments, and each generated audio file is validated from its real duration. -Existing project audio longer than 600 seconds is automatically split into ordered -parts before submission. Pass the original asset once; never split it manually or -submit one generation call per part. If a progress update is useful, say only that -ChatCut is splitting the long audio while preserving the complete content and order. -Do not add implementation details about where or how the work runs, and never expose -hidden derived audio assets. Never shorten or summarize approved content to satisfy -these limits. - -### Resolve output settings from intent and available defaults - -Preserve the selected avatar's original framing by default: omit `aspectRatio` and -let the backend resolve it to `auto`. Never use the current project ratio as the -avatar-video default. If the user explicitly asks for a different ratio, pass that -request instead. - -For an official catalog avatar, resolve the default from the `hasBackground` field -returned by `manage_avatar action="catalog"`: - -- `hasBackground: true` — pass `removeBackground: false` and preserve the original - scene. -- `hasBackground: false` — pass `removeBackground: true` so the result is a - transparent WebM when transparency is supported. - -For a saved/custom avatar without catalog metadata, keep `removeBackground: false` as -the default and preserve the source image background. Pass `removeBackground: true` -only when the user explicitly asks for a transparent background or background removal. -Do not turn this into a blocking question. Before submission, briefly tell the user -whether the original scene will be preserved or the result will be transparent. An -explicit user request overrides the catalog default. If transparency is unsupported, -explain the limitation and keep the original background. - -Do not promise crop, background removal, motion control, transparency, resolution, -or prompt-based identity creation until the live capability response confirms it. - -### Submit as soon as the required constraints are resolved - -Summarize the resolved avatar, voice or audio source, script state, aspect ratio, and -resolution before the first generation when any choice remains consequential. Then -call `submit_avatar_video` once. - -Treat long-script output as an ordered batch of avatar-video segments, not as one -audio file or one opaque job. After submitting, tell the user that generation has -started and end the turn. Never keep the turn open for it: no Bash sleep and no -repeated `track_progress` or `manage_avatar action=get-batch` reads. When a follow-up -step in this request needs the completed videos and `track_progress` describes -`action=wait` as registering an automatic follow-up, call it once with the jobIds and -a note, then end the turn. The project library shows the in-progress generation and -receives the completed video assets in the background. - -Call `track_progress` in a later user turn only when the user asks for status or resumes -a follow-up step that depends on the completed asset, such as placing it on the timeline -or reviewing the result. When a job is still non-terminal, say that generation is still -running without quoting its numeric progress: provider progress values are coarse -internal stages, not user-facing completion estimates. Use `manage_avatar -action=get-batch` when ordered segment and attempt state is needed. Keep the stable -segment key and original order. - -For a failed segment, offer a targeted retry. Do not resubmit successful segments or -the entire batch without the user's instruction. Use the management tool for supported -segment regeneration, removal, reordering, attempt selection, and acceptance. - -Generated results become normal ChatCut video assets. Add them to the timeline only -when the user requested placement or it is clearly part of the active editing task. -Do not export unless asked. - -## Provider Selection - -Provider selection is an internal ChatCut backend detail. Always call the -ChatCut-native tools and let the backend construct provider requests, resolve private -IDs, enforce capabilities, and persist results. Never call an underlying provider API -or CLI directly from this skill. - -Select only through ChatCut capabilities, catalog results, saved bindings, and the -native tools. User-facing language is limited to concepts such as: - -- **Official avatars** and **My avatars** -- **Official voices**, **cloned voices**, and **project audio** -- **Photo avatar** and **synthetic avatar** - -Provider names, engine or model names, provider-specific avatar or voice IDs, raw -provider URLs, request payloads, and provider error messages are internal. Never show -them in prose, option labels, progress updates, or failure messages. Translate failures -into actionable product language while preserving the real status. - -Do not route around ChatCut's feature gates, quotas, billing checks, safety policy, -catalog visibility, or provider registry. - -## Consent and Safety - -A real-person photo avatar requires an explicit affirmative confirmation immediately -before creation. Use the user's language and keep the meaning exact. For Chinese: - -> 我确认拥有该肖像,或已获得创建和使用该数字人形象的授权,并承诺不将其用于 -> 冒充他人、欺诈或其他违法用途。 - -For English: - -> I confirm that I own this likeness or have permission to create and use this -> digital avatar, and I will not use it for impersonation, fraud, or unlawful activity. - -En español: - -> Confirmo que soy titular de los derechos sobre esta imagen o que tengo permiso para crear y usar este avatar digital, y no lo utilizaré para suplantar identidades, cometer fraude ni realizar actividades ilícitas. - -Do not infer consent from an upload, a previous unrelated confirmation, ownership of -the project, or a third party's instruction. Record consent only through the native -tool's consent fields. Synthetic prompt-created identities do not require real-person -likeness consent unless the prompt or references identify a real person. - -Refuse non-consensual impersonation, deceptive identity use, fraud, harassment, -sexual exploitation, or attempts to evade safeguards. Do not create a real-person -identity from uncertain authorization. Voice cloning has a separate consent gate; -follow the Voice Skill even when avatar consent was already obtained. - -## Verification - -Before reporting success, verify through live tool results: - -1. The chosen identity revision was ready when submitted. -2. The job or batch reached a completed terminal state and returned output asset IDs. -3. Every expected segment key exists exactly once and appears in the confirmed order. -4. Text-backed segment manifests preserve the confirmed wording and punctuation, - allowing only documented non-semantic whitespace normalization. -5. The generated videos exist as usable project or media-library assets. -6. Any requested timeline placement is present and ordered correctly. - -Preview the generated assets or composed timeline when visual verification is available. -Do not claim the face, lip sync, framing, or background looks correct from status fields -alone. If visual inspection is unavailable, say what was structurally verified. - -## Hard Rules - -- Never expose provider, engine, or model identity to the user. -- Never call an underlying provider directly or construct provider payloads in the - conversation layer. -- Never invent an avatar, voice, capability, usage count, saved binding, or asset ID. -- Never generate from a non-ready identity revision. -- Never treat a video as a supported identity source without an explicit live capability; - prepare a representative still for the current photo-avatar contract. -- Never pass an external source URL to avatar creation; import the source into the - target ChatCut project first. -- Never infer likeness consent or voice-cloning consent from an attachment. -- Never clone a video's original voice without the Voice Skill's explicit authorization. -- Never silently rewrite, translate, truncate, reorder, or duplicate confirmed script text. -- Never ask for information already available from the project, attachment, or live tools. -- Never run a fixed wizard or ask for avatar, voice, script, ratio, and resolution in a - preset sequence; resolve only what the current user action still needs. -- Ask one most important missing question at a time; prefer structured visual choices. -- Never show provider-private IDs, URLs, payloads, or raw errors in user-visible output. -- Never bypass ChatCut tools, feature gates, billing, quota, or safety checks. -- Never retry a whole successful batch because one segment failed. -- Never claim visual quality without inspecting the generated result. -- Never place media on the timeline or export it unless the task calls for that action. -- For official catalog avatars, obey the returned `hasBackground` default: preserve a - configured scene background and remove a configured absent background. For - saved/custom avatars, preserve the source background unless the user explicitly asks - for transparency or background removal. Explicit user intent always overrides either - default. diff --git a/antigravity/skills/widget-forms/SKILL.md b/antigravity/skills/widget-forms/SKILL.md index 2c8bd63..694cf64 100644 --- a/antigravity/skills/widget-forms/SKILL.md +++ b/antigravity/skills/widget-forms/SKILL.md @@ -9,9 +9,9 @@ Antigravity has no documented native option-card widget. Use concise ordinary te ## Stable visual-preview path -When the host can display images in chat, use normal chat image bubbles only for Motion Graphic references or Design Style preset thumbnails. Otherwise present text descriptions with the same choices. +When the host can display images in chat, use normal chat image bubbles only for Motion Graphic references, Design Style preset thumbnails or character photos. Otherwise present text descriptions with the same choices. -For an MG reference or Design Style choice: +For an MG reference, Design Style or character choice: 1. Select a small relevant set, normally 3–6 options. 2. Obtain each returned thumbnail or preview-image URL from the live tool result or the pinned widget-media manifest. Never invent or rewrite the URL. @@ -20,7 +20,7 @@ For an MG reference or Design Style choice: 5. Immediately follow them with concise localized choices, using concise ordinary text choices. Keep every visible label mapped to the exact returned preset or reference id. 6. Stop and wait for the user's selection. If an image cannot be downloaded or displayed, omit that bubble and keep the corresponding choice; do not expose the source URL. -Do not use this image-bubble path for avatars, voices, audio samples, generated project video, or general project preview. +A character choice uses this path too, with each character's main image URL from `manage_character get` and its `characterId` as the stable id. Do not use it for voices, audio samples, generated project video, or general project preview. ## Text questions and choices @@ -30,7 +30,6 @@ For every other question, ask only the minimum blocking question. Use concise or - `single_choice`: use concise ordinary text choices; accept the selected label. - `explicit_consent`: quote the caller's full consent statement and require an explicit affirmative response. Never infer consent from an attachment or another answer. - `audio_reference`: ask the user to attach the source file in the conversation, then load `asset-import`. Return the imported ChatCut audio `assetId`; never treat a local path, attachment URL, or raw bytes as an asset id. -- `avatar_choice`: show names and concise returned descriptions as text. Tell the user that visual previews are available in ChatCut's Digital Humans library when previewing matters. - `voice_choice`: show names and concise returned traits as text. Provide the clean ChatCut editor link supplied by the project workflow and tell the user to audition voices there or in its voice library before choosing when sound matters. ## Audio and video restrictions @@ -46,5 +45,5 @@ This adapter implements the caller's host-neutral input contract; it does not de - `playable_single_choice`: present localized text choices with stable values mapped internally; users audition samples through the clean ChatCut editor link. Preserve every caller-provided voice id, label, traits and preview association internally; do not print raw sample URLs or pretend chat audio is playable. - `playable_preview`: tell the user the caller-provided preview is ready in ChatCut and provide the clean editor link. This is display-only; collect the caller's branch decision separately after listening. Never emit raw audio HTML or expose the runtime media URL. - MG reference choices: map labels to exact reference ids and retain one localized `__other__` choice accepting a new preference or another batch, with no separate Show more option. Other returns to discovery, not style application. Return the selected reference id to the caller; never bind `apply_preset` directly to it. Resolve bundled images through `assets/widget-media/manifest.json` relative to the plugin root; absent previews use text, never guessed CDN paths. -- Avatar choices: map each returned name to the exact `identityId`. Use names and returned descriptions in text; previews are available in ChatCut's Digital Humans library. Preserve `create_avatar` as a separate value returned to the caller, then request the attachment and use `asset-import`; it does not open a native dialog automatically. +- Character choices: map each returned name to the exact `characterId` and use the visual-preview path. A new-character option is a separate value returned to the caller; then request the portrait attachment and use `asset-import`. - Voice/clone actions: preserve `clone_voice`, `Retry`, `Apply`, and any other caller-provided stable values. A selection is not consent to cloning or a paid action; request the caller's full explicit authorization separately when required. diff --git a/claude/skills/digital-human/SKILL.md b/claude/skills/digital-human/SKILL.md deleted file mode 100644 index 4513a20..0000000 --- a/claude/skills/digital-human/SKILL.md +++ /dev/null @@ -1,529 +0,0 @@ ---- -name: digital-human -description: Create and manage script- or audio-driven AI avatar videos from an official or saved presenter, an imported portrait, or a representative still prepared from video. Use for 数字人、数字分身、虚拟人、照片开口说话、人像口播、改稿不重拍, AI avatar, AI avatar video, avatar video, talking avatar, talking photo, photo avatar, video avatar, AI presenter, virtual presenter, virtual spokesperson, digital human, or a person's digital twin, including choosing or creating the avatar and deciding its voice, script, aspect ratio, and output quality. Do not use for a static profile-picture avatar, game or 3D character creation, an industrial digital twin, ordinary B-roll, generic image-to-video, video translation, talking-head editing, lip-sync dubbing alone, or TTS-only requests. -user-invocable: true ---- - -# Digital Human - -If `manage_custom_voice`, `manage_avatar`, or `submit_avatar_video` are not loaded yet, load them with `ToolSearch` before calling them, so you use their real parameters instead of guessing. - -Create a reusable presenter identity or generate a lip-synced presenter video -without exposing the underlying provider. Treat avatar selection, speech, script, -format, consent, generation, and verification as one stateful conversation. - -## When to Use - -Use this skill when the user wants to: - -- Make an official or saved AI avatar, talking avatar, or virtual presenter speak. -- Turn a consented portrait into a reusable digital-human identity. -- Use a representative still from an imported video as a photo avatar. -- Create a synthetic AI presenter from a description, when live capabilities allow it. -- Draft or revise the narration for a digital-human video. -- Regenerate, reorder, remove, retry, or accept segments from an avatar-video batch. - -Treat **AI avatar** as the broad default English concept. **Digital twin** means a -reusable likeness of a specific real person; **photo avatar** or **talking photo** -means a still-image source; **AI presenter**, **virtual presenter**, and **virtual -spokesperson** describe the presenter's role. **Digital human** is valid but is often -used for broader enterprise or interactive experiences. - -Route adjacent requests elsewhere: - -- Speech, voice audition, or voice cloning without avatar video: use the Voice Skill. -- Ordinary footage cleanup or presenter editing: use the Talking Head Guide. -- Generic image-to-video or video generation without a speaking avatar: use the - Video Generation Skill. -- Translation or dubbing of an existing video: use the dedicated translation or - dubbing workflow when available. - -## Required Inputs - -Resolve these slots before submission, but do not ask for them in a fixed order: - -1. **Target project** — use the active project when it is unambiguous. -2. **Avatar identity and ready revision** — an official avatar, a saved avatar, or - a newly created photo or synthetic avatar. -3. **Speech source** — one exact available voice, one cloned voice, or one imported - audio asset. A text-driven source also requires the final script. -4. **Final script** — preserve the user's wording and punctuation. Show any AI-written - draft in full and let the user edit or approve it before generation. -5. **Aspect ratio** — `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. Explicit user - choice wins; otherwise omit it so the backend uses `auto` and preserves the - selected avatar's original framing. Do not substitute the current project ratio. -6. **Resolution** — `1080p` by default; use `720p` when requested or when live - capabilities require it, and explain any fallback. -7. **Optional controls** — motion guidance only when the live capability response - says it is supported. For an official catalog avatar, use its returned - `hasBackground` value: preserve the original scene when true and request a - transparent background when false. For a saved/custom avatar, preserve its original - background by default. Request a transparent background only when the user explicitly - asks to remove the background or use transparency. Do not ask the user to choose a - background mode; briefly state the resolved default before submission. An explicit - user request to preserve or remove the background always wins. - -For avatar creation also resolve: - -- A non-empty name. Keep numeric validation limits out of the question label; - only explain a limit if the submitted name actually fails validation. -- One final avatar-creation source: an imported JPEG/PNG portrait, a - representative JPEG/PNG still prepared from an imported video, or a prompt - for a synthetic identity when supported. -- Explicit consent when a real person's likeness is involved. - -Do not pass a video directly to photo-avatar creation unless the live capability -contract explicitly supports it. For an imported video, use the host's prepared -representative still or help select a clear, unobstructed, front-facing frame, then -use that image asset as the source. - -When a user attaches or references a video containing a person and says "this -avatar", "this person", "use her/him", `这个形象`, `这个人`, `用她`, `用他`, or an -equivalent phrase, resolve the person in that exact attachment as the intended -likeness source. Do not treat the attachment as the requested speech source merely -because it has audio or a transcript. If the user also supplies a new topic or an -approved script, that new topic or script wins; do not read or reuse the video's -spoken content unless the user explicitly asks for it. - -The attached video is a prospective avatar source, not a ready `identityId`. Enter -the video-to-avatar creation flow: prepare a clean representative still, obtain the -required likeness consent, create the identity, and then reuse that identity for the -requested generation. Do not show official or saved avatar choices while this -prospective source remains valid. Offer other identities only when the user declines -to use the person in the video or no eligible face frame can be prepared. - -If "this" could reasonably mean the person's likeness, the video's existing pixels, -or its spoken content, ask exactly one targeted clarification before choosing a -workflow: "Do you want to use the person in this video as the avatar and have them -speak your new script?" Do not replace this with a generic "What do you want to do -with this video?" question. - -When the user is about to upload or pick a video for avatar creation, tell them up -front that the chosen frame must show the face clearly with nothing covering it — no -burned-in subtitles, captions, watermarks, stickers, or hands over the face — and -prefer a frame with a neutral, front-facing pose. Never upload an inspection contact -sheet or any preview that carries an overlaid time label as the avatar source; use a -clean frame with no added markings. - -A video source also carries the person's voice. After the avatar is created from a -video, offer to clone that voice from the same video so the avatar speaks with it: - -1. Run the Voice Skill's cloning flow (`manage_custom_voice action=clone` with the - video's asset and explicit voice-cloning consent — likeness consent alone does - not cover the voice). -2. When the cloned voice is ready, call - `manage_avatar action=set-default-voice identityId= voiceRef=` - so it becomes this avatar's saved default voice. -3. Later generations with this avatar should use the returned `defaultVoiceRef` by - default, while still letting the user pick another voice. - -This is an offer, not an automatic step: skip it when the user declines or the video -has unusable audio (music, multiple speakers, heavy noise). - -## Workflow - -This is an intent-driven constraint resolver, not a linear wizard. Do not walk the -user through every section or ask for fields in a preset order. - -On every turn: - -1. Extract all facts already supplied by the user, attachments, active project, - selected media, saved avatar state, and prior answers in this request. -2. Resolve deictic phrases such as "this", `这个`, `这个形象`, and `这个人` against - the attached or selected media and the user's action words before inferring a - generic media workflow. -3. Infer the user's immediate requested action: explore, choose, create, revise, - generate, inspect progress, retry, reorder, remove, accept, place, or export. -4. Compute only the blockers for that action. Defaults and live saved bindings count - as resolved unless they conflict with an explicit request. -5. If nothing blocks the action, execute it immediately. -6. Otherwise ask about the single most important missing or ambiguous constraint, - preferably with structured choices, then recompute from the new state. - -For example, a user who names a ready avatar, supplies approved text, and accepts its -saved voice should not be asked to choose them again. A user inspecting a running -batch does not need to resolve a new script or aspect ratio. Consent is requested only -for the creation or cloning action that needs it. - -### Load live capabilities and state when relevant - -Use `ToolSearch` to load `manage_avatar` and `submit_avatar_video` if they are not -already visible. Call `manage_avatar` for live capabilities before -offering creation or advanced options. - -If the user has neither selected an exact identity nor supplied a prospective -likeness source, list the current official and saved identities before recommending -one. Do not infer availability from chat history, examples, or provider knowledge. -For official avatars, use the live catalog's `tags` as factual selection filters and -its `description` as recommendation guidance. Match those fields against the user's -requested presenter and use case, and never infer a missing age, gender, ethnicity, -role, setting, or capability from the preview's appearance. -For the empty saved-avatar recommendation state, call the official catalog with -`limit: 5`; do not page through the catalog or turn a full catalog page into one -recommendation form. - -An avatar attached from ChatCut's Digital Humans library or native visual picker is -already an exact selection. Reuse its attached ChatCut `avatarId` as `identityId` -across later confirmation and retry turns. An official attached identity is directly -usable by `submit_avatar_video`; it does not need to be imported or rediscovered in -the first catalog page. Before resolving speech for an attached identity, call -`manage_avatar` with `action: get` and that exact ID unless a live result in the -current turn already supplies its `defaultVoiceRef`. Never reject or replace an -attached avatar just because it is absent from the five-item recommendation page. - -Only use identities and revisions reported as ready. Pending, failed, deleting, -deleted, or orphaned identities are not valid generation inputs. - -If the digital-human tools are unavailable, explain that this generation capability -is not connected. Do not silently substitute generic video generation or call the -backend HTTP routes directly. - -### Resolve the avatar when it is missing or invalid - -Respect an explicit user choice. Otherwise: - -- A person-containing video explicitly or provisionally resolved as the likeness - source: continue the video-to-avatar creation flow. Do not enter the generic avatar - picker and do not ask the user to choose among saved or official identities. -- No ready saved avatar: show at most five suitable official avatars and include the - reserved **create** card in the same choice surface. This is a strict empty-state - contract: never render a sixth official recommendation, and never omit the - creation option merely because official avatars are available. -- Ready saved avatars exist: show them first (up to five total cards), then the - reserved create card. -- Exactly one ready saved avatar: offer it first and ask whether to use it. -- Multiple ready saved avatars: prefer higher live usage count; when counts tie, prefer - the newest creation. If usage metadata is absent, rank by newest creation only; - never invent a frequency. - -Avatar identity selection is media-preview-only and is a strict exception to the generic -single-branch `` rule. Always use visual cards through the Widget Forms -Skill, including when exactly one ready saved avatar exists. Never ask a text-only -category question such as "use the saved avatar" versus "browse official avatars", -and never use ``, ``, or label-only identity cards for avatar -selection. For an avatar card in ChatCut's native widget — official or the user's own -saved avatar — write only its exact `identityId` as `avatar-id`; the host resolves the -current name, preview video or fallback image, aspect ratio, and submitted value from the live avatar -identity even when the tool payload does not expose a preview URL. Never copy `value`, -`name`, `media`, or `aspect-ratio` onto an `avatar-id` option. If the host cannot -resolve preview media, omit that identity rather than falling back to text. Prefer -the live preview video when available and use the preview image only as its poster -or fallback. When -the user has ready saved avatars, place their cards before official recommendations. -Ask one most important missing question at a time and do not repeat answered questions. - -The raw tags below are the embedded ChatCut protocol only. In a published Codex or -Claude Code plugin, the Widget Forms Skill's host adapter takes priority: never emit -raw ChatCut tags. Map each live identity to the host's visual-choice surface using -`identityId` as its stable value and `name` as its label. In Codex, supply -`previewVideoUrl` as `previewVideo` and `previewImageUrl` as `preview` so the image is -the video's poster and failure fallback. Keep the create action -as a separate `create_avatar` value; plugin hosts return that value to the workflow -instead of opening ChatCut's native dialog automatically. -For a saved identity whose `list` row has no preview URLs, call `manage_avatar -action=get` for that exact candidate before rendering it; use the selected ready -revision's returned preview media and do not invent or copy media from another avatar. - -One reserved label-only option value opens a native editor surface instead of -submitting an answer. Write it with no `media`: - -- `create_avatar` — opens the native creation dialog where the user uploads a photo - and creates a personal avatar. Keep its card label a short "create your own - avatar" phrase (e.g. 创建专属数字人); do not mention uploading a photo — the - dialog explains that itself. - -Clicking that card does not produce a form answer. When the user finishes in the -native dialog, their chosen or newly created avatar is sent back automatically as -this question's answer with the avatar attached; treat that reply as the avatar -selection and do not re-ask. - -The selection shape is therefore one `` containing up to five avatar -preview cards (saved first, then official) followed by the creation card. Localize -the visible question and card label to the conversation language: - -```text - - - - - - -``` - -Creating an identity is separate from generating a video: - -1. Resolve the source or prompt and name. -2. Obtain required consent. -3. Create the identity through `manage_avatar`. -4. Wait until it is ready or terminal. Where `track_progress` offers - `target=avatar`, call `action=wait` with the `identityIds` and do what its - result says; otherwise re-read it with `manage_avatar action=get refresh=true` - in a later turn. -5. Save it under **My avatars**. Do not auto-generate a video merely because avatar - creation succeeded. - -When asking for a portrait in an embedded ChatCut Widget, always mark the upload as an avatar source so -the host uses the same validation, JPEG normalization, private transport storage, and -upload-readiness behavior as the native Digital Humans dialog: - -```text - - - - -``` - -Do not use an unmarked generic `` upload for avatar creation. -Published plugin forms do not provide this native upload control. Ask the user to -attach the portrait through their host, then use the Asset Import Skill and pass the -returned ChatCut image asset id to `manage_avatar`. - -### Resolve speech only when the requested action needs it - -For an official avatar, never silently use its paired voice and never immediately -open the broader voice catalog. When the live result provides `defaultVoiceRef` and -the user has not already made this choice, first ask one blocking text-only decision -in the conversation language: whether to use this avatar's official built-in voice -or choose another voice. In embedded ChatCut, render that decision with localized -``, then stop and wait. -If the user accepts, resolve speech with that exact `defaultVoiceRef` and do not call -`manage_avatar action=voices`. Call `action=voices` only after the user chooses -another voice, or when the live identity has no `defaultVoiceRef`. Do not ask for -language or script and then pre-emptively call `action=voices` in the same turn. - -For a saved or newly created avatar, reuse its saved voice binding when present and -confirm it; otherwise offer available voices. When the user settles on a voice they -want this avatar to keep, save it with `manage_avatar action=set-default-voice`; -`list`/`get` then return it as the avatar's `defaultVoiceRef`. - -For voice discovery, audition, or cloning, follow the Voice Skill and reuse its live -voice list, consent gate, and generated voice asset. Do not duplicate voice-cloning -logic here. - -When an exact voice still needs to be selected, never present voice names as prose -beside a free-text field. Use the exact `voiceRef` returned by `manage_avatar` as the -submitted value. Voice selection is audio-preview-only and is a strict exception to -the generic `` rule: - -- Never render ``, ``, prose voice names, or label-only voice - options. A voice without a returned playable `previewUrl` must be omitted from the - recommendation surface rather than presented as an unplayable choice. -- When the user wants to audition voices, prefer up to five suitable available - official, preset, or custom voices that actually include `previewUrl`. Render them as one - required ``; `hasPreview: true` - without `previewUrl` is not playable and must not be presented as an audition - card. Always append one no-media `` action, localized to the conversation language, as the - final card. This clone option is required on every recommended-voice surface. -- An official avatar's paired default voice may still be used without listing the - broader official voice catalog. Do not show that default voice as an audition - card unless an actual audio `previewUrl` was returned for it. Never reuse the - avatar image or video URL as a voice preview. -- Copy each returned `previewUrl` exactly. In particular, keep - `/voice-samples/...` root-relative; do not expand it to `chatcut.com` or invent - another host. -- When script text is also being confirmed, put the voice selector and script field - in the same widget. Do not make the user type a voice name into the script field. - -```text - - - - - - - -``` - -If the source began as a user video, ask whether the user wants that video's voice. -If yes, follow the Voice Skill's explicit voice-cloning authorization flow before -using it. An attachment alone is not permission to clone. If no, continue with the -normal voice picker. - -An existing project audio asset may drive the avatar directly. When audio is the -speech source, do not invent or require text unless the user also wants a transcript -or script review. - -### Resolve a script only for text-driven generation - -The script may be pasted by the user, derived from selected project material, or -written collaboratively. When drafting: - -1. Ask only for missing intent such as audience, goal, tone, facts, and duration. -2. Produce the complete proposed script. -3. Let the user edit it or approve it explicitly. -4. Submit exactly the confirmed content. - -Each explicit text segment must contain 1–5000 characters. Omit explicit segments -for an ordinary script so the backend generates one continuous video up to 5000 -characters. Use explicit segments only when the user requests meaningful paragraph -or shot boundaries. Longer text is split automatically and may create at most 10 -segments. Never summarize, translate, drop, duplicate, or reorder content to fit a -limit. Preserve wording and -punctuation; non-semantic line whitespace may be normalized automatically. - -Text-driven avatar generation is limited by the 5000-character segment boundary, -not by an estimated audio duration. ChatCut TTS and cloned voices are rendered to -audio before avatar-video submission. Long audio-backed text is prepared as smaller -ordered segments, and each generated audio file is validated from its real duration. -Existing project audio longer than 600 seconds is automatically split into ordered -parts before submission. Pass the original asset once; never split it manually or -submit one generation call per part. If a progress update is useful, say only that -ChatCut is splitting the long audio while preserving the complete content and order. -Do not add implementation details about where or how the work runs, and never expose -hidden derived audio assets. Never shorten or summarize approved content to satisfy -these limits. - -### Resolve output settings from intent and available defaults - -Preserve the selected avatar's original framing by default: omit `aspectRatio` and -let the backend resolve it to `auto`. Never use the current project ratio as the -avatar-video default. If the user explicitly asks for a different ratio, pass that -request instead. - -For an official catalog avatar, resolve the default from the `hasBackground` field -returned by `manage_avatar action="catalog"`: - -- `hasBackground: true` — set `removeBackground: false` and preserve the original - scene. -- `hasBackground: false` — set `removeBackground: true` so the result is a - transparent WebM when transparency is supported. - -For a saved/custom avatar without catalog metadata, keep `removeBackground: false` as -the default and preserve the source image background. Set `removeBackground: true` -only when the user explicitly asks for a transparent background or background removal. -Do not turn this into a blocking question. Before submission, briefly tell the user -whether the original scene will be preserved or the result will be transparent. An -explicit user request overrides the catalog default. If transparency is unsupported, -explain the limitation and keep the original background. - -Do not promise crop, background removal, motion control, transparency, resolution, -or prompt-based identity creation until the live capability response confirms it. - -### Submit as soon as the required constraints are resolved - -Summarize the resolved avatar, voice or audio source, script state, aspect ratio, and -resolution before the first generation when any choice remains consequential. Then -call `submit_avatar_video` once. - -Treat long-script output as an ordered batch of avatar-video segments, not as one -audio file or one opaque job. After submitting, tell the user that generation has -started and end the turn. Never keep the turn open for it: no Bash sleep and no -repeated `track_progress` or `manage_avatar action=get-batch` reads. When a follow-up -step in this request needs the completed videos and `track_progress` describes -`action=wait` as registering an automatic follow-up, call it once with the jobIds and -a note, then end the turn. The project library shows the in-progress generation and -receives the completed video assets in the background. - -Call `track_progress` in a later user turn only when the user asks for status or resumes -a follow-up step that depends on the completed asset, such as placing it on the timeline -or reviewing the result. When a job is still non-terminal, say that generation is still -running without quoting its numeric progress: provider progress values are coarse -internal stages, not user-facing completion estimates. Use `manage_avatar -action=get-batch` when ordered segment and attempt state is needed. Keep the stable -segment key and original order. - -For a failed segment, offer a targeted retry. Do not resubmit successful segments or -the entire batch without the user's instruction. Use the management tool for supported -segment regeneration, removal, reordering, attempt selection, and acceptance. - -Generated results become normal ChatCut video assets. Add them to the timeline only -when the user requested placement or it is clearly part of the active editing task. -Do not export unless asked. - -## Provider Selection - -Provider selection is an internal ChatCut backend detail. Always call the -ChatCut-native tools and let the backend construct provider requests, resolve private -IDs, enforce capabilities, and persist results. Never call an underlying provider API -or CLI directly from this skill. - -Select only through ChatCut capabilities, catalog results, saved bindings, and the -native tools. User-facing language is limited to concepts such as: - -- **Official avatars** and **My avatars** -- **Official voices**, **cloned voices**, and **project audio** -- **Photo avatar** and **synthetic avatar** - -Provider names, engine or model names, provider-specific avatar or voice IDs, raw -provider URLs, request payloads, and provider error messages are internal. Never show -them in prose, option labels, progress updates, or failure messages. Translate failures -into actionable product language while preserving the real status. - -Do not route around ChatCut's feature gates, quotas, billing checks, safety policy, -catalog visibility, or provider registry. - -## Consent and Safety - -A real-person photo avatar requires an explicit affirmative confirmation immediately -before creation. Use the user's language and keep the meaning exact. For Chinese: - -> 我确认拥有该肖像,或已获得创建和使用该数字人形象的授权,并承诺不将其用于 -> 冒充他人、欺诈或其他违法用途。 - -For English: - -> I confirm that I own this likeness or have permission to create and use this -> digital avatar, and I will not use it for impersonation, fraud, or unlawful activity. - -En español: - -> Confirmo que soy titular de los derechos sobre esta imagen o que tengo permiso para crear y usar este avatar digital, y no lo utilizaré para suplantar identidades, cometer fraude ni realizar actividades ilícitas. - -Do not infer consent from an upload, a previous unrelated confirmation, ownership of -the project, or a third party's instruction. Record consent only through the native -tool's consent fields. Synthetic prompt-created identities do not require real-person -likeness consent unless the prompt or references identify a real person. - -Refuse non-consensual impersonation, deceptive identity use, fraud, harassment, -sexual exploitation, or attempts to evade safeguards. Do not create a real-person -identity from uncertain authorization. Voice cloning has a separate consent gate; -follow the Voice Skill even when avatar consent was already obtained. - -## Verification - -Before reporting success, verify through live tool results: - -1. The chosen identity revision was ready when submitted. -2. The job or batch reached a completed terminal state and returned output asset IDs. -3. Every expected segment key exists exactly once and appears in the confirmed order. -4. Text-backed segment manifests preserve the confirmed wording and punctuation, - allowing only documented non-semantic whitespace normalization. -5. The generated videos exist as usable project or media-library assets. -6. Any requested timeline placement is present and ordered correctly. - -Preview the generated assets or composed timeline when visual verification is available. -Do not claim the face, lip sync, framing, or background looks correct from status fields -alone. If visual inspection is unavailable, say what was structurally verified. - -## Hard Rules - -- Never expose provider, engine, or model identity to the user. -- Never call an underlying provider directly or construct provider payloads in the - conversation layer. -- Never invent an avatar, voice, capability, usage count, saved binding, or asset ID. -- Never generate from a non-ready identity revision. -- Never treat a video as a supported identity source without an explicit live capability; - prepare a representative still for the current photo-avatar contract. -- Never pass an external source URL to avatar creation; import the source into the - target ChatCut project first. -- Never infer likeness consent or voice-cloning consent from an attachment. -- Never clone a video's original voice without the Voice Skill's explicit authorization. -- Never silently rewrite, translate, truncate, reorder, or duplicate confirmed script text. -- Never ask for information already available from the project, attachment, or live tools. -- Never run a fixed wizard or ask for avatar, voice, script, ratio, and resolution in a - preset sequence; resolve only what the current user action still needs. -- Ask one most important missing question at a time; prefer structured visual choices. -- Never show provider-private IDs, URLs, payloads, or raw errors in user-visible output. -- Never bypass ChatCut tools, feature gates, billing, quota, or safety checks. -- Never retry a whole successful batch because one segment failed. -- Never claim visual quality without inspecting the generated result. -- Never place media on the timeline or export it unless the task calls for that action. -- For official catalog avatars, obey the returned `hasBackground` default: preserve a - configured scene background and remove a configured absent background. For - saved/custom avatars, preserve the source background unless the user explicitly asks - for transparency or background removal. Explicit user intent always overrides either - default. diff --git a/claude/skills/digital-human/SKILL.md b/claude/skills/digital-human/SKILL.md new file mode 120000 index 0000000..79588af --- /dev/null +++ b/claude/skills/digital-human/SKILL.md @@ -0,0 +1 @@ +../../../codex/skills/digital-human/SKILL.md \ No newline at end of file diff --git a/codex/skills/digital-human/SKILL.md b/codex/skills/digital-human/SKILL.md index 0a6750e..af2179c 100644 --- a/codex/skills/digital-human/SKILL.md +++ b/codex/skills/digital-human/SKILL.md @@ -1,351 +1,149 @@ --- name: digital-human -description: Create and manage script- or audio-driven AI avatar videos from an official or saved presenter, an imported portrait, or a representative still prepared from video. Use for 数字人、数字分身、虚拟人、照片开口说话、人像口播、改稿不重拍, AI avatar, AI avatar video, avatar video, talking avatar, talking photo, photo avatar, video avatar, AI presenter, virtual presenter, virtual spokesperson, digital human, or a person's digital twin, including choosing or creating the avatar and deciding its voice, script, aspect ratio, and output quality. Do not use for a static profile-picture avatar, game or 3D character creation, an industrial digital twin, ordinary B-roll, generic image-to-video, video translation, talking-head editing, lip-sync dubbing alone, or TTS-only requests. +description: | + Create script- or audio-driven talking-head videos with a user's saved character, + portrait, or a representative still from video. Use for 数字人、角色口播、数字分身、 + 照片开口说话、人像口播, AI avatar, talking photo, AI presenter, or digital twin. + Choose between direct-to-camera speech and short character performances by the + requested shot. Do not use for static avatars, game or 3D characters, video + translation, existing talking-head editing, lip-sync dubbing alone, or TTS alone. user-invocable: true --- # Digital Human -If `manage_custom_voice`, `manage_avatar`, or `submit_avatar_video` are not loaded yet, load them with `ToolSearch` before calling them, so you use their real parameters instead of guessing. - -Create a reusable presenter identity or generate a lip-synced presenter video -without exposing the underlying provider. Treat avatar selection, speech, script, -format, consent, generation, and verification as one stateful conversation. - -## When to Use - -Use this skill when the user wants to: - -- Make an official or saved AI avatar, talking avatar, or virtual presenter speak. -- Turn a consented portrait into a reusable digital-human identity. -- Use a representative still from an imported video as a photo avatar. -- Create a synthetic AI presenter from a description, when live capabilities allow it. -- Draft or revise the narration for a digital-human video. -- Regenerate, reorder, remove, retry, or accept segments from an avatar-video batch. - -Treat **AI avatar** as the broad default English concept. **Digital twin** means a -reusable likeness of a specific real person; **photo avatar** or **talking photo** -means a still-image source; **AI presenter**, **virtual presenter**, and **virtual -spokesperson** describe the presenter's role. **Digital human** is valid but is often -used for broader enterprise or interactive experiences. - -Route adjacent requests elsewhere: - -- Speech, voice audition, or voice cloning without avatar video: use the Voice Skill. -- Ordinary footage cleanup or presenter editing: use the Talking Head Guide. -- Generic image-to-video or video generation without a speaking avatar: use the - Video Generation Skill. -- Translation or dubbing of an existing video: use the dedicated translation or - dubbing workflow when available. - -## Required Inputs - -Resolve these slots before submission, but do not ask for them in a fixed order: - -1. **Target project** — use the active project when it is unambiguous. -2. **Avatar identity and ready revision** — an official avatar, a saved avatar, or - a newly created photo or synthetic avatar. -3. **Speech source** — one exact available voice, one cloned voice, or one imported - audio asset. A text-driven source also requires the final script. -4. **Final script** — preserve the user's wording and punctuation. Show any AI-written - draft in full and let the user edit or approve it before generation. -5. **Aspect ratio** — `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. Explicit user - choice wins; otherwise omit it so the backend uses `auto` and preserves the - selected avatar's original framing. Do not substitute the current project ratio. -6. **Resolution** — `1080p` by default; use `720p` when requested or when live - capabilities require it, and explain any fallback. -7. **Optional controls** — motion guidance only when the live capability response - says it is supported. For an official catalog avatar, use its returned - `hasBackground` value: preserve the original scene when true and request a - transparent background when false. For a saved/custom avatar, preserve its original - background by default. Request a transparent background only when the user explicitly - asks to remove the background or use transparency. Do not ask the user to choose a - background mode; briefly state the resolved default before submission. An explicit - user request to preserve or remove the background always wins. - -For avatar creation also resolve: - -- A non-empty name. Keep numeric validation limits out of the question label; - only explain a limit if the submitted name actually fails validation. -- One final avatar-creation source: an imported JPEG/PNG portrait, a - representative JPEG/PNG still prepared from an imported video, or a prompt - for a synthetic identity when supported. -- Explicit consent when a real person's likeness is involved. - -Do not pass a video directly to photo-avatar creation unless the live capability -contract explicitly supports it. For an imported video, use the host's prepared -representative still or help select a clear, unobstructed, front-facing frame, then -use that image asset as the source. - -When a user attaches or references a video containing a person and says "this -avatar", "this person", "use her/him", `这个形象`, `这个人`, `用她`, `用他`, or an -equivalent phrase, resolve the person in that exact attachment as the intended -likeness source. Do not treat the attachment as the requested speech source merely -because it has audio or a transcript. If the user also supplies a new topic or an -approved script, that new topic or script wins; do not read or reuse the video's -spoken content unless the user explicitly asks for it. - -The attached video is a prospective avatar source, not a ready `identityId`. Enter -the video-to-avatar creation flow: prepare a clean representative still, obtain the -required likeness consent, create the identity, and then reuse that identity for the -requested generation. Do not show official or saved avatar choices while this -prospective source remains valid. Offer other identities only when the user declines -to use the person in the video or no eligible face frame can be prepared. - -If "this" could reasonably mean the person's likeness, the video's existing pixels, -or its spoken content, ask exactly one targeted clarification before choosing a -workflow: "Do you want to use the person in this video as the avatar and have them -speak your new script?" Do not replace this with a generic "What do you want to do -with this video?" question. - -When the user is about to upload or pick a video for avatar creation, tell them up -front that the chosen frame must show the face clearly with nothing covering it — no -burned-in subtitles, captions, watermarks, stickers, or hands over the face — and -prefer a frame with a neutral, front-facing pose. Never upload an inspection contact -sheet or any preview that carries an overlaid time label as the avatar source; use a -clean frame with no added markings. - -A video source also carries the person's voice. After the avatar is created from a -video, offer to clone that voice from the same video so the avatar speaks with it: - -1. Run the Voice Skill's cloning flow (`manage_custom_voice action=clone` with the - video's asset and explicit voice-cloning consent — likeness consent alone does - not cover the voice). -2. When the cloned voice is ready, call - `manage_avatar action=set-default-voice identityId= voiceRef=` - so it becomes this avatar's saved default voice. -3. Later generations with this avatar should use the returned `defaultVoiceRef` by - default, while still letting the user pick another voice. - -This is an offer, not an automatic step: skip it when the user declines or the video -has unusable audio (music, multiple speakers, heavy noise). - -## Workflow - -This is an intent-driven constraint resolver, not a linear wizard. Do not walk the -user through every section or ask for fields in a preset order. - -On every turn: - -1. Extract all facts already supplied by the user, attachments, active project, - selected media, saved avatar state, and prior answers in this request. -2. Resolve deictic phrases such as "this", `这个`, `这个形象`, and `这个人` against - the attached or selected media and the user's action words before inferring a - generic media workflow. -3. Infer the user's immediate requested action: explore, choose, create, revise, - generate, inspect progress, retry, reorder, remove, accept, place, or export. -4. Compute only the blockers for that action. Defaults and live saved bindings count - as resolved unless they conflict with an explicit request. -5. If nothing blocks the action, execute it immediately. -6. Otherwise ask about the single most important missing or ambiguous constraint, - preferably with structured choices, then recompute from the new state. - -For example, a user who names a ready avatar, supplies approved text, and accepts its -saved voice should not be asked to choose them again. A user inspecting a running -batch does not need to resolve a new script or aspect ratio. Consent is requested only -for the creation or cloning action that needs it. - -### Load live capabilities and state when relevant - -Use `ToolSearch` to load `manage_avatar` and `submit_avatar_video` if they are not -already visible. Call `manage_avatar` for live capabilities before -offering creation or advanced options. - -If the user has neither selected an exact identity nor supplied a prospective -likeness source, list the current official and saved identities before recommending -one. Do not infer availability from chat history, examples, or provider knowledge. -For official avatars, use the live catalog's `tags` as factual selection filters and -its `description` as recommendation guidance. Match those fields against the user's -requested presenter and use case, and never infer a missing age, gender, ethnicity, -role, setting, or capability from the preview's appearance. -For the empty saved-avatar recommendation state, call the official catalog with -`limit: 5`; do not page through the catalog or turn a full catalog page into one -recommendation form. - -An avatar attached from ChatCut's Digital Humans library or native visual picker is -already an exact selection. Reuse its attached ChatCut `avatarId` as `identityId` -across later confirmation and retry turns. An official attached identity is directly -usable by `submit_avatar_video`; it does not need to be imported or rediscovered in -the first catalog page. Before resolving speech for an attached identity, call -`manage_avatar` with `action: get` and that exact ID unless a live result in the -current turn already supplies its `defaultVoiceRef`. Never reject or replace an -attached avatar just because it is absent from the five-item recommendation page. - -Only use identities and revisions reported as ready. Pending, failed, deleting, -deleted, or orphaned identities are not valid generation inputs. - -If the digital-human tools are unavailable, explain that this generation capability -is not connected. Do not silently substitute generic video generation or call the -backend HTTP routes directly. - -### Resolve the avatar when it is missing or invalid - -Respect an explicit user choice. Otherwise: - -- A person-containing video explicitly or provisionally resolved as the likeness - source: continue the video-to-avatar creation flow. Do not enter the generic avatar - picker and do not ask the user to choose among saved or official identities. -- No ready saved avatar: show at most five suitable official avatars and include the - reserved **create** card in the same choice surface. This is a strict empty-state - contract: never render a sixth official recommendation, and never omit the - creation option merely because official avatars are available. -- Ready saved avatars exist: show them first (up to five total cards), then the - reserved create card. -- Exactly one ready saved avatar: offer it first and ask whether to use it. -- Multiple ready saved avatars: prefer higher live usage count; when counts tie, prefer - the newest creation. If usage metadata is absent, rank by newest creation only; - never invent a frequency. - -Avatar identity selection is media-preview-only and is a strict exception to the generic -single-branch `` rule. Always use visual cards through the Widget Forms -Skill, including when exactly one ready saved avatar exists. Never ask a text-only -category question such as "use the saved avatar" versus "browse official avatars", -and never use ``, ``, or label-only identity cards for avatar -selection. For an avatar card in ChatCut's native widget — official or the user's own -saved avatar — write only its exact `identityId` as `avatar-id`; the host resolves the -current name, preview video or fallback image, aspect ratio, and submitted value from the live avatar -identity even when the tool payload does not expose a preview URL. Never copy `value`, -`name`, `media`, or `aspect-ratio` onto an `avatar-id` option. If the host cannot -resolve preview media, omit that identity rather than falling back to text. Prefer -the live preview video when available and use the preview image only as its poster -or fallback. When -the user has ready saved avatars, place their cards before official recommendations. -Ask one most important missing question at a time and do not repeat answered questions. - -The raw tags below are the embedded ChatCut protocol only. In a published Codex or -Claude Code plugin, the Widget Forms Skill's host adapter takes priority: never emit -raw ChatCut tags. Map each live identity to the host's visual-choice surface using -`identityId` as its stable value and `name` as its label. In Codex, pass -`previewVideoUrl` as `previewVideo` and `previewImageUrl` as `preview` so the image is -the video's poster and failure fallback. Keep the create action -as a separate `create_avatar` value; plugin hosts return that value to the workflow -instead of opening ChatCut's native dialog automatically. -For a saved identity whose `list` row has no preview URLs, call `manage_avatar -action=get` for that exact candidate before rendering it; use the selected ready -revision's returned preview media and do not invent or copy media from another avatar. - -One reserved label-only option value opens a native editor surface instead of -submitting an answer. Write it with no `media`: - -- `create_avatar` — opens the native creation dialog where the user uploads a photo - and creates a personal avatar. Keep its card label a short "create your own - avatar" phrase (e.g. 创建专属数字人); do not mention uploading a photo — the - dialog explains that itself. - -Clicking that card does not produce a form answer. When the user finishes in the -native dialog, their chosen or newly created avatar is sent back automatically as -this question's answer with the avatar attached; treat that reply as the avatar -selection and do not re-ask. - -The selection shape is therefore one `` containing up to five avatar -preview cards (saved first, then official) followed by the creation card. Localize -the visible question and card label to the conversation language: - -```text - - - - - - -``` - -Creating an identity is separate from generating a video: - -1. Resolve the source or prompt and name. -2. Obtain required consent. -3. Create the identity through `manage_avatar`. -4. Wait until it is ready or terminal. Where `track_progress` offers - `target=avatar`, call `action=wait` with the `identityIds` and do what its - result says; otherwise re-read it with `manage_avatar action=get refresh=true` - in a later turn. -5. Save it under **My avatars**. Do not auto-generate a video merely because avatar - creation succeeded. - -When asking for a portrait in an embedded ChatCut Widget, always mark the upload as an avatar source so -the host uses the same validation, JPEG normalization, private transport storage, and -upload-readiness behavior as the native Digital Humans dialog: - -```text - - - - -``` - -Do not use an unmarked generic `` upload for avatar creation. -Published plugin forms do not provide this native upload control. Ask the user to -attach the portrait through their host, then use the Asset Import Skill and pass the -returned ChatCut image asset id to `manage_avatar`. - -### Resolve speech only when the requested action needs it - -For an official avatar, never silently use its paired voice and never immediately -open the broader voice catalog. When the live result provides `defaultVoiceRef` and -the user has not already made this choice, first ask one blocking text-only decision -in the conversation language: whether to use this avatar's official built-in voice -or choose another voice. In embedded ChatCut, render that decision with localized -``, then stop and wait. -If the user accepts, resolve speech with that exact `defaultVoiceRef` and do not call -`manage_avatar action=voices`. Call `action=voices` only after the user chooses -another voice, or when the live identity has no `defaultVoiceRef`. Do not ask for -language or script and then pre-emptively call `action=voices` in the same turn. - -For a saved or newly created avatar, reuse its saved voice binding when present and -confirm it; otherwise offer available voices. When the user settles on a voice they -want this avatar to keep, save it with `manage_avatar action=set-default-voice`; -`list`/`get` then return it as the avatar's `defaultVoiceRef`. - -For voice discovery, audition, or cloning, follow the Voice Skill and reuse its live -voice list, consent gate, and generated voice asset. Do not duplicate voice-cloning -logic here. - -When an exact voice still needs to be selected, never present voice names as prose -beside a free-text field. Use the exact `voiceRef` returned by `manage_avatar` as the -submitted value. Voice selection is audio-preview-only and is a strict exception to -the generic `` rule: - -- Never render ``, ``, prose voice names, or label-only voice - options. A voice without a returned playable `previewUrl` must be omitted from the - recommendation surface rather than presented as an unplayable choice. -- When the user wants to audition voices, prefer up to five suitable available - official, preset, or custom voices that actually include `previewUrl`. Render them as one - required ``; `hasPreview: true` - without `previewUrl` is not playable and must not be presented as an audition - card. Always append one no-media `` action, localized to the conversation language, as the - final card. This clone option is required on every recommended-voice surface. -- An official avatar's paired default voice may still be used without listing the - broader official voice catalog. Do not show that default voice as an audition - card unless an actual audio `previewUrl` was returned for it. Never reuse the - avatar image or video URL as a voice preview. -- Copy each returned `previewUrl` exactly. In particular, keep - `/voice-samples/...` root-relative; do not expand it to `chatcut.com` or invent - another host. -- When script text is also being confirmed, put the voice selector and script field - in the same widget. Do not make the user type a voice name into the script field. - -```text - - - - - - - -``` - -If the source began as a user video, ask whether the user wants that video's voice. -If yes, follow the Voice Skill's explicit voice-cloning authorization flow before -using it. An attachment alone is not permission to clone. If no, continue with the -normal voice picker. - -An existing project audio asset may drive the avatar directly. When audio is the -speech source, do not invent or require text unless the user also wants a transcript -or script review. +If `manage_character`, `manage_voice`, or `submit_avatar_video` are not loaded yet, load them with `ToolSearch` before calling them, so you use their real parameters instead of guessing. + +A character is a person saved in the user's My Assets (我的资产) with images, a description, +and an optional voice. Choose how to use that character from the requested shot. +Custom talking-head generation requires a paid plan. If the tool returns a paywall, +explain that requirement accurately; do not substitute another workflow to bypass it. + +## Choose the performance + +- **Direct-to-camera speech, any length:** use `submit_avatar_video` with + `characterId`. The first use may require preparing the talking-head appearance. +- **Actions, changing scenes, multiple shots, or just one or two lines:** follow + `video-gen` and use `submit_video`. Put the character's image URLs in `refImages`, + main image first, and include the character description in the prompt to keep + the person consistent. Each generated clip is short, typically about 5–15 seconds. + +This choice belongs to the shot, not to the character. One video may use both: +product explanations can be talking-head segments while product scenes use video +generation. If the intent is unclear, ask one question explaining the difference, +for example: + +> 你想让角色贴着镜头念稿,还是演动作、换场景?口播长度不限,第一次需要准备形象; +> 视频生成能演动作和场景,但每段很短,约 5–15 秒。 + +Talking-head generation creates a new performance; it does not preserve an original +video's shots, actions, or timing. For original-content translation use +`video-translation`; for new dialogue over existing footage use `voice` and optional +`lipsync`. Existing footage cleanup belongs to `talking-head-guide`. + +## Resolve the character + +Reuse the character the user names or attached in ChatCut. Otherwise read the +`` section a ChatCut tool result carries, or call `manage_character action="list"`. +Use the exact `characterId`; call `manage_character action="get"` for its current +images, image URLs, description, and voice. Its `presenter` describes talking-head +preparation state; the submission tool handles that state. + +When several characters could fit, ask the user to choose. Widget Forms can show +image choices using each candidate's main image URL from `manage_character get`, +or ask a concise text question. Keep each ID, name, and image from the same returned +character. An already attached character does not need another selection step. +If there are no saved characters, offer to create one from the user's portrait. + +Use `manage_character` for `list`, `get`, `create`, `update`, and `delete`: + +- Create with `name`, optional `description`, `addImages`, and optional `voice`. +- Each `addImages` entry is `{source, label?}`. `source` is a project asset reference + or an HTTPS URL. The first image is the main image: a clear, front-facing JPG/PNG + showing one person. +- Update the selected `characterId` to change its `name` or `description`, add images + with `addImages`, remove them with `removeImageIds`, or set `mainImageId`. +- Save a voice as `voice: {id, name, provider}` using the selected voice's real + values; `provider` is `custom`, `doubao`, or `elevenlabs`. Set `voice: null` to clear + it. Keep provider values internal. +- Use `delete` only when the user requests deleting that character. + +For a supplied photo and a request to make it speak, create the character with its +name and main image, optionally save its selected voice, then submit with the returned +`characterId`. Creating the library entry alone does not request video generation. +If an existing character has no main image, ask for a clear front-facing portrait +and save it with `manage_character action="update"` and `addImages` before submission. +Request the file as Widget Forms describes for this host, import it as a project +asset, and save that asset reference. + +### A person in an attached video + +When the user says “this person”, `这个形象`, `用她`, or `用他`, resolve that phrase +against the attachment and the requested action. A video is a prospective likeness +source, not a saved character. Prepare a clean representative JPG/PNG still and use +it in `addImages`. The face must be clear, front-facing, and unobstructed by captions, +watermarks, stickers, hands, or inspection time labels. Do not use a contact sheet. + +If the user supplies a new topic or approved script, use that content. The video's +audio or transcript is not automatically the requested speech. If the intended use +is unclear, ask whether they want this person to speak the new script. + +Offer to clone the video's voice when useful, following the Voice Skill and its +separate explicit voice-cloning consent. Skip this when declined or when the source +has unusable audio. Once ready, the selected clone can be saved with +`manage_character action="update"` and `voice`; likeness consent does not authorize +voice cloning. + +## Resolve only what the next action needs + +Reuse the active project, attachments, prior answers, and current character state. +Ask only about the most important remaining blocker; do not run a fixed wizard. +Progress checks do not require resolving another script or output format. + +Before talking-head submission resolve: + +1. **Character:** the user's chosen `characterId` and a usable main image. +2. **Speech:** final text with the character's voice or an explicit `voiceRef`, or + a project audio asset containing the exact desired speech. +3. **Script approval:** preserve user-supplied wording and punctuation. Show an + AI-written draft in full for editing or approval before generation. +4. **Framing:** omit `aspectRatio` to preserve the character image's framing with + `auto`. Explicit choices may use `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. + Do not substitute the project ratio. +5. **Quality:** default to `1080p`; use `720p` when requested or required by the + tool, explaining any fallback. + +Omit `removeBackground` unless the user explicitly wants to keep or remove it; +the default preserves the source image background. Briefly state that default +without making it another question. Use `motionPrompt` for requested motion guidance +within the tool's supported behavior. Do not promise unsupported crop, transparency, +resolution, or motion control; report tool limitations plainly. + +### Speech and voice auditions + +Use `speech: {type:"voice", text}` to use the character's saved voice automatically. +Omitting `voiceRef` uses that binding; do not ask the user to confirm it again. +An explicit voice change overrides it. If there is no saved voice, let the user +choose through `manage_voice action="list"` and save the result with +`manage_character action="update"` and `voice`, or pass the selected `voiceRef` +in `speech` for this generation only. Keep the saved voice object and the returned +submission `voiceRef` distinct; do not invent one from a display name. + +For discovery, audition, and cloning, follow the Voice Skill and use voices available +for avatar video. Keep each voice's returned name, reference, and preview together. +Show up to five suitable playable +voices through Widget Forms, with a localized clone-voice action. A `hasPreview` +flag without a playable `previewUrl` is insufficient. Copy URLs exactly and never +use a character's image or video as an audio preview. + +Keep each voice's playable preview with its choice; Widget Forms decides how this +host lets the user audition it. When the script +also needs confirmation, include the voice selector and script field in one widget. +An upload or choice is not voice-cloning authorization. + +An existing audio asset drives speech with `speech: {type:"audio-asset", assetId}`. +It is the final spoken content, not a cloning sample. Do not require text unless +the user also wants transcription or script review. ### Resolve a script only for text-driven generation @@ -360,8 +158,8 @@ written collaboratively. When drafting: Each explicit text segment must contain 1–5000 characters. Omit explicit segments for an ordinary script so the backend generates one continuous video up to 5000 characters. Use explicit segments only when the user requests meaningful paragraph -or shot boundaries. Longer text is split automatically and may create at most 10 -segments. Never summarize, translate, drop, duplicate, or reorder content to fit a +or shot boundaries. Longer text is split automatically into ordered +segments; the tool rejects a script that exceeds its segment limit. Never summarize, translate, drop, duplicate, or reorder content to fit a limit. Preserve wording and punctuation; non-semantic line whitespace may be normalized automatically. @@ -377,89 +175,55 @@ Do not add implementation details about where or how the work runs, and never ex hidden derived audio assets. Never shorten or summarize approved content to satisfy these limits. -### Resolve output settings from intent and available defaults - -Preserve the selected avatar's original framing by default: omit `aspectRatio` and -let the backend resolve it to `auto`. Never use the current project ratio as the -avatar-video default. If the user explicitly asks for a different ratio, pass that -request instead. - -For an official catalog avatar, resolve the default from the `hasBackground` field -returned by `manage_avatar action="catalog"`: - -- `hasBackground: true` — pass `removeBackground: false` and preserve the original - scene. -- `hasBackground: false` — pass `removeBackground: true` so the result is a - transparent WebM when transparency is supported. - -For a saved/custom avatar without catalog metadata, keep `removeBackground: false` as -the default and preserve the source image background. Pass `removeBackground: true` -only when the user explicitly asks for a transparent background or background removal. -Do not turn this into a blocking question. Before submission, briefly tell the user -whether the original scene will be preserved or the result will be transparent. An -explicit user request overrides the catalog default. If transparency is unsupported, -explain the limitation and keep the original background. - -Do not promise crop, background removal, motion control, transparency, resolution, -or prompt-based identity creation until the live capability response confirms it. - -### Submit as soon as the required constraints are resolved - -Summarize the resolved avatar, voice or audio source, script state, aspect ratio, and -resolution before the first generation when any choice remains consequential. Then -call `submit_avatar_video` once. - -Treat long-script output as an ordered batch of avatar-video segments, not as one -audio file or one opaque job. After submitting, tell the user that generation has -started and end the turn. Never keep the turn open for it: no Bash sleep and no -repeated `track_progress` or `manage_avatar action=get-batch` reads. When a follow-up -step in this request needs the completed videos and `track_progress` describes -`action=wait` as registering an automatic follow-up, call it once with the jobIds and -a note, then end the turn. The project library shows the in-progress generation and -receives the completed video assets in the background. - -Call `track_progress` in a later user turn only when the user asks for status or resumes -a follow-up step that depends on the completed asset, such as placing it on the timeline -or reviewing the result. When a job is still non-terminal, say that generation is still -running without quoting its numeric progress: provider progress values are coarse -internal stages, not user-facing completion estimates. Use `manage_avatar -action=get-batch` when ordered segment and attempt state is needed. Keep the stable -segment key and original order. - -For a failed segment, offer a targeted retry. Do not resubmit successful segments or -the entire batch without the user's instruction. Use the management tool for supported -segment regeneration, removal, reordering, attempt selection, and acceptance. - -Generated results become normal ChatCut video assets. Add them to the timeline only -when the user requested placement or it is clearly part of the active editing task. -Do not export unless asked. - -## Provider Selection - -Provider selection is an internal ChatCut backend detail. Always call the -ChatCut-native tools and let the backend construct provider requests, resolve private -IDs, enforce capabilities, and persist results. Never call an underlying provider API -or CLI directly from this skill. - -Select only through ChatCut capabilities, catalog results, saved bindings, and the -native tools. User-facing language is limited to concepts such as: - -- **Official avatars** and **My avatars** -- **Official voices**, **cloned voices**, and **project audio** -- **Photo avatar** and **synthetic avatar** - -Provider names, engine or model names, provider-specific avatar or voice IDs, raw -provider URLs, request payloads, and provider error messages are internal. Never show -them in prose, option labels, progress updates, or failure messages. Translate failures -into actionable product language while preserving the real status. - -Do not route around ChatCut's feature gates, quotas, billing checks, safety policy, -catalog visibility, or provider registry. +### Submit and prepare the appearance + +Summarize consequential resolved choices, then call `submit_avatar_video` with +`characterId` and `speech`. Use `segments`, `aspectRatio`, `resolution`, +`removeBackground`, and `motionPrompt` only as needed. The tool shows the live credit +estimate in a confirmation card before anything is charged; do not calculate a separate +price. + +On a character's first talking-head use, the tool may return that the appearance +needs preparation, the credit cost, and a request for authorization without creating +a video. Tell the user that returned cost and ask them to explicitly confirm their +right to use this person's likeness using the consent wording below. After the user +agrees, repeat the same submission parameters with `confirmedConsent: true`. +Do not set it based on a photo upload or character selection. + +Preparation takes a few minutes. While it runs the tool says so and tells you how to +continue: tell the user, register the automatic follow-up it names with `track_progress`, +and end the turn. When the follow-up arrives, repeat the same submission parameters +without `confirmedConsent`. Do not report preparation as a completed video. +A paywall means custom talking-head generation requires a paid plan; report it +without claiming that preparation or generation succeeded. + +After submission returns generation `jobIds`, call `track_progress` with +`target="generation"`, `action="wait"`, and those `jobIds`, unless the user only +asked to queue work. Follow its wait contract: if it registers an automatic follow-up, +end the turn normally; otherwise await its result. Do not add sleeps or busy polling. +Use `action="status"` for a later progress question. Do not quote coarse numeric +provider progress as a completion estimate. + +Keep long-script segments in their original order. To redo a segment, call +`submit_avatar_video` again for that segment with the same character and the intended +speech/settings; do not regenerate successful segments without a request. Arrange, +remove, or replace generated clips on the timeline as the editing task requires. +Keep the old result until the user accepts a replacement. Generated clips are normal +video assets; place or export them only when the task calls for it. + +### Completed videos that do not meet the likeness request + +A completed avatar video can still fail the user's likeness requirements. When they ask to correct lips, cheeks or other facial proportions, check which changes the actual tools support before offering another generation. Do not describe the same character, photo and audio as a stricter facial correction unless a supported change plausibly addresses that specific correction; describe what will change and its limits. Explain any limitation plainly. A retry may cost credits and may not improve the likeness; retain the existing result until the user accepts a replacement. Honor an explicit, informed retry request, but do not keep proposing paid repeats of unchanged inputs after they have failed. If the user asks to stop, not export, or seek a credit review, follow that request and use the existing support route without requiring another generation or an export. + +When a generation tool returns `FEATURE_NOT_INCLUDED` for a `generation.*` feature, use its payer and access details. If the project owner needs Pro, explain that the owner pays and must upgrade; the member's own Pro does not unlock the project. Do not give the member a personal upgrade link or send them to their own billing menu. They can find the owner in the project's existing members view when available. Otherwise share the returned upgrade link, or ChatCut’s avatar menu → Upgrade. Existing credits are preserved and generation still consumes credits after upgrading. This rejection does not charge for generation; existing agent conversation fees are separate, so do not claim the whole conversation was free. Preserve the exact original tool input and project, then stop. Do not automatically search for or substitute stock media; offer alternatives only if the user explicitly asks. + +When the user returns with “continue”, “继续”, “jixu”, or says they upgraded, recheck current payer eligibility and credits for the original input with `estimate_generation` when supported, otherwise the original tool’s precheck. If there is no separate precheck, one retry of the original tool lets its server check current access. If eligible, submit the original input once; if still blocked, explain the current blocker and stop. Do not ask the user to choose between the original generation and stock media again or state the old rejection as current status without checking. If a job was already accepted, track it instead of submitting again. Never switch generation tools, providers or services to bypass a gate. ## Consent and Safety -A real-person photo avatar requires an explicit affirmative confirmation immediately -before creation. Use the user's language and keep the meaning exact. For Chinese: +Before preparing a real person's talking-head appearance, obtain explicit affirmative +consent after disclosing the tool's returned preparation cost. Use the user's language +and preserve the full meaning. For Chinese: > 我确认拥有该肖像,或已获得创建和使用该数字人形象的授权,并承诺不将其用于 > 冒充他人、欺诈或其他违法用途。 @@ -473,57 +237,24 @@ En español: > Confirmo que soy titular de los derechos sobre esta imagen o que tengo permiso para crear y usar este avatar digital, y no lo utilizaré para suplantar identidades, cometer fraude ni realizar actividades ilícitas. -Do not infer consent from an upload, a previous unrelated confirmation, ownership of -the project, or a third party's instruction. Record consent only through the native -tool's consent fields. Synthetic prompt-created identities do not require real-person -likeness consent unless the prompt or references identify a real person. +Do not infer consent from an upload, a prior unrelated confirmation, project +ownership, or a third party's instruction. Pass `confirmedConsent: true` only after +the user's explicit agreement for this preparation. Follow the Voice Skill's separate +consent gate for voice cloning. Refuse non-consensual impersonation, deceptive identity use, fraud, harassment, -sexual exploitation, or attempts to evade safeguards. Do not create a real-person -identity from uncertain authorization. Voice cloning has a separate consent gate; -follow the Voice Skill even when avatar consent was already obtained. +sexual exploitation, or attempts to evade safeguards. Use ChatCut-native tools; +let them enforce access, billing, and preparation state. Do not call provider APIs +directly. Keep provider names, private IDs, URLs, payloads, and raw provider errors +out of user-facing copy; explain the actual failure and next step in product terms. ## Verification -Before reporting success, verify through live tool results: - -1. The chosen identity revision was ready when submitted. -2. The job or batch reached a completed terminal state and returned output asset IDs. -3. Every expected segment key exists exactly once and appears in the confirmed order. -4. Text-backed segment manifests preserve the confirmed wording and punctuation, - allowing only documented non-semantic whitespace normalization. -5. The generated videos exist as usable project or media-library assets. -6. Any requested timeline placement is present and ordered correctly. - -Preview the generated assets or composed timeline when visual verification is available. -Do not claim the face, lip sync, framing, or background looks correct from status fields -alone. If visual inspection is unavailable, say what was structurally verified. - -## Hard Rules - -- Never expose provider, engine, or model identity to the user. -- Never call an underlying provider directly or construct provider payloads in the - conversation layer. -- Never invent an avatar, voice, capability, usage count, saved binding, or asset ID. -- Never generate from a non-ready identity revision. -- Never treat a video as a supported identity source without an explicit live capability; - prepare a representative still for the current photo-avatar contract. -- Never pass an external source URL to avatar creation; import the source into the - target ChatCut project first. -- Never infer likeness consent or voice-cloning consent from an attachment. -- Never clone a video's original voice without the Voice Skill's explicit authorization. -- Never silently rewrite, translate, truncate, reorder, or duplicate confirmed script text. -- Never ask for information already available from the project, attachment, or live tools. -- Never run a fixed wizard or ask for avatar, voice, script, ratio, and resolution in a - preset sequence; resolve only what the current user action still needs. -- Ask one most important missing question at a time; prefer structured visual choices. -- Never show provider-private IDs, URLs, payloads, or raw errors in user-visible output. -- Never bypass ChatCut tools, feature gates, billing, quota, or safety checks. -- Never retry a whole successful batch because one segment failed. -- Never claim visual quality without inspecting the generated result. -- Never place media on the timeline or export it unless the task calls for that action. -- For official catalog avatars, obey the returned `hasBackground` default: preserve a - configured scene background and remove a configured absent background. For - saved/custom avatars, preserve the source background unless the user explicitly asks - for transparency or background removal. Explicit user intent always overrides either - default. +Before reporting success, verify from live results that generation completed and +returned usable video assets, every expected segment is present once in the confirmed +order, and the text preserves approved wording and punctuation. Verify any requested +timeline placement. Preparation or queued jobs alone are not success. + +Preview the generated assets or composed timeline when visual inspection is available. +Do not claim that likeness, lip sync, framing, or background looks correct from status +fields alone. State what was structurally verified if visual inspection is unavailable. diff --git a/codex/skills/widget-forms/SKILL.md b/codex/skills/widget-forms/SKILL.md index ba00e8f..7b73c8f 100644 --- a/codex/skills/widget-forms/SKILL.md +++ b/codex/skills/widget-forms/SKILL.md @@ -66,13 +66,11 @@ and localizing the labels: The Other answer returns to discovery, without applying a style or starting production. Keep the image-option IDs unchanged in the next step. -Map an AI-avatar identity choice to one `single` field with `variant:"visual"`. -Use the live `identityId` as the option id, the returned `name` as its label, -`previewVideoUrl` as `previewVideo`, and `previewImageUrl` as `preview`. The -image remains the poster and automatic fallback if video loading fails. Keep -`create_avatar` as a label-only option. Selecting it returns that value to the -workflow; it does not open ChatCut's native avatar dialog, so request the source -attachment separately and use `asset-import`. +Map a character choice to one `single` field with `variant:"visual"`. Use the +`characterId` as the option id, the returned `name` as its label, and the +character's main image URL from `manage_character get` as `preview`. A +new-character option is label-only; selecting it returns to the workflow, so +request the portrait attachment separately and use `asset-import`. Do not call `ask_followup_questions` solely to display a `playable_preview`, because a passive preview is not a question. Use Codex's supported audio @@ -103,13 +101,12 @@ key. Runtime media URLs are not in the bundled manifest: do not embed them in Elicitation HTML. Present them through the host's normal safe link/audio surface, then use a label-only confirmation control. -AI-avatar previews are runtime catalog media and are not bundled in the plugin -manifest. Keep each authoritative `identityId` mapped to its returned `name`, -use label-only selectable cards in Elicitation, and tell the user that live -video previews are available in ChatCut's AI Avatars library. Never embed or -print the provider preview URL. A `create_avatar` selection returns to the -workflow and then follows the normal conversation-attachment plus -`asset-import` path. +Character images are runtime media and are not in the bundled manifest. Keep +each `characterId` mapped to its returned `name`, use label-only selectable +cards in Elicitation, and tell the user they can see each character's photos in +ChatCut's My Assets. Never embed or print the image URL. A new-character +selection returns to the workflow and then follows the normal +conversation-attachment plus `asset-import` path. For MG reference choices, look up the exact reference ID in the same manifest (`entries[referenceId]`, for example a `mg-reference:style:…` key). An image entry diff --git a/grok/skills/digital-human/SKILL.md b/grok/skills/digital-human/SKILL.md index 019f580..6652fbc 100644 --- a/grok/skills/digital-human/SKILL.md +++ b/grok/skills/digital-human/SKILL.md @@ -40,8 +40,8 @@ video's shots, actions, or timing. For original-content translation use ## Resolve the character -Reuse the character the user attached through **+ → 我的资产 → 角色**. Otherwise -read the context's `` section or call `manage_character action="list"`. +Reuse the character the user names or attached in ChatCut. Otherwise read the +`` section a ChatCut tool result carries, or call `manage_character action="list"`. Use the exact `characterId`; call `manage_character action="get"` for its current images, image URLs, description, and voice. Its `presenter` describes talking-head preparation state; the submission tool handles that state. @@ -70,8 +70,8 @@ name and main image, optionally save its selected voice, then submit with the re `characterId`. Creating the library entry alone does not request video generation. If an existing character has no main image, ask for a clear front-facing portrait and save it with `manage_character action="update"` and `addImages` before submission. -Ask for it with a `` upload in Widget Forms and save the submitted project -asset reference. +Request the file as Widget Forms describes for this host, import it as a project +asset, and save that asset reference. ### A person in an attached video @@ -133,8 +133,8 @@ voices through Widget Forms, with a localized clone-voice action. A `hasPreview` flag without a playable `previewUrl` is insufficient. Copy URLs exactly and never use a character's image or video as an audio preview. -Embedded voice choices use `` with `media-kind="audio"` on each -playable option. Do not replace auditions with text-only names. When the script +Keep each voice's playable preview with its choice; Widget Forms decides how this +host lets the user audition it. When the script also needs confirmation, include the voice selector and script field in one widget. An upload or choice is not voice-cloning authorization. diff --git a/grok/skills/widget-forms/SKILL.md b/grok/skills/widget-forms/SKILL.md index ccb93f1..c4d384d 100644 --- a/grok/skills/widget-forms/SKILL.md +++ b/grok/skills/widget-forms/SKILL.md @@ -9,9 +9,9 @@ Use ordinary chat text for questions and choices. Use chat image bubbles only fo ## Stable visual-preview path -The only media-preview path to rely on is a normal chat image bubble, and only for Motion Graphic references or Design Style preset thumbnails. +The only media-preview path to rely on is a normal chat image bubble, and only for Motion Graphic references, Design Style preset thumbnails or character photos. -For an MG reference or Design Style choice: +For an MG reference, Design Style or character choice: 1. Select a small relevant set, normally 3–6 options. 2. Obtain each returned thumbnail or preview-image URL from the live tool result or the pinned widget-media manifest. Never invent or rewrite the URL. @@ -20,7 +20,7 @@ For an MG reference or Design Style choice: 5. Immediately follow them with concise localized choices, using either Grok Bot's native option cards or ordinary text options. Keep every visible label mapped to the exact returned preset or reference id. 6. Stop and wait for the user's selection. If an image cannot be downloaded or displayed, omit that bubble and keep the corresponding choice; do not expose the source URL. -Do not use this image-bubble path for avatars, voices, audio samples, generated project video, or general project preview. +A character choice uses this path too, with each character's main image URL from `manage_character get` and its `characterId` as the stable id. Do not use it for voices, audio samples, generated project video, or general project preview. ## Text questions and choices @@ -30,7 +30,6 @@ For every other question, ask only the minimum blocking question. Use either Gro - `single_choice`: use short native option cards or ordinary text choices; accept the selected label. - `explicit_consent`: quote the caller's full consent statement and require an explicit affirmative response. Never infer consent from an attachment or another answer. - `audio_reference`: ask the user to attach the source file in the conversation, then load `asset-import`. Return the imported ChatCut audio `assetId`; never treat a local path, attachment URL, or raw bytes as an asset id. -- `avatar_choice`: show names and concise returned descriptions as text. Tell the user that visual previews are available in ChatCut's Digital Humans library when previewing matters. - `voice_choice`: show names and concise returned traits as text. Tell the user to audition voices in the ChatCut editor or voice library before choosing when sound matters. ## Widget requests from other workflows @@ -51,8 +50,9 @@ stop-and-wait points, and present each control this way: - `playable_single_choice` or another playable picker: `voice_choice` above. - `playable_preview` for a newly cloned voice: the cloned-voice handling under Audio and video restrictions below. -- `` or an avatar picker: `avatar_choice` above, unless the choice - is an MG reference or Design Style preset, which uses the visual-preview path. +- `` with images, such as a character, MG reference or Design + Style preset: the visual-preview path above. With audio options: + `voice_choice` above. - `` and other editor actions: unavailable here; follow the workflow's external-host branch instead. diff --git a/kimi/skills/digital-human/SKILL.md b/kimi/skills/digital-human/SKILL.md index 2432939..a9f24ce 100644 --- a/kimi/skills/digital-human/SKILL.md +++ b/kimi/skills/digital-human/SKILL.md @@ -1,338 +1,148 @@ --- name: digital-human -description: Create and manage script- or audio-driven AI avatar videos from an official or saved presenter, an imported portrait, or a representative still prepared from video. Use for 数字人、数字分身、虚拟人、照片开口说话、人像口播、改稿不重拍, AI avatar, AI avatar video, avatar video, talking avatar, talking photo, photo avatar, video avatar, AI presenter, virtual presenter, virtual spokesperson, digital human, or a person's digital twin, including choosing or creating the avatar and deciding its voice, script, aspect ratio, and output quality. Do not use for a static profile-picture avatar, game or 3D character creation, an industrial digital twin, ordinary B-roll, generic image-to-video, video translation, talking-head editing, lip-sync dubbing alone, or TTS-only requests. +description: | + Create script- or audio-driven talking-head videos with a user's saved character, + portrait, or a representative still from video. Use for 数字人、角色口播、数字分身、 + 照片开口说话、人像口播, AI avatar, talking photo, AI presenter, or digital twin. + Choose between direct-to-camera speech and short character performances by the + requested shot. Do not use for static avatars, game or 3D characters, video + translation, existing talking-head editing, lip-sync dubbing alone, or TTS alone. --- # Digital Human -Create a reusable presenter identity or generate a lip-synced presenter video -without exposing the underlying provider. Treat avatar selection, speech, script, -format, consent, generation, and verification as one stateful conversation. - -## When to Use - -Use this skill when the user wants to: - -- Make an official or saved AI avatar, talking avatar, or virtual presenter speak. -- Turn a consented portrait into a reusable digital-human identity. -- Use a representative still from an imported video as a photo avatar. -- Create a synthetic AI presenter from a description, when live capabilities allow it. -- Draft or revise the narration for a digital-human video. -- Regenerate, reorder, remove, retry, or accept segments from an avatar-video batch. - -Treat **AI avatar** as the broad default English concept. **Digital twin** means a -reusable likeness of a specific real person; **photo avatar** or **talking photo** -means a still-image source; **AI presenter**, **virtual presenter**, and **virtual -spokesperson** describe the presenter's role. **Digital human** is valid but is often -used for broader enterprise or interactive experiences. - -Route adjacent requests elsewhere: - -- Speech, voice audition, or voice cloning without avatar video: use the Voice Skill. -- Ordinary footage cleanup or presenter editing: use the Talking Head Guide. -- Generic image-to-video or video generation without a speaking avatar: use the - Video Generation Skill. -- Translation or dubbing of an existing video: use the dedicated translation or - dubbing workflow when available. - -## Required Inputs - -Resolve these slots before submission, but do not ask for them in a fixed order: - -1. **Target project** — use the active project when it is unambiguous. -2. **Avatar identity and ready revision** — an official avatar, a saved avatar, or - a newly created photo or synthetic avatar. -3. **Speech source** — one exact available voice, one cloned voice, or one imported - audio asset. A text-driven source also requires the final script. -4. **Final script** — preserve the user's wording and punctuation. Show any AI-written - draft in full and let the user edit or approve it before generation. -5. **Aspect ratio** — `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. Explicit user - choice wins; otherwise omit it so the backend uses `auto` and preserves the - selected avatar's original framing. Do not substitute the current project ratio. -6. **Resolution** — `1080p` by default; use `720p` when requested or when live - capabilities require it, and explain any fallback. -7. **Optional controls** — motion guidance only when the live capability response - says it is supported. For an official catalog avatar, use its returned - `hasBackground` value: preserve the original scene when true and request a - transparent background when false. For a saved/custom avatar, preserve its original - background by default. Request a transparent background only when the user explicitly - asks to remove the background or use transparency. Do not ask the user to choose a - background mode; briefly state the resolved default before submission. An explicit - user request to preserve or remove the background always wins. - -For avatar creation also resolve: - -- A non-empty name. Keep numeric validation limits out of the question label; - only explain a limit if the submitted name actually fails validation. -- One final avatar-creation source: an imported JPEG/PNG portrait, a - representative JPEG/PNG still prepared from an imported video, or a prompt - for a synthetic identity when supported. -- Explicit consent when a real person's likeness is involved. - -Do not pass a video directly to photo-avatar creation unless the live capability -contract explicitly supports it. For an imported video, use the host's prepared -representative still or help select a clear, unobstructed, front-facing frame, then -use that image asset as the source. - -When a user attaches or references a video containing a person and says "this -avatar", "this person", "use her/him", `这个形象`, `这个人`, `用她`, `用他`, or an -equivalent phrase, resolve the person in that exact attachment as the intended -likeness source. Do not treat the attachment as the requested speech source merely -because it has audio or a transcript. If the user also supplies a new topic or an -approved script, that new topic or script wins; do not read or reuse the video's -spoken content unless the user explicitly asks for it. - -The attached video is a prospective avatar source, not a ready `identityId`. Enter -the video-to-avatar creation flow: prepare a clean representative still, obtain the -required likeness consent, create the identity, and then reuse that identity for the -requested generation. Do not show official or saved avatar choices while this -prospective source remains valid. Offer other identities only when the user declines -to use the person in the video or no eligible face frame can be prepared. - -If "this" could reasonably mean the person's likeness, the video's existing pixels, -or its spoken content, ask exactly one targeted clarification before choosing a -workflow: "Do you want to use the person in this video as the avatar and have them -speak your new script?" Do not replace this with a generic "What do you want to do -with this video?" question. - -When the user is about to upload or pick a video for avatar creation, tell them up -front that the chosen frame must show the face clearly with nothing covering it — no -burned-in subtitles, captions, watermarks, stickers, or hands over the face — and -prefer a frame with a neutral, front-facing pose. Never upload an inspection contact -sheet or any preview that carries an overlaid time label as the avatar source; use a -clean frame with no added markings. - -A video source also carries the person's voice. After the avatar is created from a -video, offer to clone that voice from the same video so the avatar speaks with it: - -1. Run the Voice Skill's cloning flow (`manage_custom_voice action=clone` with the - video's asset and explicit voice-cloning consent — likeness consent alone does - not cover the voice). -2. When the cloned voice is ready, call - `manage_avatar action=set-default-voice identityId= voiceRef=` - so it becomes this avatar's saved default voice. -3. Later generations with this avatar should use the returned `defaultVoiceRef` by - default, while still letting the user pick another voice. - -This is an offer, not an automatic step: skip it when the user declines or the video -has unusable audio (music, multiple speakers, heavy noise). - -## Workflow - -This is an intent-driven constraint resolver, not a linear wizard. Do not walk the -user through every section or ask for fields in a preset order. - -On every turn: - -1. Extract all facts already supplied by the user, attachments, active project, - selected media, saved avatar state, and prior answers in this request. -2. Resolve deictic phrases such as "this", `这个`, `这个形象`, and `这个人` against - the attached or selected media and the user's action words before inferring a - generic media workflow. -3. Infer the user's immediate requested action: explore, choose, create, revise, - generate, inspect progress, retry, reorder, remove, accept, place, or export. -4. Compute only the blockers for that action. Defaults and live saved bindings count - as resolved unless they conflict with an explicit request. -5. If nothing blocks the action, execute it immediately. -6. Otherwise ask about the single most important missing or ambiguous constraint, - preferably with structured choices, then recompute from the new state. - -For example, a user who names a ready avatar, supplies approved text, and accepts its -saved voice should not be asked to choose them again. A user inspecting a running -batch does not need to resolve a new script or aspect ratio. Consent is requested only -for the creation or cloning action that needs it. - -### Load live capabilities and state when relevant - -Use `ToolSearch` to load `manage_avatar` and `submit_avatar_video` if they are not -already visible. Call `manage_avatar` for live capabilities before -offering creation or advanced options. - -If the user has neither selected an exact identity nor supplied a prospective -likeness source, list the current official and saved identities before recommending -one. Do not infer availability from chat history, examples, or provider knowledge. -For official avatars, use the live catalog's `tags` as factual selection filters and -its `description` as recommendation guidance. Match those fields against the user's -requested presenter and use case, and never infer a missing age, gender, ethnicity, -role, setting, or capability from the preview's appearance. -For the empty saved-avatar recommendation state, call the official catalog with -`limit: 5`; do not page through the catalog or turn a full catalog page into one -recommendation form. - -An avatar attached from ChatCut's Digital Humans library or native visual picker is -already an exact selection. Reuse its attached ChatCut `avatarId` as `identityId` -across later confirmation and retry turns. An official attached identity is directly -usable by `submit_avatar_video`; it does not need to be imported or rediscovered in -the first catalog page. Before resolving speech for an attached identity, call -`manage_avatar` with `action: get` and that exact ID unless a live result in the -current turn already supplies its `defaultVoiceRef`. Never reject or replace an -attached avatar just because it is absent from the five-item recommendation page. - -Only use identities and revisions reported as ready. Pending, failed, deleting, -deleted, or orphaned identities are not valid generation inputs. - -If the digital-human tools are unavailable, explain that this generation capability -is not connected. Do not silently substitute generic video generation or call the -backend HTTP routes directly. - -### Resolve the avatar when it is missing or invalid - -Respect an explicit user choice. Otherwise: - -- A person-containing video explicitly or provisionally resolved as the likeness - source: continue the video-to-avatar creation flow. Do not enter the generic avatar - picker and do not ask the user to choose among saved or official identities. -- No ready saved avatar: show at most five suitable official avatars and include the - reserved **create** card in the same choice surface. This is a strict empty-state - contract: never render a sixth official recommendation, and never omit the - creation option merely because official avatars are available. -- Ready saved avatars exist: show them first (up to five total cards), then the - reserved create card. -- Exactly one ready saved avatar: offer it first and ask whether to use it. -- Multiple ready saved avatars: prefer higher live usage count; when counts tie, prefer - the newest creation. If usage metadata is absent, rank by newest creation only; - never invent a frequency. - -Avatar identity selection is media-preview-only and is a strict exception to the generic -single-branch `` rule. Always use visual cards through the Widget Forms -Skill, including when exactly one ready saved avatar exists. Never ask a text-only -category question such as "use the saved avatar" versus "browse official avatars", -and never use ``, ``, or label-only identity cards for avatar -selection. For an avatar card in ChatCut's native widget — official or the user's own -saved avatar — write only its exact `identityId` as `avatar-id`; the host resolves the -current name, preview video or fallback image, aspect ratio, and submitted value from the live avatar -identity even when the tool payload does not expose a preview URL. Never copy `value`, -`name`, `media`, or `aspect-ratio` onto an `avatar-id` option. If the host cannot -resolve preview media, omit that identity rather than falling back to text. Prefer -the live preview video when available and use the preview image only as its poster -or fallback. When -the user has ready saved avatars, place their cards before official recommendations. -Ask one most important missing question at a time and do not repeat answered questions. - -The raw tags below describe the embedded ChatCut protocol. In Kimi Work, use the host's supported preview/choice mechanics. Map each identity's `identityId`, `name`, `previewVideoUrl`, and `previewImageUrl` to that surface when available. Keep `create_avatar` as a separate choice returned to this workflow. -For a saved identity whose `list` row has no preview URLs, call `manage_avatar -action=get` for that exact candidate before rendering it; use the selected ready -revision's returned preview media and do not invent or copy media from another avatar. - -One reserved label-only option value opens a native editor surface instead of -submitting an answer. Write it with no `media`: - -- `create_avatar` — opens the native creation dialog where the user uploads a photo - and creates a personal avatar. Keep its card label a short "create your own - avatar" phrase (e.g. Create your own avatar); do not mention uploading a photo — the - dialog explains that itself. - -Clicking that card does not produce a form answer. When the user finishes in the -native dialog, their chosen or newly created avatar is sent back automatically as -this question's answer with the avatar attached; treat that reply as the avatar -selection and do not re-ask. - -The selection shape is therefore one `` containing up to five avatar -preview cards (saved first, then official) followed by the creation card. Localize -the visible question and card label to the conversation language: - -```text - - - - - - -``` - -Creating an identity is separate from generating a video: - -1. Resolve the source or prompt and name. -2. Obtain required consent. -3. Create the identity through `manage_avatar`. -4. Poll until it is ready or terminal. -5. Save it under **My avatars**. Do not auto-generate a video merely because avatar - creation succeeded. - -When asking for a portrait in an embedded ChatCut Widget, always mark the upload as an avatar source so -the host uses the same validation, JPEG normalization, private transport storage, and -upload-readiness behavior as the native Digital Humans dialog: - -```text - - - - -``` - -Do not use an unmarked generic `` upload for avatar creation. -Published plugin forms do not provide this native upload control. Ask the user to -attach the portrait through their host, then use the Asset Import Skill and pass the -returned ChatCut image asset id to `manage_avatar`. - -### Resolve speech only when the requested action needs it - -For an official avatar, never silently use its paired voice and never immediately -open the broader voice catalog. When the live result provides `defaultVoiceRef` and -the user has not already made this choice, first ask one blocking text-only decision -in the conversation language: whether to use this avatar's official built-in voice -or choose another voice. In embedded ChatCut, render that decision with localized -``, then stop and wait. -If the user accepts, resolve speech with that exact `defaultVoiceRef` and do not call -`manage_avatar action=voices`. Call `action=voices` only after the user chooses -another voice, or when the live identity has no `defaultVoiceRef`. Do not ask for -language or script and then pre-emptively call `action=voices` in the same turn. - -For a saved or newly created avatar, reuse its saved voice binding when present and -confirm it; otherwise offer available voices. When the user settles on a voice they -want this avatar to keep, save it with `manage_avatar action=set-default-voice`; -`list`/`get` then return it as the avatar's `defaultVoiceRef`. - -For voice discovery, audition, or cloning, follow the Voice Skill and reuse its live -voice list, consent gate, and generated voice asset. Do not duplicate voice-cloning -logic here. - -When an exact voice still needs to be selected, never present voice names as prose -beside a free-text field. Use the exact `voiceRef` returned by `manage_avatar` as the -submitted value. Voice selection is audio-preview-only and is a strict exception to -the generic `` rule: - -- Never render ``, ``, prose voice names, or label-only voice - options. A voice without a returned playable `previewUrl` must be omitted from the - recommendation surface rather than presented as an unplayable choice. -- When the user wants to audition voices, prefer up to five suitable available - official, preset, or custom voices that actually include `previewUrl`. Render them as one - required ``; `hasPreview: true` - without `previewUrl` is not playable and must not be presented as an audition - card. Always append one no-media `` action, localized to the conversation language, as the - final card. This clone option is required on every recommended-voice surface. -- An official avatar's paired default voice may still be used without listing the - broader official voice catalog. Do not show that default voice as an audition - card unless an actual audio `previewUrl` was returned for it. Never reuse the - avatar image or video URL as a voice preview. -- Copy each returned `previewUrl` exactly. In particular, keep - `/voice-samples/...` root-relative; do not expand it to `chatcut.com` or invent - another host. -- When script text is also being confirmed, put the voice selector and script field - in the same widget. Do not make the user type a voice name into the script field. - -```text - - - - - - - -``` - -If the source began as a user video, ask whether the user wants that video's voice. -If yes, follow the Voice Skill's explicit voice-cloning authorization flow before -using it. An attachment alone is not permission to clone. If no, continue with the -normal voice picker. - -An existing project audio asset may drive the avatar directly. When audio is the -speech source, do not invent or require text unless the user also wants a transcript -or script review. +If `manage_character`, `manage_voice`, or `submit_avatar_video` are not loaded yet, load them with `ToolSearch` before calling them, so you use their real parameters instead of guessing. + +A character is a person saved in the user's My Assets (我的资产) with images, a description, +and an optional voice. Choose how to use that character from the requested shot. +Custom talking-head generation requires a paid plan. If the tool returns a paywall, +explain that requirement accurately; do not substitute another workflow to bypass it. + +## Choose the performance + +- **Direct-to-camera speech, any length:** use `submit_avatar_video` with + `characterId`. The first use may require preparing the talking-head appearance. +- **Actions, changing scenes, multiple shots, or just one or two lines:** follow + `video-gen` and use `submit_video`. Put the character's image URLs in `refImages`, + main image first, and include the character description in the prompt to keep + the person consistent. Each generated clip is short, typically about 5–15 seconds. + +This choice belongs to the shot, not to the character. One video may use both: +product explanations can be talking-head segments while product scenes use video +generation. If the intent is unclear, ask one question explaining the difference, +for example: + +> 你想让角色贴着镜头念稿,还是演动作、换场景?口播长度不限,第一次需要准备形象; +> 视频生成能演动作和场景,但每段很短,约 5–15 秒。 + +Talking-head generation creates a new performance; it does not preserve an original +video's shots, actions, or timing. For original-content translation use +`video-translation`; for new dialogue over existing footage use `voice` and optional +`lipsync`. Existing footage cleanup belongs to `talking-head-guide`. + +## Resolve the character + +Reuse the character the user names or attached in ChatCut. Otherwise read the +`` section a ChatCut tool result carries, or call `manage_character action="list"`. +Use the exact `characterId`; call `manage_character action="get"` for its current +images, image URLs, description, and voice. Its `presenter` describes talking-head +preparation state; the submission tool handles that state. + +When several characters could fit, ask the user to choose. Widget Forms can show +image choices using each candidate's main image URL from `manage_character get`, +or ask a concise text question. Keep each ID, name, and image from the same returned +character. An already attached character does not need another selection step. +If there are no saved characters, offer to create one from the user's portrait. + +Use `manage_character` for `list`, `get`, `create`, `update`, and `delete`: + +- Create with `name`, optional `description`, `addImages`, and optional `voice`. +- Each `addImages` entry is `{source, label?}`. `source` is a project asset reference + or an HTTPS URL. The first image is the main image: a clear, front-facing JPG/PNG + showing one person. +- Update the selected `characterId` to change its `name` or `description`, add images + with `addImages`, remove them with `removeImageIds`, or set `mainImageId`. +- Save a voice as `voice: {id, name, provider}` using the selected voice's real + values; `provider` is `custom`, `doubao`, or `elevenlabs`. Set `voice: null` to clear + it. Keep provider values internal. +- Use `delete` only when the user requests deleting that character. + +For a supplied photo and a request to make it speak, create the character with its +name and main image, optionally save its selected voice, then submit with the returned +`characterId`. Creating the library entry alone does not request video generation. +If an existing character has no main image, ask for a clear front-facing portrait +and save it with `manage_character action="update"` and `addImages` before submission. +Request the file as Widget Forms describes for this host, import it as a project +asset, and save that asset reference. + +### A person in an attached video + +When the user says “this person”, `这个形象`, `用她`, or `用他`, resolve that phrase +against the attachment and the requested action. A video is a prospective likeness +source, not a saved character. Prepare a clean representative JPG/PNG still and use +it in `addImages`. The face must be clear, front-facing, and unobstructed by captions, +watermarks, stickers, hands, or inspection time labels. Do not use a contact sheet. + +If the user supplies a new topic or approved script, use that content. The video's +audio or transcript is not automatically the requested speech. If the intended use +is unclear, ask whether they want this person to speak the new script. + +Offer to clone the video's voice when useful, following the Voice Skill and its +separate explicit voice-cloning consent. Skip this when declined or when the source +has unusable audio. Once ready, the selected clone can be saved with +`manage_character action="update"` and `voice`; likeness consent does not authorize +voice cloning. + +## Resolve only what the next action needs + +Reuse the active project, attachments, prior answers, and current character state. +Ask only about the most important remaining blocker; do not run a fixed wizard. +Progress checks do not require resolving another script or output format. + +Before talking-head submission resolve: + +1. **Character:** the user's chosen `characterId` and a usable main image. +2. **Speech:** final text with the character's voice or an explicit `voiceRef`, or + a project audio asset containing the exact desired speech. +3. **Script approval:** preserve user-supplied wording and punctuation. Show an + AI-written draft in full for editing or approval before generation. +4. **Framing:** omit `aspectRatio` to preserve the character image's framing with + `auto`. Explicit choices may use `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. + Do not substitute the project ratio. +5. **Quality:** default to `1080p`; use `720p` when requested or required by the + tool, explaining any fallback. + +Omit `removeBackground` unless the user explicitly wants to keep or remove it; +the default preserves the source image background. Briefly state that default +without making it another question. Use `motionPrompt` for requested motion guidance +within the tool's supported behavior. Do not promise unsupported crop, transparency, +resolution, or motion control; report tool limitations plainly. + +### Speech and voice auditions + +Use `speech: {type:"voice", text}` to use the character's saved voice automatically. +Omitting `voiceRef` uses that binding; do not ask the user to confirm it again. +An explicit voice change overrides it. If there is no saved voice, let the user +choose through `manage_voice action="list"` and save the result with +`manage_character action="update"` and `voice`, or pass the selected `voiceRef` +in `speech` for this generation only. Keep the saved voice object and the returned +submission `voiceRef` distinct; do not invent one from a display name. + +For discovery, audition, and cloning, follow the Voice Skill and use voices available +for avatar video. Keep each voice's returned name, reference, and preview together. +Show up to five suitable playable +voices through Widget Forms, with a localized clone-voice action. A `hasPreview` +flag without a playable `previewUrl` is insufficient. Copy URLs exactly and never +use a character's image or video as an audio preview. + +Keep each voice's playable preview with its choice; Widget Forms decides how this +host lets the user audition it. When the script +also needs confirmation, include the voice selector and script field in one widget. +An upload or choice is not voice-cloning authorization. + +An existing audio asset drives speech with `speech: {type:"audio-asset", assetId}`. +It is the final spoken content, not a cloning sample. Do not require text unless +the user also wants transcription or script review. ### Resolve a script only for text-driven generation @@ -347,8 +157,8 @@ written collaboratively. When drafting: Each explicit text segment must contain 1–5000 characters. Omit explicit segments for an ordinary script so the backend generates one continuous video up to 5000 characters. Use explicit segments only when the user requests meaningful paragraph -or shot boundaries. Longer text is split automatically and may create at most 10 -segments. Never summarize, translate, drop, duplicate, or reorder content to fit a +or shot boundaries. Longer text is split automatically into ordered +segments; the tool rejects a script that exceeds its segment limit. Never summarize, translate, drop, duplicate, or reorder content to fit a limit. Preserve wording and punctuation; non-semantic line whitespace may be normalized automatically. @@ -364,88 +174,55 @@ Do not add implementation details about where or how the work runs, and never ex hidden derived audio assets. Never shorten or summarize approved content to satisfy these limits. -### Resolve output settings from intent and available defaults +### Submit and prepare the appearance -Preserve the selected avatar's original framing by default: omit `aspectRatio` and -let the backend resolve it to `auto`. Never use the current project ratio as the -avatar-video default. If the user explicitly asks for a different ratio, pass that -request instead. +Summarize consequential resolved choices, then call `submit_avatar_video` with +`characterId` and `speech`. Use `segments`, `aspectRatio`, `resolution`, +`removeBackground`, and `motionPrompt` only as needed. The tool shows the live credit +estimate in a confirmation card before anything is charged; do not calculate a separate +price. -For an official catalog avatar, resolve the default from the `hasBackground` field -returned by `manage_avatar action="catalog"`: +On a character's first talking-head use, the tool may return that the appearance +needs preparation, the credit cost, and a request for authorization without creating +a video. Tell the user that returned cost and ask them to explicitly confirm their +right to use this person's likeness using the consent wording below. After the user +agrees, repeat the same submission parameters with `confirmedConsent: true`. +Do not set it based on a photo upload or character selection. -- `hasBackground: true` — pass `removeBackground: false` and preserve the original - scene. -- `hasBackground: false` — pass `removeBackground: true` so the result is a - transparent WebM when transparency is supported. +Preparation takes a few minutes. While it runs the tool says so and tells you how to +continue: tell the user, register the automatic follow-up it names with `track_progress`, +and end the turn. When the follow-up arrives, repeat the same submission parameters +without `confirmedConsent`. Do not report preparation as a completed video. +A paywall means custom talking-head generation requires a paid plan; report it +without claiming that preparation or generation succeeded. -For a saved/custom avatar without catalog metadata, keep `removeBackground: false` as -the default and preserve the source image background. Pass `removeBackground: true` -only when the user explicitly asks for a transparent background or background removal. -Do not turn this into a blocking question. Before submission, briefly tell the user -whether the original scene will be preserved or the result will be transparent. An -explicit user request overrides the catalog default. If transparency is unsupported, -explain the limitation and keep the original background. +After submission returns generation `jobIds`, call `track_progress` with +`target="generation"`, `action="wait"`, and those `jobIds`, unless the user only +asked to queue work. Follow its wait contract: if it registers an automatic follow-up, +end the turn normally; otherwise await its result. Do not add sleeps or busy polling. +Use `action="status"` for a later progress question. Do not quote coarse numeric +provider progress as a completion estimate. -Do not promise crop, background removal, motion control, transparency, resolution, -or prompt-based identity creation until the live capability response confirms it. +Keep long-script segments in their original order. To redo a segment, call +`submit_avatar_video` again for that segment with the same character and the intended +speech/settings; do not regenerate successful segments without a request. Arrange, +remove, or replace generated clips on the timeline as the editing task requires. +Keep the old result until the user accepts a replacement. Generated clips are normal +video assets; place or export them only when the task calls for it. -### Submit as soon as the required constraints are resolved +### Completed videos that do not meet the likeness request -Summarize the resolved avatar, voice or audio source, script state, aspect ratio, and -resolution before the first generation when any choice remains consequential. Then -call `submit_avatar_video` once. +A completed avatar video can still fail the user's likeness requirements. When they ask to correct lips, cheeks or other facial proportions, check which changes the actual tools support before offering another generation. Do not describe the same character, photo and audio as a stricter facial correction unless a supported change plausibly addresses that specific correction; describe what will change and its limits. Explain any limitation plainly. A retry may cost credits and may not improve the likeness; retain the existing result until the user accepts a replacement. Honor an explicit, informed retry request, but do not keep proposing paid repeats of unchanged inputs after they have failed. If the user asks to stop, not export, or seek a credit review, follow that request and use the existing support route without requiring another generation or an export. -Treat long-script output as an ordered batch of avatar-video segments, not as one -audio file or one opaque job. After submitting, tell the user that generation has -started and end the turn. Never call `track_progress`, `manage_avatar action=get-batch`, -or add a Bash sleep in the same turn merely to keep the turn open, including when a -queued follow-up step will eventually need the completed asset. The project library -shows the in-progress generation and receives the completed video assets in the -background. +When a generation tool returns `FEATURE_NOT_INCLUDED` for a `generation.*` feature, use its payer and access details. If the project owner needs Pro, explain that the owner pays and must upgrade; the member's own Pro does not unlock the project. Do not give the member a personal upgrade link or send them to their own billing menu. They can find the owner in the project's existing members view when available. Otherwise share the returned upgrade link, or ChatCut’s avatar menu → Upgrade. Existing credits are preserved and generation still consumes credits after upgrading. This rejection does not charge for generation; existing agent conversation fees are separate, so do not claim the whole conversation was free. Preserve the exact original tool input and project, then stop. Do not automatically search for or substitute stock media; offer alternatives only if the user explicitly asks. -Call `track_progress` in a later user turn only when the user asks for status or resumes -a follow-up step that depends on the completed asset, such as placing it on the timeline -or reviewing the result. When a job is still non-terminal, say that generation is still -running without quoting its numeric progress: provider progress values are coarse -internal stages, not user-facing completion estimates. Use `manage_avatar -action=get-batch` when ordered segment and attempt state is needed. Keep the stable -segment key and original order. - -For a failed segment, offer a targeted retry. Do not resubmit successful segments or -the entire batch without the user's instruction. Use the management tool for supported -segment regeneration, removal, reordering, attempt selection, and acceptance. - -Generated results become normal ChatCut video assets. Add them to the timeline only -when the user requested placement or it is clearly part of the active editing task. -Do not export unless asked. - -## Provider Selection - -Provider selection is an internal ChatCut backend detail. Always call the -ChatCut-native tools and let the backend construct provider requests, resolve private -IDs, enforce capabilities, and persist results. Never call an underlying provider API -or CLI directly from this skill. - -Select only through ChatCut capabilities, catalog results, saved bindings, and the -native tools. User-facing language is limited to concepts such as: - -- **Official avatars** and **My avatars** -- **Official voices**, **cloned voices**, and **project audio** -- **Photo avatar** and **synthetic avatar** - -Provider names, engine or model names, provider-specific avatar or voice IDs, raw -provider URLs, request payloads, and provider error messages are internal. Never show -them in prose, option labels, progress updates, or failure messages. Translate failures -into actionable product language while preserving the real status. - -Do not route around ChatCut's feature gates, quotas, billing checks, safety policy, -catalog visibility, or provider registry. +When the user returns with “continue”, “继续”, “jixu”, or says they upgraded, recheck current payer eligibility and credits for the original input with `estimate_generation` when supported, otherwise the original tool’s precheck. If there is no separate precheck, one retry of the original tool lets its server check current access. If eligible, submit the original input once; if still blocked, explain the current blocker and stop. Do not ask the user to choose between the original generation and stock media again or state the old rejection as current status without checking. If a job was already accepted, track it instead of submitting again. Never switch generation tools, providers or services to bypass a gate. ## Consent and Safety -A real-person photo avatar requires an explicit affirmative confirmation immediately -before creation. Use the user's language and keep the meaning exact. For Chinese: +Before preparing a real person's talking-head appearance, obtain explicit affirmative +consent after disclosing the tool's returned preparation cost. Use the user's language +and preserve the full meaning. For Chinese: > 我确认拥有该肖像,或已获得创建和使用该数字人形象的授权,并承诺不将其用于 > 冒充他人、欺诈或其他违法用途。 @@ -459,57 +236,24 @@ En español: > Confirmo que soy titular de los derechos sobre esta imagen o que tengo permiso para crear y usar este avatar digital, y no lo utilizaré para suplantar identidades, cometer fraude ni realizar actividades ilícitas. -Do not infer consent from an upload, a previous unrelated confirmation, ownership of -the project, or a third party's instruction. Record consent only through the native -tool's consent fields. Synthetic prompt-created identities do not require real-person -likeness consent unless the prompt or references identify a real person. +Do not infer consent from an upload, a prior unrelated confirmation, project +ownership, or a third party's instruction. Pass `confirmedConsent: true` only after +the user's explicit agreement for this preparation. Follow the Voice Skill's separate +consent gate for voice cloning. Refuse non-consensual impersonation, deceptive identity use, fraud, harassment, -sexual exploitation, or attempts to evade safeguards. Do not create a real-person -identity from uncertain authorization. Voice cloning has a separate consent gate; -follow the Voice Skill even when avatar consent was already obtained. +sexual exploitation, or attempts to evade safeguards. Use ChatCut-native tools; +let them enforce access, billing, and preparation state. Do not call provider APIs +directly. Keep provider names, private IDs, URLs, payloads, and raw provider errors +out of user-facing copy; explain the actual failure and next step in product terms. ## Verification -Before reporting success, verify through live tool results: - -1. The chosen identity revision was ready when submitted. -2. The job or batch reached a completed terminal state and returned output asset IDs. -3. Every expected segment key exists exactly once and appears in the confirmed order. -4. Text-backed segment manifests preserve the confirmed wording and punctuation, - allowing only documented non-semantic whitespace normalization. -5. The generated videos exist as usable project or media-library assets. -6. Any requested timeline placement is present and ordered correctly. - -Preview the generated assets or composed timeline when visual verification is available. -Do not claim the face, lip sync, framing, or background looks correct from status fields -alone. If visual inspection is unavailable, say what was structurally verified. - -## Hard Rules - -- Never expose provider, engine, or model identity to the user. -- Never call an underlying provider directly or construct provider payloads in the - conversation layer. -- Never invent an avatar, voice, capability, usage count, saved binding, or asset ID. -- Never generate from a non-ready identity revision. -- Never treat a video as a supported identity source without an explicit live capability; - prepare a representative still for the current photo-avatar contract. -- Never pass an external source URL to avatar creation; import the source into the - target ChatCut project first. -- Never infer likeness consent or voice-cloning consent from an attachment. -- Never clone a video's original voice without the Voice Skill's explicit authorization. -- Never silently rewrite, translate, truncate, reorder, or duplicate confirmed script text. -- Never ask for information already available from the project, attachment, or live tools. -- Never run a fixed wizard or ask for avatar, voice, script, ratio, and resolution in a - preset sequence; resolve only what the current user action still needs. -- Ask one most important missing question at a time; prefer structured visual choices. -- Never show provider-private IDs, URLs, payloads, or raw errors in user-visible output. -- Never bypass ChatCut tools, feature gates, billing, quota, or safety checks. -- Never retry a whole successful batch because one segment failed. -- Never claim visual quality without inspecting the generated result. -- Never place media on the timeline or export it unless the task calls for that action. -- For official catalog avatars, obey the returned `hasBackground` default: preserve a - configured scene background and remove a configured absent background. For - saved/custom avatars, preserve the source background unless the user explicitly asks - for transparency or background removal. Explicit user intent always overrides either - default. +Before reporting success, verify from live results that generation completed and +returned usable video assets, every expected segment is present once in the confirmed +order, and the text preserves approved wording and punctuation. Verify any requested +timeline placement. Preparation or queued jobs alone are not success. + +Preview the generated assets or composed timeline when visual inspection is available. +Do not claim that likeness, lip sync, framing, or background looks correct from status +fields alone. State what was structurally verified if visual inspection is unavailable. diff --git a/qwen/skills/digital-human/SKILL.md b/qwen/skills/digital-human/SKILL.md index c9328e7..b08c74e 100644 --- a/qwen/skills/digital-human/SKILL.md +++ b/qwen/skills/digital-human/SKILL.md @@ -3,272 +3,145 @@ name: "数字人口播" version: "2.0.4" name_en: "digital-human" name_zh: "数字人口播" -description: "使用合适的数字人形象制作讲解视频。" -description_en: "Create and manage script- or audio-driven AI avatar videos from an official or saved presenter, an imported portrait, or a representative still prepared from video. Use for 数字人、数字分身、虚拟人、照片开口说话、人像口播、改稿不重拍, AI avatar, AI avatar video, avatar video, talking avatar, talking photo, photo avatar, video avatar, AI presenter, virtual presenter, virtual spokesperson, digital human, or a person's digital twin, including choosing or creating the avatar and deciding its voice, script, aspect ratio, and output quality. Do not use for a static profile-picture avatar, game or 3D character creation, an industrial digital twin, ordinary B-roll, generic image-to-video, video translation, talking-head editing, lip-sync dubbing alone, or TTS-only requests." -description_zh: "使用合适的数字人形象制作讲解视频。" +description: "用你保存的角色或人像照片制作口播视频。" +description_en: "Create script- or audio-driven talking-head videos with a user's saved character, portrait, or a representative still from video. Choose between direct-to-camera speech and short character performances by the requested shot." +description_zh: "用你保存的角色或人像照片制作口播视频。" user-invocable: "false" displayName: "数字人口播" --- # Digital Human -Create a reusable presenter identity or generate a lip-synced presenter video -without exposing the underlying provider. Treat avatar selection, speech, script, -format, consent, generation, and verification as one stateful conversation. - -## When to Use - -Use this skill when the user wants to: - -- Make an official or saved AI avatar, talking avatar, or virtual presenter speak. -- Turn a consented portrait into a reusable digital-human identity. -- Use a representative still from an imported video as a photo avatar. -- Create a synthetic AI presenter from a description, when live capabilities allow it. -- Draft or revise the narration for a digital-human video. -- Regenerate, reorder, remove, retry, or accept segments from an avatar-video batch. - -Treat **AI avatar** as the broad default English concept. **Digital twin** means a -reusable likeness of a specific real person; **photo avatar** or **talking photo** -means a still-image source; **AI presenter**, **virtual presenter**, and **virtual -spokesperson** describe the presenter's role. **Digital human** is valid but is often -used for broader enterprise or interactive experiences. - -Route adjacent requests elsewhere: - -- Speech, voice audition, or voice cloning without avatar video: use the Voice Skill. -- Ordinary footage cleanup or presenter editing: use the Talking Head Guide. -- Generic image-to-video or video generation without a speaking avatar: use the - Video Generation Skill. -- Translation or dubbing of an existing video: use the dedicated translation or - dubbing workflow when available. - -## Required Inputs - -Resolve these slots before submission, but do not ask for them in a fixed order: - -1. **Target project** — use the active project when it is unambiguous. -2. **Avatar identity and ready revision** — an official avatar, a saved avatar, or - a newly created photo or synthetic avatar. -3. **Speech source** — one exact available voice, one cloned voice, or one imported - audio asset. A text-driven source also requires the final script. -4. **Final script** — preserve the user's wording and punctuation. Show any AI-written - draft in full and let the user edit or approve it before generation. -5. **Aspect ratio** — `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. Explicit user - choice wins; otherwise omit it so the backend uses `auto` and preserves the - selected avatar's original framing. Do not substitute the current project ratio. -6. **Resolution** — `1080p` by default; use `720p` when requested or when live - capabilities require it, and explain any fallback. -7. **Optional controls** — motion guidance only when the live capability response - says it is supported. For an official catalog avatar, use its returned - `hasBackground` value: preserve the original scene when true and request a - transparent background when false. For a saved/custom avatar, preserve its original - background by default. Request a transparent background only when the user explicitly - asks to remove the background or use transparency. Do not ask the user to choose a - background mode; briefly state the resolved default before submission. An explicit - user request to preserve or remove the background always wins. - -For avatar creation also resolve: - -- A non-empty name. Keep numeric validation limits out of the question label; - only explain a limit if the submitted name actually fails validation. -- One final avatar-creation source: an imported JPEG/PNG portrait, a - representative JPEG/PNG still prepared from an imported video, or a prompt - for a synthetic identity when supported. -- Explicit consent when a real person's likeness is involved. - -Do not pass a video directly to photo-avatar creation unless the live capability -contract explicitly supports it. For an imported video, use the host's prepared -representative still or help select a clear, unobstructed, front-facing frame, then -use that image asset as the source. - -When a user attaches or references a video containing a person and says "this -avatar", "this person", "use her/him", `这个形象`, `这个人`, `用她`, `用他`, or an -equivalent phrase, resolve the person in that exact attachment as the intended -likeness source. Do not treat the attachment as the requested speech source merely -because it has audio or a transcript. If the user also supplies a new topic or an -approved script, that new topic or script wins; do not read or reuse the video's -spoken content unless the user explicitly asks for it. - -The attached video is a prospective avatar source, not a ready `identityId`. Enter -the video-to-avatar creation flow: prepare a clean representative still, obtain the -required likeness consent, create the identity, and then reuse that identity for the -requested generation. Do not show official or saved avatar choices while this -prospective source remains valid. Offer other identities only when the user declines -to use the person in the video or no eligible face frame can be prepared. - -If "this" could reasonably mean the person's likeness, the video's existing pixels, -or its spoken content, ask exactly one targeted clarification before choosing a -workflow: "Do you want to use the person in this video as the avatar and have them -speak your new script?" Do not replace this with a generic "What do you want to do -with this video?" question. - -When the user is about to upload or pick a video for avatar creation, tell them up -front that the chosen frame must show the face clearly with nothing covering it — no -burned-in subtitles, captions, watermarks, stickers, or hands over the face — and -prefer a frame with a neutral, front-facing pose. Never upload an inspection contact -sheet or any preview that carries an overlaid time label as the avatar source; use a -clean frame with no added markings. - -A video source also carries the person's voice. After the avatar is created from a -video, offer to clone that voice from the same video so the avatar speaks with it: - -1. Run the Voice Skill's cloning flow (`manage_custom_voice action=clone` with the - video's asset and explicit voice-cloning consent — likeness consent alone does - not cover the voice). -2. When the cloned voice is ready, call - `manage_avatar action=set-default-voice identityId= voiceRef=` - so it becomes this avatar's saved default voice. -3. Later generations with this avatar should use the returned `defaultVoiceRef` by - default, while still letting the user pick another voice. - -This is an offer, not an automatic step: skip it when the user declines or the video -has unusable audio (music, multiple speakers, heavy noise). - -## Workflow - -This is an intent-driven constraint resolver, not a linear wizard. Do not walk the -user through every section or ask for fields in a preset order. - -On every turn: - -1. Extract all facts already supplied by the user, attachments, active project, - selected media, saved avatar state, and prior answers in this request. -2. Resolve deictic phrases such as "this", `这个`, `这个形象`, and `这个人` against - the attached or selected media and the user's action words before inferring a - generic media workflow. -3. Infer the user's immediate requested action: explore, choose, create, revise, - generate, inspect progress, retry, reorder, remove, accept, place, or export. -4. Compute only the blockers for that action. Defaults and live saved bindings count - as resolved unless they conflict with an explicit request. -5. If nothing blocks the action, execute it immediately. -6. Otherwise ask about the single most important missing or ambiguous constraint, - preferably with structured choices, then recompute from the new state. - -For example, a user who names a ready avatar, supplies approved text, and accepts its -saved voice should not be asked to choose them again. A user inspecting a running -batch does not need to resolve a new script or aspect ratio. Consent is requested only -for the creation or cloning action that needs it. - -### Load live capabilities and state when relevant - -Use Qwen tool discovery, when available, to locate `manage_avatar` and `submit_avatar_video` if they are not -already visible. Call `manage_avatar` for live capabilities before -offering creation or advanced options. - -If the user has neither selected an exact identity nor supplied a prospective -likeness source, list the current official and saved identities before recommending -one. Do not infer availability from chat history, examples, or provider knowledge. -For official avatars, use the live catalog's `tags` as factual selection filters and -its `description` as recommendation guidance. Match those fields against the user's -requested presenter and use case, and never infer a missing age, gender, ethnicity, -role, setting, or capability from the preview's appearance. -For the empty saved-avatar recommendation state, call the official catalog with -`limit: 5`; do not page through the catalog or turn a full catalog page into one -recommendation form. - -An avatar attached from ChatCut's Digital Humans library or native visual picker is -already an exact selection. Reuse its attached ChatCut `avatarId` as `identityId` -across later confirmation and retry turns. An official attached identity is directly -usable by `submit_avatar_video`; it does not need to be imported or rediscovered in -the first catalog page. Before resolving speech for an attached identity, call -`manage_avatar` with `action: get` and that exact ID unless a live result in the -current turn already supplies its `defaultVoiceRef`. Never reject or replace an -attached avatar just because it is absent from the five-item recommendation page. - -Only use identities and revisions reported as ready. Pending, failed, deleting, -deleted, or orphaned identities are not valid generation inputs. - -If the digital-human tools are unavailable, explain that this generation capability -is not connected. Do not silently substitute generic video generation or call the -backend HTTP routes directly. - -### Resolve the avatar when it is missing or invalid - -Respect an explicit user choice. Otherwise: - -- A person-containing video explicitly or provisionally resolved as the likeness - source: continue the video-to-avatar creation flow. Do not enter the generic avatar - picker and do not ask the user to choose among saved or official identities. -- No ready saved avatar: show at most five suitable official avatars and include the - reserved **create** card in the same choice surface. This is a strict empty-state - contract: never render a sixth official recommendation, and never omit the - creation option merely because official avatars are available. -- Ready saved avatars exist: show them first (up to five total cards), then the - reserved create card. -- Exactly one ready saved avatar: offer it first and ask whether to use it. -- Multiple ready saved avatars: prefer higher live usage count; when counts tie, prefer - the newest creation. If usage metadata is absent, rank by newest creation only; - never invent a frequency. - -Use `widget-forms` for avatar selection in Qwen. Keep each live `identityId` -as the stable value and its returned name as the label. Show returned preview -video/images only through a supported host surface. If previews are unavailable, -disclose that and ask a labeled choice; do not fabricate visual cards or omit -every usable identity just because Qwen cannot render ChatCut's native UI. - -For a saved identity missing preview URLs, call `manage_avatar action=get` for -that candidate and use only its selected ready revision's media. Saved identities -come first, followed by suitable official identities, at most five in total. -Keep a separate `create_avatar` choice labeled "创建专属数字人" or its localized -equivalent. It is a normal answer that continues source/name/consent intake, -not a command that opens a native dialog or uploads an image automatically. - -Creating an identity is separate from generating a video: - -1. Resolve the source or prompt and name. -2. Obtain required consent. -3. Create the identity through `manage_avatar`. -4. Poll until it is ready or terminal. -5. Save it under **My avatars**. Do not auto-generate a video merely because avatar - creation succeeded. - -For a new portrait-based identity, ask for the missing portrait through Qwen's -attachment interface or use the image the user already selected. Follow -`asset-import` and pass the returned ChatCut image asset id to `manage_avatar`. -Collect the missing name and explicit consent with Qwen's native questions or -ordinary chat. An attachment is not authorization, and is not itself an assetId. - -### Resolve speech only when the requested action needs it - -For an official avatar, never silently use its paired voice and never immediately -open the broader voice catalog. When the live result provides `defaultVoiceRef` and -the user has not already made this choice, first ask one blocking text-only decision -in the conversation language: whether to use this avatar's official built-in voice -or choose another voice. Ask through `widget-forms` using Qwen native -questions or ordinary chat, then stop and wait. -If the user accepts, resolve speech with that exact `defaultVoiceRef` and do not call -`manage_avatar action=voices`. Call `action=voices` only after the user chooses -another voice, or when the live identity has no `defaultVoiceRef`. Do not ask for -language or script and then pre-emptively call `action=voices` in the same turn. - -For a saved or newly created avatar, reuse its saved voice binding when present and -confirm it; otherwise offer available voices. When the user settles on a voice they -want this avatar to keep, save it with `manage_avatar action=set-default-voice`; -`list`/`get` then return it as the avatar's `defaultVoiceRef`. - -For voice discovery, audition, or cloning, follow the Voice Skill and reuse its live -voice list, consent gate, and generated voice asset. Do not duplicate voice-cloning -logic here. - -When a voice still needs selecting, follow `widget-forms` and retain the exact -returned `voiceRef` for every label. Offer up to five suitable live candidates -and a separate `clone_voice` choice. Preview only actual returned audio URLs; -`hasPreview: true` alone is not playable, and an avatar image/video is not a -voice sample. Preserve root-relative `/voice-samples/...` values rather than -inventing a hostname. If Qwen cannot play them, disclose the limitation and -ask the choice without claiming an audition took place. Use the Voice Skill -for cloning and authorization. When script approval is also needed, collect -the text and voice choice together only if the native question UI supports it; -otherwise present the complete script in chat and obtain explicit approval. - -If the source began as a user video, ask whether the user wants that video's voice. -If yes, follow the Voice Skill's explicit voice-cloning authorization flow before -using it. An attachment alone is not permission to clone. If no, continue with the -normal voice picker. - -An existing project audio asset may drive the avatar directly. When audio is the -speech source, do not invent or require text unless the user also wants a transcript -or script review. +A character is a person saved in the user's My Assets (我的资产) with images, a description, +and an optional voice. Choose how to use that character from the requested shot. +Custom talking-head generation requires a paid plan. If the tool returns a paywall, +explain that requirement accurately; do not substitute another workflow to bypass it. + +## Choose the performance + +- **Direct-to-camera speech, any length:** use `submit_avatar_video` with + `characterId`. The first use may require preparing the talking-head appearance. +- **Actions, changing scenes, multiple shots, or just one or two lines:** follow + `video-gen` and use `submit_video`. Put the character's image URLs in `refImages`, + main image first, and include the character description in the prompt to keep + the person consistent. Each generated clip is short, typically about 5–15 seconds. + +This choice belongs to the shot, not to the character. One video may use both: +product explanations can be talking-head segments while product scenes use video +generation. If the intent is unclear, ask one question explaining the difference, +for example: + +> 你想让角色贴着镜头念稿,还是演动作、换场景?口播长度不限,第一次需要准备形象; +> 视频生成能演动作和场景,但每段很短,约 5–15 秒。 + +Talking-head generation creates a new performance; it does not preserve an original +video's shots, actions, or timing. For original-content translation use +`video-translation`; for new dialogue over existing footage use `voice` and optional +`lipsync`. Existing footage cleanup belongs to `talking-head-guide`. + +## Resolve the character + +Reuse the character the user names or attached in ChatCut. Otherwise read the +`` section a ChatCut tool result carries, or call `manage_character action="list"`. +Use the exact `characterId`; call `manage_character action="get"` for its current +images, image URLs, description, and voice. Its `presenter` describes talking-head +preparation state; the submission tool handles that state. + +When several characters could fit, ask the user to choose. Widget Forms can show +image choices using each candidate's main image URL from `manage_character get`, +or ask a concise text question. Keep each ID, name, and image from the same returned +character. An already attached character does not need another selection step. +If there are no saved characters, offer to create one from the user's portrait. + +Use `manage_character` for `list`, `get`, `create`, `update`, and `delete`: + +- Create with `name`, optional `description`, `addImages`, and optional `voice`. +- Each `addImages` entry is `{source, label?}`. `source` is a project asset reference + or an HTTPS URL. The first image is the main image: a clear, front-facing JPG/PNG + showing one person. +- Update the selected `characterId` to change its `name` or `description`, add images + with `addImages`, remove them with `removeImageIds`, or set `mainImageId`. +- Save a voice as `voice: {id, name, provider}` using the selected voice's real + values; `provider` is `custom`, `doubao`, or `elevenlabs`. Set `voice: null` to clear + it. Keep provider values internal. +- Use `delete` only when the user requests deleting that character. + +For a supplied photo and a request to make it speak, create the character with its +name and main image, optionally save its selected voice, then submit with the returned +`characterId`. Creating the library entry alone does not request video generation. +If an existing character has no main image, ask for a clear front-facing portrait +and save it with `manage_character action="update"` and `addImages` before submission. +Request the file as Widget Forms describes for this host, import it as a project +asset, and save that asset reference. + +### A person in an attached video + +When the user says “this person”, `这个形象`, `用她`, or `用他`, resolve that phrase +against the attachment and the requested action. A video is a prospective likeness +source, not a saved character. Prepare a clean representative JPG/PNG still and use +it in `addImages`. The face must be clear, front-facing, and unobstructed by captions, +watermarks, stickers, hands, or inspection time labels. Do not use a contact sheet. + +If the user supplies a new topic or approved script, use that content. The video's +audio or transcript is not automatically the requested speech. If the intended use +is unclear, ask whether they want this person to speak the new script. + +Offer to clone the video's voice when useful, following the Voice Skill and its +separate explicit voice-cloning consent. Skip this when declined or when the source +has unusable audio. Once ready, the selected clone can be saved with +`manage_character action="update"` and `voice`; likeness consent does not authorize +voice cloning. + +## Resolve only what the next action needs + +Reuse the active project, attachments, prior answers, and current character state. +Ask only about the most important remaining blocker; do not run a fixed wizard. +Progress checks do not require resolving another script or output format. + +Before talking-head submission resolve: + +1. **Character:** the user's chosen `characterId` and a usable main image. +2. **Speech:** final text with the character's voice or an explicit `voiceRef`, or + a project audio asset containing the exact desired speech. +3. **Script approval:** preserve user-supplied wording and punctuation. Show an + AI-written draft in full for editing or approval before generation. +4. **Framing:** omit `aspectRatio` to preserve the character image's framing with + `auto`. Explicit choices may use `auto`, `1:1`, `4:5`, `5:4`, `9:16`, or `16:9`. + Do not substitute the project ratio. +5. **Quality:** default to `1080p`; use `720p` when requested or required by the + tool, explaining any fallback. + +Omit `removeBackground` unless the user explicitly wants to keep or remove it; +the default preserves the source image background. Briefly state that default +without making it another question. Use `motionPrompt` for requested motion guidance +within the tool's supported behavior. Do not promise unsupported crop, transparency, +resolution, or motion control; report tool limitations plainly. + +### Speech and voice auditions + +Use `speech: {type:"voice", text}` to use the character's saved voice automatically. +Omitting `voiceRef` uses that binding; do not ask the user to confirm it again. +An explicit voice change overrides it. If there is no saved voice, let the user +choose through `manage_voice action="list"` and save the result with +`manage_character action="update"` and `voice`, or pass the selected `voiceRef` +in `speech` for this generation only. Keep the saved voice object and the returned +submission `voiceRef` distinct; do not invent one from a display name. + +For discovery, audition, and cloning, follow the Voice Skill and use voices available +for avatar video. Keep each voice's returned name, reference, and preview together. +Show up to five suitable playable +voices through Widget Forms, with a localized clone-voice action. A `hasPreview` +flag without a playable `previewUrl` is insufficient. Copy URLs exactly and never +use a character's image or video as an audio preview. + +Keep each voice's playable preview with its choice; Widget Forms decides how this +host lets the user audition it. When the script +also needs confirmation, include the voice selector and script field in one widget. +An upload or choice is not voice-cloning authorization. + +An existing audio asset drives speech with `speech: {type:"audio-asset", assetId}`. +It is the final spoken content, not a cloning sample. Do not require text unless +the user also wants transcription or script review. ### Resolve a script only for text-driven generation @@ -283,8 +156,8 @@ written collaboratively. When drafting: Each explicit text segment must contain 1–5000 characters. Omit explicit segments for an ordinary script so the backend generates one continuous video up to 5000 characters. Use explicit segments only when the user requests meaningful paragraph -or shot boundaries. Longer text is split automatically and may create at most 10 -segments. Never summarize, translate, drop, duplicate, or reorder content to fit a +or shot boundaries. Longer text is split automatically into ordered +segments; the tool rejects a script that exceeds its segment limit. Never summarize, translate, drop, duplicate, or reorder content to fit a limit. Preserve wording and punctuation; non-semantic line whitespace may be normalized automatically. @@ -300,88 +173,55 @@ Do not add implementation details about where or how the work runs, and never ex hidden derived audio assets. Never shorten or summarize approved content to satisfy these limits. -### Resolve output settings from intent and available defaults +### Submit and prepare the appearance -Preserve the selected avatar's original framing by default: omit `aspectRatio` and -let the backend resolve it to `auto`. Never use the current project ratio as the -avatar-video default. If the user explicitly asks for a different ratio, pass that -request instead. +Summarize consequential resolved choices, then call `submit_avatar_video` with +`characterId` and `speech`. Use `segments`, `aspectRatio`, `resolution`, +`removeBackground`, and `motionPrompt` only as needed. The tool shows the live credit +estimate in a confirmation card before anything is charged; do not calculate a separate +price. -For an official catalog avatar, resolve the default from the `hasBackground` field -returned by `manage_avatar action="catalog"`: +On a character's first talking-head use, the tool may return that the appearance +needs preparation, the credit cost, and a request for authorization without creating +a video. Tell the user that returned cost and ask them to explicitly confirm their +right to use this person's likeness using the consent wording below. After the user +agrees, repeat the same submission parameters with `confirmedConsent: true`. +Do not set it based on a photo upload or character selection. -- `hasBackground: true` — pass `removeBackground: false` and preserve the original - scene. -- `hasBackground: false` — pass `removeBackground: true` so the result is a - transparent WebM when transparency is supported. +Preparation takes a few minutes. While it runs the tool says so and tells you how to +continue: tell the user, register the automatic follow-up it names with `track_progress`, +and end the turn. When the follow-up arrives, repeat the same submission parameters +without `confirmedConsent`. Do not report preparation as a completed video. +A paywall means custom talking-head generation requires a paid plan; report it +without claiming that preparation or generation succeeded. -For a saved/custom avatar without catalog metadata, keep `removeBackground: false` as -the default and preserve the source image background. Pass `removeBackground: true` -only when the user explicitly asks for a transparent background or background removal. -Do not turn this into a blocking question. Before submission, briefly tell the user -whether the original scene will be preserved or the result will be transparent. An -explicit user request overrides the catalog default. If transparency is unsupported, -explain the limitation and keep the original background. +After submission returns generation `jobIds`, call `track_progress` with +`target="generation"`, `action="wait"`, and those `jobIds`, unless the user only +asked to queue work. Follow its wait contract: if it registers an automatic follow-up, +end the turn normally; otherwise await its result. Do not add sleeps or busy polling. +Use `action="status"` for a later progress question. Do not quote coarse numeric +provider progress as a completion estimate. -Do not promise crop, background removal, motion control, transparency, resolution, -or prompt-based identity creation until the live capability response confirms it. +Keep long-script segments in their original order. To redo a segment, call +`submit_avatar_video` again for that segment with the same character and the intended +speech/settings; do not regenerate successful segments without a request. Arrange, +remove, or replace generated clips on the timeline as the editing task requires. +Keep the old result until the user accepts a replacement. Generated clips are normal +video assets; place or export them only when the task calls for it. -### Submit as soon as the required constraints are resolved +### Completed videos that do not meet the likeness request -Summarize the resolved avatar, voice or audio source, script state, aspect ratio, and -resolution before the first generation when any choice remains consequential. Then -call `submit_avatar_video` once. +A completed avatar video can still fail the user's likeness requirements. When they ask to correct lips, cheeks or other facial proportions, check which changes the actual tools support before offering another generation. Do not describe the same character, photo and audio as a stricter facial correction unless a supported change plausibly addresses that specific correction; describe what will change and its limits. Explain any limitation plainly. A retry may cost credits and may not improve the likeness; retain the existing result until the user accepts a replacement. Honor an explicit, informed retry request, but do not keep proposing paid repeats of unchanged inputs after they have failed. If the user asks to stop, not export, or seek a credit review, follow that request and use the existing support route without requiring another generation or an export. -Treat long-script output as an ordered batch of avatar-video segments, not as one -audio file or one opaque job. After submitting, tell the user that generation has -started and end the turn. Never call `track_progress`, `manage_avatar action=get-batch`, -or add a Bash sleep in the same turn merely to keep the turn open, including when a -queued follow-up step will eventually need the completed asset. The project library -shows the in-progress generation and receives the completed video assets in the -background. +When a generation tool returns `FEATURE_NOT_INCLUDED` for a `generation.*` feature, use its payer and access details. If the project owner needs Pro, explain that the owner pays and must upgrade; the member's own Pro does not unlock the project. Do not give the member a personal upgrade link or send them to their own billing menu. They can find the owner in the project's existing members view when available. Otherwise share the returned upgrade link, or ChatCut’s avatar menu → Upgrade. Existing credits are preserved and generation still consumes credits after upgrading. This rejection does not charge for generation; existing agent conversation fees are separate, so do not claim the whole conversation was free. Preserve the exact original tool input and project, then stop. Do not automatically search for or substitute stock media; offer alternatives only if the user explicitly asks. -Call `track_progress` in a later user turn only when the user asks for status or resumes -a follow-up step that depends on the completed asset, such as placing it on the timeline -or reviewing the result. When a job is still non-terminal, say that generation is still -running without quoting its numeric progress: provider progress values are coarse -internal stages, not user-facing completion estimates. Use `manage_avatar -action=get-batch` when ordered segment and attempt state is needed. Keep the stable -segment key and original order. - -For a failed segment, offer a targeted retry. Do not resubmit successful segments or -the entire batch without the user's instruction. Use the management tool for supported -segment regeneration, removal, reordering, attempt selection, and acceptance. - -Generated results become normal ChatCut video assets. Add them to the timeline only -when the user requested placement or it is clearly part of the active editing task. -Do not export unless asked. - -## Provider Selection - -Provider selection is an internal ChatCut backend detail. Always call the -ChatCut-native tools and let the backend construct provider requests, resolve private -IDs, enforce capabilities, and persist results. Never call an underlying provider API -or CLI directly from this skill. - -Select only through ChatCut capabilities, catalog results, saved bindings, and the -native tools. User-facing language is limited to concepts such as: - -- **Official avatars** and **My avatars** -- **Official voices**, **cloned voices**, and **project audio** -- **Photo avatar** and **synthetic avatar** - -Provider names, engine or model names, provider-specific avatar or voice IDs, raw -provider URLs, request payloads, and provider error messages are internal. Never show -them in prose, option labels, progress updates, or failure messages. Translate failures -into actionable product language while preserving the real status. - -Do not route around ChatCut's feature gates, quotas, billing checks, safety policy, -catalog visibility, or provider registry. +When the user returns with “continue”, “继续”, “jixu”, or says they upgraded, recheck current payer eligibility and credits for the original input with `estimate_generation` when supported, otherwise the original tool’s precheck. If there is no separate precheck, one retry of the original tool lets its server check current access. If eligible, submit the original input once; if still blocked, explain the current blocker and stop. Do not ask the user to choose between the original generation and stock media again or state the old rejection as current status without checking. If a job was already accepted, track it instead of submitting again. Never switch generation tools, providers or services to bypass a gate. ## Consent and Safety -A real-person photo avatar requires an explicit affirmative confirmation immediately -before creation. Use the user's language and keep the meaning exact. For Chinese: +Before preparing a real person's talking-head appearance, obtain explicit affirmative +consent after disclosing the tool's returned preparation cost. Use the user's language +and preserve the full meaning. For Chinese: > 我确认拥有该肖像,或已获得创建和使用该数字人形象的授权,并承诺不将其用于 > 冒充他人、欺诈或其他违法用途。 @@ -395,57 +235,24 @@ En español: > Confirmo que soy titular de los derechos sobre esta imagen o que tengo permiso para crear y usar este avatar digital, y no lo utilizaré para suplantar identidades, cometer fraude ni realizar actividades ilícitas. -Do not infer consent from an upload, a previous unrelated confirmation, ownership of -the project, or a third party's instruction. Record consent only through the native -tool's consent fields. Synthetic prompt-created identities do not require real-person -likeness consent unless the prompt or references identify a real person. +Do not infer consent from an upload, a prior unrelated confirmation, project +ownership, or a third party's instruction. Pass `confirmedConsent: true` only after +the user's explicit agreement for this preparation. Follow the Voice Skill's separate +consent gate for voice cloning. Refuse non-consensual impersonation, deceptive identity use, fraud, harassment, -sexual exploitation, or attempts to evade safeguards. Do not create a real-person -identity from uncertain authorization. Voice cloning has a separate consent gate; -follow the Voice Skill even when avatar consent was already obtained. +sexual exploitation, or attempts to evade safeguards. Use ChatCut-native tools; +let them enforce access, billing, and preparation state. Do not call provider APIs +directly. Keep provider names, private IDs, URLs, payloads, and raw provider errors +out of user-facing copy; explain the actual failure and next step in product terms. ## Verification -Before reporting success, verify through live tool results: - -1. The chosen identity revision was ready when submitted. -2. The job or batch reached a completed terminal state and returned output asset IDs. -3. Every expected segment key exists exactly once and appears in the confirmed order. -4. Text-backed segment manifests preserve the confirmed wording and punctuation, - allowing only documented non-semantic whitespace normalization. -5. The generated videos exist as usable project or media-library assets. -6. Any requested timeline placement is present and ordered correctly. - -Preview the generated assets or composed timeline when visual verification is available. -Do not claim the face, lip sync, framing, or background looks correct from status fields -alone. If visual inspection is unavailable, say what was structurally verified. - -## Hard Rules - -- Never expose provider, engine, or model identity to the user. -- Never call an underlying provider directly or construct provider payloads in the - conversation layer. -- Never invent an avatar, voice, capability, usage count, saved binding, or asset ID. -- Never generate from a non-ready identity revision. -- Never treat a video as a supported identity source without an explicit live capability; - prepare a representative still for the current photo-avatar contract. -- Never pass an external source URL to avatar creation; import the source into the - target ChatCut project first. -- Never infer likeness consent or voice-cloning consent from an attachment. -- Never clone a video's original voice without the Voice Skill's explicit authorization. -- Never silently rewrite, translate, truncate, reorder, or duplicate confirmed script text. -- Never ask for information already available from the project, attachment, or live tools. -- Never run a fixed wizard or ask for avatar, voice, script, ratio, and resolution in a - preset sequence; resolve only what the current user action still needs. -- Ask one most important missing question at a time; prefer structured visual choices. -- Never show provider-private IDs, URLs, payloads, or raw errors in user-visible output. -- Never bypass ChatCut tools, feature gates, billing, quota, or safety checks. -- Never retry a whole successful batch because one segment failed. -- Never claim visual quality without inspecting the generated result. -- Never place media on the timeline or export it unless the task calls for that action. -- For official catalog avatars, obey the returned `hasBackground` default: preserve a - configured scene background and remove a configured absent background. For - saved/custom avatars, preserve the source background unless the user explicitly asks - for transparency or background removal. Explicit user intent always overrides either - default. +Before reporting success, verify from live results that generation completed and +returned usable video assets, every expected segment is present once in the confirmed +order, and the text preserves approved wording and punctuation. Verify any requested +timeline placement. Preparation or queued jobs alone are not success. + +Preview the generated assets or composed timeline when visual inspection is available. +Do not claim that likeness, lip sync, framing, or background looks correct from status +fields alone. State what was structurally verified if visual inspection is unavailable. diff --git a/qwen/skills/widget-forms/SKILL.md b/qwen/skills/widget-forms/SKILL.md index 03b5201..ca45f13 100644 --- a/qwen/skills/widget-forms/SKILL.md +++ b/qwen/skills/widget-forms/SKILL.md @@ -15,6 +15,6 @@ displayName: "创作偏好" 保留工作流传入的字段含义与选项 ID:文字输入就是文字问题,单选保留标签与真实 ID 的映射;需要明确同意的操作使用未预选的确认项并等待回答。已有明确授权不重复询问。 -声音、MG 和数字人的预览,使用千问实际支持的图片/音频/视频展示方式。没有可用预览界面时说明可在 ChatCut 中预览,提供文字选择并保留稳定 ID;不要假造播放器、拼接 HTML 或将没有查看过的预览描述为已确认。 +声音、MG 和角色的预览,使用千问实际支持的图片/音频/视频展示方式。没有可用预览界面时说明可在 ChatCut 中预览,提供文字选择并保留稳定 ID;不要假造播放器、拼接 HTML 或将没有查看过的预览描述为已确认。 文件通过对话附件或用户提供的可读路径取得,再遵循 `asset-import`;不要把本地路径当作云端 assetId。声线克隆的同意和素材要求遵循 `voice`,附件本身不代表授权。 From 3b03b5ce6985f2fd459f83cca8abc1be556faf46 Mon Sep 17 00:00:00 2001 From: "chatcut-stingray[bot]" <354286180+chatcut-stingray[bot]@users.noreply.github.com> Date: Wed, 18 Nov 2026 11:55:44 +0000 Subject: [PATCH 2/2] Test character migration with merged host adapters --- tests/digital-human.test.mjs | 65 ++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 tests/digital-human.test.mjs diff --git a/tests/digital-human.test.mjs b/tests/digital-human.test.mjs new file mode 100644 index 0000000..4ab9949 --- /dev/null +++ b/tests/digital-human.test.mjs @@ -0,0 +1,65 @@ +import assert from "node:assert/strict"; +import { readFile, readlink } from "node:fs/promises"; +import test from "node:test"; + +const root = new URL("../", import.meta.url); +const read = (path) => readFile(new URL(path, root), "utf8"); + +for (const host of ["codex", "claude", "antigravity", "grok", "kimi", "qwen"]) { + test(`${host} digital-human uses characters without retired avatar actions`, async () => { + const skill = await read(`${host}/skills/digital-human/SKILL.md`); + assert.match(skill, /manage_character/); + assert.match(skill, /characterId/); + assert.match(skill, /confirmedConsent/); + assert.doesNotMatch(skill, /manage_avatar|identityId|create_avatar|avatar_choice/); + }); +} + +test("Claude and Antigravity share the character skill through repository-relative links", async () => { + for (const host of ["claude", "antigravity"]) { + assert.equal( + await readlink(new URL(`${host}/skills/digital-human/SKILL.md`, root)), + "../../../codex/skills/digital-human/SKILL.md", + ); + } +}); + +test("character forms preserve portrait layout, creation and Other without legacy identities", async () => { + const forms = await read("codex/skills/widget-forms/SKILL.md"); + const codex = forms.split("## Codex\n")[1].split("## Claude Code\n")[0]; + const characters = codex.split("Map a character choice")[1].split("Map a caption style choice")[0]; + assert.match(characters, /variant:"visual"/); + assert.match(characters, /id:"avatar"/); + assert.match(characters, /characterId/); + assert.match(characters, /manage_character get/); + assert.match(characters, /create_character/); + assert.match(characters, /__other__/); + assert.match(characters, /otherPlaceholder/); + assert.match(characters, /asset-import/); + assert.doesNotMatch(forms, /identityId|create_avatar|Digital Humans|AI Avatars/); +}); + +test("Claude retains private-media fallbacks and voice Other alongside character choices", async () => { + const forms = (await read("codex/skills/widget-forms/SKILL.md")).split("## Claude Code\n")[1]; + assert.match(forms, /sampleMediaKey/); + assert.match(forms, /Ready custom voices[\s\S]*label-only/); + assert.match(forms, /data-other/); + assert.match(forms, /elicit-other/); + assert.match(forms, /characterId/); + assert.match(forms, /label-only selectable cards/); + assert.match(forms, /My Assets/); + assert.match(forms, /Never embed or print image or\s+private preview URLs/); + const skill = await read("codex/skills/digital-human/SKILL.md"); + assert.match(skill, /Other returns to discovery/); + assert.match(skill, /do not discard a ready custom voice/); +}); + +test("caption preview and custom-style application guidance survives character migration", async () => { + const forms = await read("codex/skills/widget-forms/SKILL.md"); + assert.match(forms, /publishedTemplates/); + assert.match(forms, /captionTarget/); + assert.match(forms, /catalogTemplateId/); + assert.match(forms, /project-bound custom styles/); + assert.match(forms, /edit_captions action:"style"/); + assert.match(forms, /Claude Code does not render `captionPreview`/); +});