FreeVideo 0.4.38

MANIFEST.in

@@ -94,5 +94,8 @@ include web/assets/tour/ca-record-wave.webp
 include web/assets/tour/cc-speech.mp4
 include web/assets/tour/cc-speech.webp
 include web/assets/tour/cc-speech-wave.webp
+include web/assets/tour/cd-narration.mp4
+include web/assets/tour/cd-narration.webp
+include web/assets/tour/cd-narration-wave.webp
 include web/assets/tour/c10-source.webp
 include web/assets/tour/fox-sheet.webp

__init__.py

@@ -53,6 +53,8 @@ async def comfy_entrypoint():
     editor_export_routes()
     from .freevideo_engine.comfy_capabilities import register as capabilities_routes
     capabilities_routes()
+    from .freevideo_engine.comfy_monitor import register as monitor_routes
+    monitor_routes()
     from .freevideo_engine.subtitle_ears import register as subtitle_ears_routes
     subtitle_ears_routes()
     from .freevideo_engine.editor_music import register as editor_music_routes

docs/community-results.md

@@ -124,6 +124,20 @@ Thanks to the community members who sent diagnostic reports. FreeVideo already r
 | 5060 | 8 GB | 48 GB | 768×1344 | 14 s | Medium | 0.3.25 | 3474 s | 0.3.25 | Already optimal |
 | 3060 | 12 GB | 32 GB | 896×1120 | 10 s | Light | 0.3.25 | 1045 s | 0.4.0 | 1025 s (−20 s) |
 | 3060 Laptop | 6 GB | 40 GB | 960×544 | 15 s | Light⁵ | 0.4.24 | 713 s | 0.4.24 | Thermal‑limited¹ |
+| 5060 | 8 GB | 64 GB | 544×960 | 11.5 s | Light | 0.4.26 | 743 s¹³ | 0.4.26 | Already optimal¹² |
+| 3050 Laptop | 6 GB | 16 GB | 1184×672 | 10 s | High | 0.4.23 | 5773 s | 0.4.31 | 4300 s (−1470 s) |
+| 5080 | 16 GB | 32 GB | 1920×1088 | 10 s | Light | 0.4.18 | 1850 s | 0.4.21 | 860 s (−990 s) |
+| 4060 Ti | 8 GB | 32 GB | 960×544 | 6 s | Light | 0.4.27 | 241 s | 0.4.28 | 190 s (−50 s) |
+| 3060 | 12 GB | 32 GB | 672×1184 | 3 s | Light | 0.4.28 | 469 s | 0.4.28 | 430 s (−40 s) |
+| 5080 Laptop | 16 GB | 64 GB | 1344×768 | 10 s | Light | 0.4.29 | 647 s | 0.4.29 | Already optimal |
+| 4060 Laptop | 8 GB | 16 GB | 1344×768 | 10 s | Light | 0.4.30 | 1056 s | 0.4.30 | Already optimal |
+| 5070 Laptop | 8 GB | 32 GB | 768×1344 | 2.3 s | Max¹² | 0.4.30 | 2195 s | 0.4.30 | Compatibility level 3¹⁴ |
+| 5080 Laptop | 16 GB | 64 GB | 2496×832 | 8 s | Light⁵ | 0.4.30 | 777 s | 0.4.30 | Already optimal |
+| 4060 Laptop | 8 GB | 32 GB | 1344×768 | 9.4 s | Light | 0.4.31 | 806 s | 0.4.31 | Already optimal |
+| 4060 Laptop | 8 GB | 32 GB | 1344×768 | 10 s | Light | 0.4.32 | 802 s | 0.4.32 | Already optimal |
+| 3050 Laptop | 8 GB | 40 GB | 960×544 | 5 s | Light | 0.4.33 | 890 s | 0.4.33 | Already optimal |
+| 3070 Laptop | 8 GB | 64 GB | 960×544 | 5 s | Light | 0.4.33 | 260 s | 0.4.33 | Already optimal |
+| 3070 | 8 GB | 64 GB | 736×736 | 10 s | Light | 0.4.35 | 471 s | 0.4.35 | Already optimal |
 
 - Time: end to end, from submission until the video is saved.
 - After: the estimated time, derived from the report's stage times and from complete planned runs on our test machines (RTX 3090 and RTX 5060 Ti on Windows, RTX PRO 6000 on Linux) under the reporting machine's VRAM and RAM limits, not from reruns on that machine.
@@ -141,3 +155,4 @@ Thanks to the community members who sent diagnostic reports. FreeVideo already r
 - ¹¹ With 16 GB of RAM, the text encoder and part of the video model cannot stay in memory and are read from the drive; text encoding and the first pass, which wait on these reads, took about 460 s of the 887 s.
 - ¹² Task with a reference: every step also processes the reference video, images or audio, so each step takes longer than a plain request of the same size, up to about twice as long with a long reference.
 - ¹³ Preview (first pass only).
+- ¹⁴ Compatibility level 3 was set manually; it is the slowest level and made each step about 2.4 times slower than the automatic setting for this request, and the GPU core clock also stayed at 900 MHz against the 1815 MHz it reached. With the level off, the same request takes about 1100 s.

docs/zh-CN/community-results.md

@@ -124,6 +124,20 @@
 | 5060 | 8 GB | 48 GB | 768×1344 | 14 s | 标⁠准 | 0.3.25 | 3474 s | 0.3.25 | 已⁠是⁠最⁠优 |
 | 3060 | 12 GB | 32 GB | 896×1120 | 10 s | 轻⁠量 | 0.3.25 | 1045 s | 0.4.0 | 1025 s⁠（−20 s⁠） |
 | 3060 笔⁠记⁠本 | 6 GB | 40 GB | 960×544 | 15 s | 轻⁠量⁵ | 0.4.24 | 713 s | 0.4.24 | 散⁠热⁠受⁠限¹ |
+| 5060 | 8 GB | 64 GB | 544×960 | 11.5 s | 轻⁠量 | 0.4.26 | 743 s¹³ | 0.4.26 | 已⁠是⁠最⁠优¹² |
+| 3050 笔⁠记⁠本 | 6 GB | 16 GB | 1184×672 | 10 s | 精⁠细 | 0.4.23 | 5773 s | 0.4.31 | 4300 s⁠（−1470 s⁠） |
+| 5080 | 16 GB | 32 GB | 1920×1088 | 10 s | 轻⁠量 | 0.4.18 | 1850 s | 0.4.21 | 860 s⁠（−990 s⁠） |
+| 4060 Ti | 8 GB | 32 GB | 960×544 | 6 s | 轻⁠量 | 0.4.27 | 241 s | 0.4.28 | 190 s⁠（−50 s⁠） |
+| 3060 | 12 GB | 32 GB | 672×1184 | 3 s | 轻⁠量 | 0.4.28 | 469 s | 0.4.28 | 430 s⁠（−40 s⁠） |
+| 5080 笔⁠记⁠本 | 16 GB | 64 GB | 1344×768 | 10 s | 轻⁠量 | 0.4.29 | 647 s | 0.4.29 | 已⁠是⁠最⁠优 |
+| 4060 笔⁠记⁠本 | 8 GB | 16 GB | 1344×768 | 10 s | 轻⁠量 | 0.4.30 | 1056 s | 0.4.30 | 已⁠是⁠最⁠优 |
+| 5070 笔⁠记⁠本 | 8 GB | 32 GB | 768×1344 | 2.3 s | 极⁠致¹² | 0.4.30 | 2195 s | 0.4.30 | 兼⁠容⁠等⁠级⁠3¹⁴ |
+| 5080 笔⁠记⁠本 | 16 GB | 64 GB | 2496×832 | 8 s | 轻⁠量⁵ | 0.4.30 | 777 s | 0.4.30 | 已⁠是⁠最⁠优 |
+| 4060 笔⁠记⁠本 | 8 GB | 32 GB | 1344×768 | 9.4 s | 轻⁠量 | 0.4.31 | 806 s | 0.4.31 | 已⁠是⁠最⁠优 |
+| 4060 笔⁠记⁠本 | 8 GB | 32 GB | 1344×768 | 10 s | 轻⁠量 | 0.4.32 | 802 s | 0.4.32 | 已⁠是⁠最⁠优 |
+| 3050 笔⁠记⁠本 | 8 GB | 40 GB | 960×544 | 5 s | 轻⁠量 | 0.4.33 | 890 s | 0.4.33 | 已⁠是⁠最⁠优 |
+| 3070 笔⁠记⁠本 | 8 GB | 64 GB | 960×544 | 5 s | 轻⁠量 | 0.4.33 | 260 s | 0.4.33 | 已⁠是⁠最⁠优 |
+| 3070 | 8 GB | 64 GB | 736×736 | 10 s | 轻⁠量 | 0.4.35 | 471 s | 0.4.35 | 已⁠是⁠最⁠优 |
 
 - 用时：从提交到视频保存完成的端到端用时。
 - 优化后：预估用时，根据报告中的分段用时，结合测试机（Windows RTX 3090 和 RTX 5060 Ti、Linux RTX PRO 6000）在报告机器的显存和内存上限下运行完整规划的实测推算，不是在报告机器上重跑。
@@ -141,3 +155,4 @@
 - ¹¹ 16 GB 内存留不下文本编码器和视频模型的一部分，需要从硬盘读取；887 秒里约 460 秒是等待这些读取的文本编码和第一遍。
 - ¹² 带参考的任务：每一步还要处理参考视频、参考图片或参考音频，所以每步耗时比相同尺寸的普通请求长；参考较长时约为两倍。
 - ¹³ 预览（只做第一遍）。
+- ¹⁴ 兼容等级 3 为手动设置，是最慢的等级，对这个请求每步比自动设置慢约 2.4 倍；显卡核心时钟同时固定在 900 MHz（本次运行曾达到 1815 MHz）。关闭该等级后，同样的请求约需 1100 秒。

freevideo_engine/activation_staging.py

@@ -1,21 +1,47 @@
 """Contiguous D2H head groups with double-buffered CPU projection tiles."""
 import collections
+import contextlib
 import torch
 
+# The first copies through a freshly locked buffer, and the first tiles of its
+# first projection, are followed by a wait for the device. A deferred CUDA
+# "invalid argument" from a queued locked copy is otherwise reported by
+# whichever later call happens to run (reports on the second pass's first layer
+# name three different statements), outside the handler that continues in
+# pageable memory. Waiting here raises it inside that handler, with the stage
+# named. A handful of waits per buffer set, none afterwards.
+CHECKED_COPIES = 3
+
 # Every refused locked-page copy of this process, residual stream and head
 # readouts alike, so a report shows whether the fallback fired, which copy was
 # refused and what CUDA said. The newest entries are kept; the count is exact.
 _REFUSED = collections.deque(maxlen=32)
 _refused_total = 0
 
 
-def record_refusal(source, copy, error):
+def record_refusal(source, copy, error, checked=False):
     global _refused_total
     lines = str(error).strip().splitlines()
-    _REFUSED.append(dict(source=source, copy=copy, error=(lines[0] if lines else '')[:200]))
+    row = dict(source=source, copy=copy, error=(lines[0] if lines else '')[:200])
+    if checked:
+        row['checked'] = True  # raised by the wait after one of the first copies, not by the copy call
+    _REFUSED.append(row)
     _refused_total += 1
 
 
+@contextlib.contextmanager
+def checked(active, stage, **fields):
+    """Wait for the device after `stage` while `active`; name the stage on a CUDA error."""
+    try:
+        yield
+        if active and torch.cuda.is_initialized():
+            torch.cuda.synchronize()
+    except RuntimeError as error:
+        if active and not isinstance(error, torch.cuda.OutOfMemoryError) and hasattr(error, 'add_note'):
+            error.add_note('checked stage: %s %s' % (stage, ' '.join('%s=%s' % item for item in fields.items())))
+        raise
+
+
 def refusals_mark():
     return _refused_total
 
@@ -38,36 +64,55 @@ class LockedCopies:
     and the buffers stay locked.
     """
 
-    def __init__(self):
+    def __init__(self, checked_copies=CHECKED_COPIES):
         self.locked = True
         self.fallbacks = 0
+        # One budget per direction: the host-to-device copies (the report's refused statement) are not
+        # used up by the device-to-host copies that come first in a layer.
+        self.unchecked = dict(to_host=checked_copies, to_device=checked_copies)
 
-    def _leave(self, device, copy, error):
-        record_refusal('readout', copy, error)
+    def _leave(self, device, copy, error, waited=False):
+        record_refusal('readout', copy, error, checked=waited)
         if device.type == 'cuda':
             torch.cuda.synchronize(device)
         self.locked = False
         self.fallbacks += 1
 
+    def _wait(self, device, copy):
+        # Inside the handler of the copy that was just queued.
+        if self.unchecked[copy] > 0:
+            self.unchecked[copy] -= 1
+            if device.type == 'cuda':
+                torch.cuda.synchronize(device)
+                return True
+        return False
+
     def to_host(self, destination, source):
         if self.locked:
+            waited = False
             try:
                 destination.copy_(source, non_blocking=True)
+                waited = True  # from here on an error comes from the wait
+                self._wait(source.device, 'to_host')
                 return
             except torch.cuda.OutOfMemoryError:
                 raise
             except RuntimeError as error:
-                self._leave(source.device, 'to_host', error)
+                self._leave(source.device, 'to_host', error, waited)
         destination.copy_(source.to('cpu'))
 
     def to_device(self, source, device):
         if self.locked:
+            waited = False
             try:
-                return source.to(device, non_blocking=True)
+                value = source.to(device, non_blocking=True)
+                waited = True  # from here on an error comes from the wait
+                self._wait(torch.device(device), 'to_device')
+                return value
             except torch.cuda.OutOfMemoryError:
                 raise
             except RuntimeError as error:
-                self._leave(torch.device(device), 'to_device', error)
+                self._leave(torch.device(device), 'to_device', error, waited)
         return source.clone().to(device)
 
 

freevideo_engine/agent/guide.py

@@ -262,8 +262,9 @@
     'otherwise make its reference pictures with POST /cast/{id}/generate. '
     'Before shooting, tell the user the casting in one sentence. Later shots use the card\'s current pictures. '
     'Keep cards you made so later films can reuse them.',
-    'Pictures. When the user wants a picture, or a first frame or reference picture that does not exist yet, make '
-    'it with freevideo_generate_image; to change a picture, pass it in images. The result is an input file: use '
+    'Pictures. When the user wants a picture, make it with freevideo_generate_image and stop there: do not read the video '
+    'prompt guide, plan or start a video, and tell the user where the picture is. The same tool makes a first frame or '
+    'reference picture that does not exist yet; to change a picture, pass it in images. The result is an input file: use '
     'result.file as a first frame, as a reference, or in images again. When where people stand in the shot matters, '
     'make the first frame with scene_image and people (cast card IDs or pictures, x, height, facing left/right, '
     'framing full/half), then generate the video from result.file. Use dimensions divisible by 32. '
@@ -306,7 +307,8 @@
     '先选角，再拍摄。先搜索角色库，使用用户指定的卡片或明确符合要求的卡片；拿不准时询问用户。'
     '只有找不到合适卡片时才新建：用户提供了照片就用照片，否则通过 POST /cast/{id}/generate 生成参考图。'
     '开拍前用一句话告诉用户选角结果。后续镜头使用卡片当前的图片。保留你创建的卡片，方便以后拍片复用。',
-    '图片。用户要一张图，或者缺一张首帧、参考图时，用 freevideo_generate_image 生成；要改一张图，把它放进 images。'
+    '图片。用户要一张图时，用 freevideo_generate_image 生成，到此为止：不读视频提示词指南，不做视频规划，也不生成视频，告诉用户图片在哪里。'
+    '缺一张首帧、参考图时也用它生成；要改一张图，把它放进 images。'
     '结果是一个输入文件：result.file 可以当首帧、当参考图，也可以再放进 images。镜头里人物站位要紧时，'
     '先用 people（角色图和各自的位置）做出首帧，再用它生成视频。',
     '默认切镜头。能切镜头的地方就切：新镜头直接用对应的参考图（人物卡、场景图）生成，不接上一段视频。'
@@ -458,7 +460,9 @@ def connection_discovery(root, base, lang=None):
 def introduction(zh, offers):
     """What the agent tells the user first: only what this installation is known to carry through."""
     rows = ['按描述生成视频，可以给首帧图片或参考图' if zh else
-            'Make a video from a description, optionally from a first-frame picture or reference pictures']
+            'Make a video from a description, optionally from a first-frame picture or reference pictures',
+            '按描述生成图片，也可以修改、合成已有的图片；只要图片时不会生成视频' if zh else
+            'Make a picture from a description, or change and combine pictures; a request for a picture alone makes no video']
     if offers.get('editing'):
         rows.append('剪辑：把片段排上时间线，分割、裁剪、删除，再导出成片' if zh else
                     'Edit: put clips on a timeline, split, trim and remove them, then export the finished video')
@@ -489,6 +493,7 @@ def connection_guide(root, base, lang=None, offers=None, resumed=False):
         '# client is the name the user sees as connected: put your own, for example "Claude Code" or "Codex".',
         'call("POST", "/connect/hello", {"client": "Local Agent", "via": "http"})',
         'print(call("GET", ""))',
+        '# A request for a picture alone: POST /jobs {"kind": "image", "prompt": ..., "label": ..., "ref": ...} and no video job; the result is result.file.',
         '# Only after the user asks for a video:',
         '# label is the short name the user sees on the job, in their language: what the video shows, 2 to 8 characters.',
         '# seconds and aspect are what the user asked for (10 seconds and 16:9 when they said nothing): send them the first time.',
@@ -557,7 +562,7 @@ def connection_guide(root, base, lang=None, offers=None, resumed=False):
                      'without id). Its "session" is the conversation the user is in (it changes when the user opens or returns to another one; always post to the session the latest wait named); POST ' + base + PREFIX + '/sessions/{session}/events '
                      'with {"type":"message","text":"..."} (MCP: freevideo_session_post). Say only one short sentence '
                      'there first, at most 12 words, no list and no examples (the page already shows examples), for example '
-                     '"Connected. Describe the video you want.", in the language the user writes to you in. Give the introduction in your own window as usual, but it '
+                     '"Connected. Describe the video or picture you want.", in the language the user writes to you in. Give the introduction in your own window as usual, but it '
                      'does not end your turn: right after it, call the wait. Then stay in this conversation: call the wait again; it answers within about 50 seconds, and '
                      '"again": true means the user has not spoken yet, so call it again at once, time after time, with no '
                      'limit. Nothing new is not a reason to finish: do not end your reply and do not stop to ask the '