Skip to content

feat(processing): add read-only artifact processing status inspection (part of #2130) - #2568

Merged
AlexStocks merged 9 commits into
oceanbase:masterfrom
guamiao:feat/2130-processing-status
Nov 12, 2026
Merged

AlexStocks merged 9 commits into
oceanbase:masterfrom
guamiao:feat/2130-processing-status

Conversation

@guamiao

@guamiao guamiao commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Backend inspection layer for issue #2130: a read-only processing status inspector that provides a unified processing state snapshot for every artifact family, reusing existing durable data. HTTP endpoint wiring will follow in a separate PR.

Changes

File Lines
src/powercontext/builtin/processing_status.py +400
tests/builtin/test_processing_status.py +340

Verification

uv run ruff check -> All checks passed
uv run pytest tests/builtin/test_processing_status.py -> 27 passed
uv run pytest tests/builtin/test_processing_diagnostics.py -> 104 passed (no regression)

Related

Review fixes and CI evidence

Review fixes are included in bf3b5d95 and have individual replies in the original review threads. Validation results, failed-job links and the limits of the CI investigation are documented in this evidence summary. CI is not green; no definitive claim that the remaining failures are unrelated is made.

…oceanbase#2130)

Introduce ProcessingStatusInspector, a read-only inspection layer that
provides a unified processing status snapshot for every artifact family,
reusing existing durable data without new tables, writes, or model calls.

- ProcessingStatusInspector queries processing intents (memory/experience/
  profile), Topic Memory recovery status, and Dream records (skill/handoff/
  prompt) to produce a bounded ArtifactProcessingStatus per scope+family.
- ProcessingStatusState (idle/pending/blocked/unknown) with explicit
  UNKNOWN for unavailable or inapplicable paths.
- ProcessingStatusBlock links durable blocks to safe ProcessingDiagnostics
  via the existing classify_block_code vocabulary.
- DreamOperationStatus surfaces queued/running/succeeded/failed runs
  filtered by family operation bindings.
- ProcessingStatusApplication + ScopedProcessingStatusApplication follow
  the project's for_scope pattern, ready for server HTTP integration.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
guamiao and others added 2 commits November 10, 2026 00:12
- import validate_scope_id from powercontext.builtin.sources (not limits)
- apply ruff format to new files

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@guamiao
guamiao force-pushed the feat/2130-processing-status branch from 5132884 to dcc582e Compare November 9, 2026 16:42
return datetime.now(UTC)


__all__ = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 新模块没有任何生产入口,Implements issue #2130 与可交付物不一致

git grep 本 head 全仓,ProcessingStatusApplication / ScopedProcessingStatusApplication / ProcessingStatusInspector 只出现在本文件与 tests/builtin/test_processing_status.py:没有 builtin/__init__ 导出、没有 server/app.py 路由、没有 CLI 命令,也没有 Dashboard 引用。issue #2130 的交付物是「每个 artifact 页面上的只读 Processing status 分节」,本 PR 描述同时写着 Implements issue #2130 和 ready for HTTP integration,两者不能同时成立。

影响:合并后这 399 行不在任何生产路径上可执行,只是新增了一份与 src/powercontext/builtin/runtime/processing_observation.py::observe_processing 同域的第二实现(该实现已在 master 上从同一批 intents 行推导 pending/dirty/acknowledged/idle,本模块旁路 MySQL/OceanBase 分支并丢失本地 scheduler 观测)。按 AGENTS.md「domain rules 归属 runtime/Artifact service」与「callers 契约相同时共享代码」,两份状态推导并存会在接入 Dashboard 时立刻分叉。

建议:本 PR 直接接到 Dashboard 或 HTTP 入口并给出端到端证据;若确实要分层提交,请改掉正文的 Implements issue #2130,并把 ProcessingStatusApplication 与既有 observe_processing 的职责边界写清楚。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed review. I will update the PR title and description to clarify this is the backend inspection layer (not the full Dashboard implementation), and remove the 'Implements issue #2130' claim. The HTTP wiring will be a follow-up PR.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复验结论:本条未解决。head 仍为 f4b5455b(与本线程回复时的 commit_id 相同),回复中承诺的标题/正文修正与代码入口均未落地。

1. 承诺的元信息修正未执行

  • 标题仍为 feat(processing): add read-only artifact processing status inspection (#2130)
  • 正文首行仍为 Implements issue #2130: a read-only processing status inspection layer ...
  • 正文仍在 ProcessingStatusApplication ... ready for HTTP integration 处声称可接线
  • PR updated_at = 2026-11-10T04:39:47Z,晚于回复(02:38:42Z),但上述文本未变 ⇒ 不是"还没推送",是没改

2. 生产入口仍然为零
git grep 本 head 全仓,ProcessingStatusInspector 仅 2 个文件命中:

src/powercontext/builtin/processing_status.py:116,369,388
tests/builtin/test_processing_status.py:26,85,89,...

ArtifactProcessingStatus 在本文件外 0 命中;src/powercontext/server/ 下无任何引用。第 388 行是模块内 ScopedProcessingStatusApplication.status() 自建实例,没有 server/app.py 路由、没有 builtin/__init__ 导出、没有 CLI/Dashboard 调用方。

3. 另两条 P2 已确认修复(本 head 实跑)

  • 第 200 行谓词含 progress.requested_generation > progress.handled_generation
  • 第 248/251 行改为 classify_block_code("topic_memory", error_code) 与 stage="topic_memory"
  • tests/builtin/test_processing_status.py:27 passed
  • 变异验证有效:还原 getattr(raw_block, "stage", None) 或删掉 requested>handled 分支,用例均转为失败

收口选项(二选一)

  • A:按回复所述改标题与正文——去掉 (#2130)、把 Implements issue #2130 改为 Part of issue #2130,并删去 ready for HTTP integration 之类会让人以为已接线的表述;PR 定位写成"backend inspection layer,HTTP 接线在后续 PR"
  • B:在本 PR 内补 server/app.py 路由(按 for_scope 模式挂只读 GET),使这 399 行进入生产路径

A 与 B 都不做的话,合并后该模块不可达,"Implements #2130" 与可交付物仍然不一致。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough re-review. The PR title and body have been updated per option A:

The other two P2 issues (stage hardcoding, requested>handled predicate) were already fixed in earlier commits and confirmed by your re-verification. Tests: 27 passed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following option A, the title and description explicitly identify this as a backend slice of #2130. HTTP/OpenAPI and Dashboard presentation are follow-up work; this PR does not complete or close #2130. In bf3b5d9, the inspector also reuses project_processing_observation from the existing observation implementation instead of maintaining a separate intent-state algorithm. The bilingual coverage matrix explains each fact's source and the boundary between durable invocation facts, Source positions, local executor observations, persistent blocks, and Dream history.

raw_block = getattr(raw, "block", None)
if raw_block is not None:
block_id = getattr(raw_block, "block_id", None) or ""
stage = getattr(raw_block, "stage", None) or None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] stage 属性在真实 block 模型上不存在,block.diagnostics 在生产路径恒为 None

TopicMemoryWorkBlock(src/powercontext/builtin/artifacts/topic_memory/processing.py)的字段是 attempt_id/attempts/binding_name/block_id/blocked_at/budget_source_through/error_code/last_error_code/legacy_source_through/requests/retried_at/retry_request_generation/scope_id/source_after/source_through/target_source_through/tokens,没有 stage。因此 getattr(raw_block, "stage", None) 恒为 None,而 processing_diagnostics.classify_block_code(stage, code) 只在 stage == "topic_memory" 时返回 ProcessingDiagnostics,其余一律返回 None ⇒ 本模块承诺的「safe diagnostic details」永远不会出现。

实测(真实模型,非 Mock):用真实的 TopicMemoryProcessingStatus + TopicMemoryWorkBlock(error_code="window_attempt_limit") 调 ProcessingStatusInspector.status(..., "topic-memory"),输出 state: blocked / stage: None / error_code: window_attempt_limit / diagnostics: None。tests/builtin/test_processing_status.py::TestProcessingStatusInspector::test_topic_memory_maps_blocked_state_with_diagnostics 之所以绿,是因为它 Mock(block_id=..., stage="topic_memory", error_code=...) 造了一个生产中不存在的字段,属于断言 Mock 自身行为。

建议:按 block 自己的词汇决定 stage(例如 topic-memory 分支直接传常量 "topic_memory"),或给 TopicMemoryWorkBlock 增加真实字段;并把该用例改成用真实模型构造,否则无法检出这条路径退化。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b084ae5: TopicMemoryWorkBlock has no stage field, so the code now hardcodes before calling classify_block_code. This ensures diagnostics are actually produced for topic-memory blocks. The test mock was also updated to not include a stage field, reflecting the real model.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed in the current bf3b5d9 implementation: Topic Memory block diagnostics explicitly use classify_block_code("topic_memory", error_code) and do not read a nonexistent stage attribute. The replacement regression now uses real SQLite budget/block records, covers both persisted blocks and passive legacy projections, and asserts that diagnostics are populated. It no longer relies on a Mock with a fictitious stage field.

requested_generation=intent.requested_generation,
handled_generation=intent.handled_generation,
)
if progress.pending_inputs or progress.outstanding_work:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 状态谓词丢掉了 requested_generation > handled_generation,已受理但未处理的工作会报 idle

source_cursor / source_head 被赋成 clean_generation / dirty_generation(第 192-193 行),于是 pending_inputs 与 outstanding_work 恒等,两个字段退化为同一个判据;requested_generation / handled_generation 只被塞进 progress,不参与第 199 行的状态判定。而 master 的 runtime/processing_observation.py::observe_processing 对同一行数据的判定顺序是 requested > handled → pending、dirty > clean → dirty、handled → acknowledged、否则 idle。

实测(真实 intents 行,dirty=4 clean=4 requested=4 handled=3):本模块 status(...) == idle、pending_inputs == False、outstanding_work == False,同一行在 observe_processing 的语义下应是 pending。触发条件是手动 retry / 显式 request 已受理但 Worker 尚未 ack,此时界面会显示「空闲」,与 issue #2130 要求的「已接收但未生成」正相反。

建议:按既有语义实现状态判定(pending → dirty → acknowledged → idle),或复用 observe_processing 的结果做投影;并补一条 requested > handled 且 dirty == clean 的用例,当前 24 条用例里没有覆盖这个组合。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b084ae5: the state predicate now includes so acknowledged-but-unprocessed work correctly reports PENDING (matching observe_processing semantics). Added a test case for dirty==clean, requested>handled that would have failed before this fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in bf3b5d9 by sharing the existing durable-intent projection. A request with requested_generation > handled_generation remains pending even without pending Source input; test_unhandled_request_is_pending_without_source_work verifies this through the public application with real SQLite records. Invocation counters and actual Source cursor/head are now separate facts rather than interchangeable progress values.

guamiao and others added 5 commits November 10, 2026 10:25
…tage

- State now reports PENDING when requested_generation > handled_generation,
  matching observe_processing semantics (dirty==clean but work acknowledged).
- TopicMemoryWorkBlock has no stage field; hardcode "topic_memory" for
  classify_block_code so diagnostics are actually produced.
- Add test for requested>handled with dirty==clean.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@AlexStocks AlexStocks added the 1.6.0 v1.6.0 release tracking label Nov 10, 2026
@guamiao guamiao changed the title feat(processing): add read-only artifact processing status inspection (#2130) feat(processing): add read-only artifact processing status inspection (part of #2130) Nov 10, 2026

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The backend-only scope is now clear. Four correctness issues remain in the inspection contract: reads can persist Topic Memory blocks, Source positions are derived from unrelated counters, and Dream snapshots can hide pending work or present historical failures as current blocks. Details and reproductions are inline.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@guamiao

guamiao commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

@AlexStocks @hidb4ai The current head bf3b5d9 addresses the review findings; each original inline thread now has a specific reply describing the code and regression coverage. This remains a backend slice of #2130: HTTP/OpenAPI and Dashboard delivery are follow-up work. Please re-review the current head when convenient.

Validation: Linux make check passed; the final status/observation/statistics selection passed 61 tests with 1 skipped. CI Python 3.11/3.14 unit tests, Python 3.12 E2E, worker observability and SQLite acceptance passed. External-backend acceptance is not claimed from SQLite tests.

Remaining CI evidence

  • DSH job 130373596045 passed both 469-test JS runs, the 8 JS E2E tests and npm archive verification. Its runtime suite failed the same setup/include-tree test as feat(processing): classify Dream, Handoff, and Evidence processing failures with safe diagnostics (#2126) #2539 (18 passed, 1 failed): the fixture CLI setup subprocess exited 1 without its underlying output appearing in the job log. A local WSL attempt failed earlier in native DSH package resolution, so it does not reproduce this CI error. The master DSH failure was a different JS assertion and cannot establish the same baseline defect. The CI root cause remains open.
  • Ubuntu installation job 130373597458 timed out after 300 seconds during the initial bootstrap-cn install of version 1.2.0. Its uploaded diagnostics show a 404 from the USTC uv bootstrap URL followed by successful official-source fallback, about 13.78 seconds installing Python from NJU, and about 76 seconds resolving plus 183 seconds preparing dependencies through the Tsinghua PyPI mirror. The global install path passed and recorded much shorter resolution/preparation times. This supports mirror latency as a contributor, but the precise command executing at timeout is not established by the untimestamped install log.

This PR does not change DSH setup, the installers, dependency declarations/lockfiles or their CI configuration. No causal link to these changes has been demonstrated; I am not claiming definitive independence or that CI is green. CI result also fails as the aggregate of these job failures.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed bf3b5d9 against the four previous reproductions; all four are fixed. Inspection stays passive, Source positions are read independently of invocation counters, and Dream pending work is separated from historical outcomes. The backend-only scope is explicit, with no new database tables or HTTP/MCP/CLI endpoints.

Approving this code change. The DSH and Ubuntu installation checks remain red and need to be cleared separately. Local behavioral validation covered SQLite, not OceanBase/SeekDB acceptance.

@AlexStocks
AlexStocks merged commit b1768a6 into oceanbase:master Nov 12, 2026
66 of 69 checks passed
jackie-cqz added a commit to jackie-cqz/powercontext that referenced this pull request Nov 13, 2026
Resolve the conflict with the standalone seekdb maintenance identity (oceanbase#2585)
and the 30 other commits that landed since the previous merge, including the
processing status projection (oceanbase#2568) and the structured task stop reasons
(oceanbase#2580).

- `release/upgrade-support.json`: keep the rationale sentence added upstream
  for the seekdb maintenance identity, append the sentence about this branch's
  code-organization changes, and recompute the persistence digest after
  formatting. The generated table and index DDL for all 76 tables is
  identical to master on the SQLite and MySQL dialects.

Follow the moves and removals into code that arrived with the merge:

- `tests/builtin/runtime/test_context_reuse.py` imports access control from
  `powercontext.builtin.authz`.
- The `scheduler_path` argument is dropped from the new calls in
  `test_artifact_merge_experience.py` and `test_dashboard.py`.
- The startup-deadline test in `test_real_experience_skill_harness.py` stops
  passing a scheduler database path to `_start_server` and
  `_start_configured_server`, which no longer take one. It passed the path
  positionally, so a search for the `scheduler_path=` keyword could not find it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.6.0 v1.6.0 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants