[Zxf-xufeng]

Which issue or RFC does this PR close?

Consolidates the database CI reliability work from #2515. No issue is automatically closed.

Rationale for this change

Running seekDB and OceanBase matrices on every PR and master push creates a backlog of expensive database jobs. Keep SQLite feedback on ordinary changes and run native database acceptance once per night, while preserving mandatory results for every selected workload.

What changes are included in this PR?

• PRs and master pushes run selected SQLite replay, schema capture and all four SQLite installed-release upgrade configurations. Shared inputs, missing change baselines and ci:full cannot enable native database jobs on these events.
• Main's daily 0 18 * * * schedule (02:00 Asia/Shanghai; 18:00 UTC) runs full database acceptance, including seekDB startup/migrations, OceanBase recovery/FTS/migration shards, native acceptance, all 12 upgrade configurations, all backend schema replays/captures and short lock probes. Explicit manual runs remain available; extended lock observation remains manual.
• Keep scope selection, workflow matrices and required-result gates consistent. Deferred native jobs may skip on PRs; missing or skipped selected nightly work still fails. The sqlite upgrade subset records deferred cases and cannot authorize publication or produce publication artifacts. Release acceptance retains the full matrix.
• Retain bounded image pulls/startup, native diagnostics before timeout/teardown, the external seekDB watchdog, complete migration-shard coverage, separate grant-migration cleanup budgets and isolated fixture data with shared snapshot schemas.
• Select Harness replay and native OceanBase acceptance independently, retain separate evidence checks, and reserve SQLite setup/cleanup time outside its replay deadline. Reuse Main's formatting checks while retaining complete standalone validation.
• Generate a report after every scheduled or manually dispatched Main completion, including failure and cancellation. A separate read-only workflow records the exact source attempt, SHA, Beijing timestamps, every job result/duration, abnormal steps and evidence links. Publish Markdown in the job summary and retain Markdown/JSON artifacts for 30 days; incomplete metadata fails reporting explicitly.
• Keep Actions → Main → Run workflow available with branch selection and full upgrade acceptance by default. Document manual execution and report access in English and Chinese.

The branch includes master commit c919c0e2d. The final diff is limited to database CI, supporting scripts, test fixtures and documentation; production runtime, integrations, Gateway, Desktop and release compatibility metadata match that master commit.

Are there any user-facing changes?

PR and master-push database checks use SQLite; seekDB and OceanBase acceptance runs nightly or manually. Scheduled validation uses the default branch and takes effect after merge. A nightly result is not evidence for an unmerged PR head. Public APIs and persisted formats are unchanged.

How was this change tested?

• make check and the final uv run prek run -a: passed.
• actionlint v1.7.12 on all six changed workflows: passed. Repository action-pin validation: passed.
• Focused CI policy, workflow, upgrade, migration-shard, watchdog and Compose regression run: 469 passed, one platform skip. One existing Compose failure-path fixture exceeded its 60-second test limit once; it passed unchanged in isolation (45.72 seconds for the test body). Assertions and deadlines were retained.
• Follow-up tests exercise the final nightly policy and evaluate the workflow routing expressions for PR, push, merge queue, scheduled and manual metadata. These are local simulations, not GitHub scheduler execution.
• Actual SQLite grant-migration and snapshot/authorization regressions: 42 passed, with 53 native cases deselected.
• Report tests: 15 passed; existing CI policy/pipeline tests: 202 passed. Final make check passed, including 61 integration-manifest checks.
• Read-only report generation against actual completed scheduled run 42744760796: all 94 jobs captured (60 success, 21 failure, 13 skipped), with the source failure preserved. This validates metadata collection/rendering, not deployment of the new workflow listener.
• Real seekDB/OceanBase acceptance and the first scheduled GitHub run/report trigger remain to be verified. No throughput improvement is claimed from local tests.

AI usage statement

OpenAI Codex assisted with analysis, implementation, conflict/integration review, regression tests and PR preparation.

[hireview001]

• CI failed: CI result (failure).
• CI failed: Upgrade release gate (failure).
• CI failed: CI preflight (failure).
• CI failed: Main (failure). Failed steps: CI preflight / Check License Header; Upgrade release gate / Require the selected parallel acceptance workflow; CI result / Require successful selected checks.

[knqiufan]

The existing acceptance-timeout finding still applies: the combined OceanBase setup, native and replay steps can consume more than the enclosing job budget, leaving no guaranteed time for evidence and cleanup. Please address the existing thread. The focused scope and workflow tests passed locally; no real OceanBase job was rerun.

[hidb4ai]

The existing P2 acceptance-timeout finding remains valid at this head. Extracting setup increases the combined step budget from 65 to 80 minutes while the OceanBase job still allows only 75 minutes, so a slow combined run can be cancelled before evidence collection and cleanup. Please make the total fit within the job deadline with time reserved for those final steps. No additional independent findings.

[Zxf-xufeng]

@knqiufan, in response to your review: fixed in 342cf9ad. The OceanBase job now allows 90 minutes, covering 15 minutes of startup, 50 minutes of native acceptance and 15 minutes of replay, with 10 minutes reserved for diagnostics, evidence and cleanup. A regression check protects that reserve for all three selection modes. I also replied in the original inline thread. Local checks pass; the new CI run is still in progress, so real OceanBase acceptance is not yet confirmed.

[Zxf-xufeng]

@hidb4ai, in response to your review: addressed in 342cf9ad. The combined workload's 80-minute step budget now fits within a 90-minute job deadline, leaving 10 minutes for diagnostics, evidence and cleanup. The workflow comment and regression checks reflect this allocation. The branch also includes #2515's fixes and repairs the cross-step container/deadline handoff and JUnit upload path. Local validation passes; the new CI run is still in progress.

[Zxf-xufeng]

@hireview001, following up on your CI failure review after the new run 42380905358 completed for 342cf9ad: it also failed, but at a different point in license-tool setup.

The previous run failed when installing license-eye into /usr/local/bin without write permission. This branch includes master's writable-prefix change, and the new logs confirm INSTALL_DIR points to the runner's temporary directory. However, CI preflight hit the license step's 10-minute timeout: Go setup took about 6 minutes 51 seconds, followed by dependency downloads/building license-eye. The actual license scan was skipped, so successful installation and scanning remain unverified.

The upgrade gate records SCOPE_RESULT=failure and UPGRADE_RESULT=skipped; its failure and the final CI result are downstream consequences of preflight, not executed upgrade-test failures. Native OceanBase and upgrade acceptance did not run, so this run does not validate the acceptance fixes or all 12 upgrade configurations.

The CI issue is not yet verified as resolved. License-tool setup must complete successfully before the remaining acceptance can be validated.

[knqiufan]

lgtm

[AlexStocks]

@Zxf-xufeng plz fix the ci failure.

[Zxf-xufeng]

@hireview001 Following up on the earlier CI failure notice, now that the verification run has finished:

The reported license-header stage passed in CI preflight. The complete scan found no invalid headers; the scan budget now accommodates its measured three-minute runtime.

The overall run is still red for a different reason: Python 3.14 E2E hit the Gateway fixture's 300-second build limit while downloading Rust dependencies. Its 34 setup errors share that cause; the downstream upgrade and final gates correctly failed because acceptance could not run.

Commit 0eacec4 adds a cached, separately bounded Gateway prebuild before pytest and reserves time for the build, tests, and cache cleanup. The new CI run is pending verification, so I am not claiming the overall CI failure is resolved yet.

[hidb4ai]

Re-reviewed at 0eacec40. My previous acceptance-timeout objection is addressed: the bounded OceanBase steps total 117 minutes within a 125-minute job, and SQLite replay has 40 minutes within a 50-minute job. I found no additional confirmed correctness defect.

Validation remains incomplete: Main run 42776126403 was cancelled, with harness and upgrade acceptance skipped. Local review also hit the existing test_oceanbase_compose_does_not_start_server_after_migration_failure[verify] test's 60-second deadline twice on this head; the base completed in 52.7 seconds. I cannot attribute that timing failure to this patch. The real seekDB query checks do not establish OceanBase or complete replay/upgrade acceptance.

[hidb4ai]

LGTM

[hidb4ai]

Post-merge re-review of a4e70f77 (merged as 1026e37d, with an identical source tree): the previous timeout finding is resolved. I found no additional correctness defect attributable to this diff.

The first scheduled Main run finished with 90 successful, 13 skipped and 6 failed jobs. All 12 upgrade cases passed, and the OceanBase native acceptance step passed all 22 selected tests. The report workflow also succeeded and correctly preserved Main's failure. Scheduling and reporting now have actual execution evidence, but full acceptance is still incomplete.

Remaining CI follow-up:

• OceanBase migration shard 1 hit exit=124 after 300 seconds in test_concurrent_publication_does_not_reject_managed_startup[oceanbase-server-False]. The test and per-process deadline are unchanged; the cause is not established.
• DSH acceptance failed the native include-tree setup test. The DSH implementation is unchanged by this PR; this remains a separate investigation.
• SQLite and OceanBase replay could not start powercontext-e2e-container. The subsequent fix in fix(ci): restore harness startup and acceptance execution #2537 is now merged, but this scheduled run predates that fix and does not verify recovery.

Please verify these follow-ups in a complete scheduled or manual run before treating nightly acceptance as green.

[hidb4ai]

Follow-up to the previous verification note: #2526 remains unchanged at a4e70f77 and is already merged.

The later manual Main run at 6c960c58 passed both SQLite and OceanBase replay acceptance. The previously timed-out test_concurrent_publication_does_not_reject_managed_startup[oceanbase-server-False] also passed in this run. These are new execution results for the follow-up tree, not a rerun of the original PR head.

Full acceptance remains incomplete for different failures:

• OceanBase shard 0: the background metrics test failed because the Background CLI did not become live.
• OceanBase shard 1: test_supported_long_query_preserves_all_terms_in_both_http_searches[oceanbase-hybrid-ascii-1000] reached its 300-second deadline (exit=124).
• DSH package: the automatic-status deadline assertion failed in tests/status.spec.ts.
• Gateway compatibility setup: Bun installation failed with an npm ENOENT rename error.

The report workflow succeeded and preserved Main's failure. Please investigate these failures on the later commit and obtain a complete passing run. They are not established regressions introduced by #2526; the original timeout finding remains resolved.