Skip to content

Translation checks avoid repeated row and interval scans - #834

Merged
zlareb1 merged 1 commit into
mainfrom
fix/i18n-check-traversals
Nov 18, 2026
Merged

zlareb1 merged 1 commit into
mainfrom
fix/i18n-check-traversals

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Related: #786

Changelog category (leave one):

  • Performance Improvement

Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):

Translation validation indexes protected ranges and table rows to avoid repeated scans on large reference pages. Request boundaries, labels and source offsets are preserved.

Why

List protection scans every protected interval for each line. Table reconciliation repeatedly normalizes remaining candidate cells when the two Markdown dialects produce different rows. Large reference tables multiply that work during translation validation.

Scope and risk

Two pure parser paths change. Protected-line lookups use sorted starts and prefix maximum ends, retaining overlapping and unsorted intervals and exact-start behavior. Table matching queues source-row indexes by their normalized labels, preserving the first matching duplicate, source offsets, unmatched dialect rows and original remaining-row order.

Regression cases check those semantics and bound interval visits and cell copies as input grows. Existing container, table-reflow and request-boundary cases remain in the selected checks.

Compatibility, rollout and rollback

There is no wire, schema, dependency, configuration or provider change. Operators need no migration or configuration step. Rolling back restores the previous parser traversal.

Tests

Six new regression cases produce two expected failures before the repair. Afterward, 18 selected directly related cases pass. Ruff and format checks pass for both files. Full suites run in PR CI and must pass before merge.

A bounded profile on the existing 214,207-byte memory-operation reference compares old and new functions on the same input. All 82 request strings and all 6,777 labels, including offsets and metadata, are exactly equal. Under cProfile, request planning takes 2.81 versus 0.72 seconds and table-label extraction 8.18 versus 1.51 seconds. These are one-input observations, not production latency guarantees. No provider calls run.

Profile the two paths on each checkout with:

python -c 'import cProfile; from pathlib import Path; from scripts.i18n import chunks, translate; text = Path("docs/reference/operations/memory.md").read_text(); cProfile.runctx("translate.english_requests(text); chunks.table_labels(text)", globals(), locals(), sort="cumulative")'

Reproduce the new semantic and work-bound regressions with pytest tests/unit/test_i18n_request_bounds.py -q -k 'site_list_boundaries or site_lists_do_not_revisit or table_labels'.

AI assistance in the code (leave one; writing this description with AI is fine):

  • AI-generated (an agent wrote most of the diff).

  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • make format-check, make lint, make typecheck and make test-unit pass; the integration suite passes if server behaviour changed.
  • Generated files are regenerated in this PR, not by hand: the SDK op tables, docs/reference/configuration.md and the docs/reference/ tables (python scripts/codegen/docs_reference.py), the reach manifest, schemas/schema.sql (make schema-dump) and the Codex plugin copies (python3 scripts/codegen/agent_plugin_sync.py).
  • A new migration is additive and numbered after the latest on main; no merged migration is edited.
  • A behaviour change to any file a plugin ships (hooks/, lib/, bin/, skills/, .mcp.json, the Codex plugin-defaults.json) bumps both plugin manifests' version, and no other open PR uses that version.
  • The frontend builds (npm --prefix frontend run build) and its tests pass, if frontend/ changed.
  • Docs under docs/ describe the system as it now is, and this description names any operator steps and wire changes.
  • Repository text has no dates, people, host names, internal namespaces, ticket or decision numbers; the history goes in this description (tests/unit/test_public_copy.py).
  • Commits are signed off (git commit -s), as CONTRIBUTING.md asks.

Index normalized table rows while preserving first-match offsets and unmatched row order. Use prefix maximum ends for protected-line lookups.

Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
@zlareb1
zlareb1 merged commit 521e52f into main Nov 18, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant