Skip to content

Prove one-session-per-document readiness over loopback serve - #201

Merged
chouswei merged 15 commits into
masterfrom
cursor/doc-gate-readiness-9931
Oct 8, 2026
Merged

chouswei merged 15 commits into
masterfrom
cursor/doc-gate-readiness-9931

Conversation

@chouswei

@chouswei chouswei commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Product-gate readiness for one MemNet session per document on memnet-llm 0.19.18 (12753ab / v0.19.18). The gate talks to memnet-serve on loopback only (no MCP front). Synthetic data. No engine or cap-default change. No version bump. Do not merge.

This VM: Intel(R) Xeon(R) Processor, 4 cores, KVM. The face host may be slower.

Report (items 2–7)

Item Verdict Measured
2 Session churn yes 110 cycles × 1800 parts. Live 0→0. RSS after close stayed flat around 70–72 MiB.
3 Snapshot round-trip note Single-line unicode / pipe / quotes: exact dump. Raw newlines: @ERR: FIELD_COUNT. Write-once: not in MemNet.
4 Save-on-expire yes @ERR: session_expired|snap_available and expire snapshot written.
5 ACL over serve note acl_who / acl_denied / acl_scope on pin_map/mutate/export. save/load/close do not who-check. Bind skipped on serve.
6 Envelope + RSS yes / note Reply {exit_code, stdout, stderr} only. No per-session RSS. 1800-part RSS mean 4.38 MiB.
7 GQL note CREATE 32 props / MATCH…SET / hop: yes. count() RETURN and ISO INSERT: no. M=151 Truncation: yes.

docs/cap-contract.md: unchanged (19/19). Admin usage live vs true: confirmed, not fixed.

Extra probes (E11–E17)

Item Verdict Measured
E11 session_load vs ingest yes 3000-node snap loads @STAT: loaded|3000. Not ingest_budget. Path: load_snapshot_text (parse_line + upsert). Fulldoc 7500-row snap also loads at MEMNET_MAX_ROWS=10000, still not ingest_budget.
E12 MEMNET_MAX_ROWS yes Counts nodes plus edges. Tiny: @ERR: limit_exceeded|rows 5/4. Fulldoc 3000+4500=7500: default 5000 refuses @ERR: limit_exceeded|rows 5001/5000 on write and on session_load. Pi 10000 holds rows=7500 edges=4500. Read is not the session cap: hub M=50 → ## Truncation truncated=true M=50 omitted=2956 reason=max_rows; hub M=4000 → @ERR: response_too_large|response 9491260 bytes exceeds cap 4194304.
E13 16 KiB strings note CREATE/SET/pin_map in RAM: yes (CJK, \, $, both quotes, newlines, |). Snapshot load: FIELD_COUNT or value_bytes 16384/4096. GQL mutate skips pipe 4096/32768 (bug 4). Escapes: \\ \' \" \n \r \t.
E14 list IN note Lists store. 'k' IN p.citeKeys is not a filter: @ERR: cue_conflict|SET Q =2. Locators are KEY=VAL equality. leftover glob works on a small graph.
E16 latency note n=200, bar 300 ms p95. (a) SET+del 1+add 2: p50 115.686 / p95 133.181 / max 155.571 ms. (b) reverse pin_map M=400: p50 113.096 / p95 129.613 / max 163.101 ms. No native delete-refused-while-referenced (DETACH DELETE exits 0, dangling edges). Documented edge DELETE: @ERR: not_found|DELETE matched no element. Working edge DROP: MATCH (n WHERE true)-[r {id}]->() DELETE r.
E17 WHERE CONTAINS note Not a product substring filter. MATCH … WHERE n.value CONTAINS '测例' RETURN n → @ERR: product_gate|agent surface forbids RETURN …. MATCH … WHERE … SET GraphGlot-parses (CJK, $, \, both quote kinds) but lowering drops WHERE: @ERR: cue_conflict|SET Q =1500. Unique MATCH still SET when CONTAINS would miss. Inline MATCH (n WHERE … CONTAINS) → @ERR: parse_error|unsupported MATCH shape. Bare WHERE → @ERR: parse_error|unsupported MATCH continuation. STARTS WITH / ENDS WITH / =~ are the same ignored-WHERE path. Working: query find --keyword / pin_map --keyword (casefold across fields, --limit / --max-rows). leftover read list --where value=*测例* works small; fulldoc → @ERR: response_too_large|response 4659616 bytes exceeds cap 4194304. Needle escaping is GQL string rules only (\\ \' \" \n \r \t); CJK and $ unescaped; both '…' and "…" parse for the CONTAINS operand. Substitute latency n=200, find --limit 50, warm fulldoc: common 测例 p50 67.282 / p95 84.020 / max 94.672 ms (50 of 1500); rare Part 1500 p50 51.183 / p95 69.090 / max 85.424 ms (1 hit).

Fat USR payload on this fixture is about 4.4 MiB UTF-8 (1500 × 2–4 KiB), not 1 MiB.

E18 snapshot value cap / whitespace / property count

yes. The leftover snapshot value_bytes cap (4096) is measured on the decoded field after split_payload, not on the escaped snapshot line. validate_values uses > not >=, so a 4096-byte decoded value loads. join_payload doubles \ and | only; that expansion is not the cap. parse_line measures the raw snapshot line vs line_bytes 32768 first. SCHEMA register vs max_fields=32. emit_record writes SCHEMA columns only. No engine change.

Citations: tag_map.py validate_values (len(val.encode("utf-8")) > caps.max_value_bytes after values = split_payload(payload)); parse_line raw line.encode vs max_line_bytes; wire.join_payload field.replace("\\", "\\\\").replace("|", "\\|"); CR/LF → newline_in_value (tab not checked); _register_user_tag len(field_names) > caps.max_fields; output.emit_record SCHEMA columns only.

Live path: mutate CREATE (pipe/CREATE as the gate uses) → session save → session load into a fresh session → pin_map cue. Binary search skipped (4000 \ and | already exact). Proof: /opt/cursor/artifacts/doc-gate-readiness-e18.log.

Case Wire shape Save / load Exact?
E18a 4000 \ CREATE (:USR {id: 'USR_a4kbs', key: 'e18', value: <blob utf8=4000 chars=4000>, recycle: ''}) 0 / 0, no @ERR yes (escaped 8000, snap line 8021)
E18a 4000 | CREATE (:USR {id: 'USR_b4kpp', key: 'e18', value: <blob utf8=4000 chars=4000>, recycle: ''}) 0 / 0 yes (escaped 8000, snap line 8021)
E18a 4000 " CREATE (:USR {id: 'USR_c4kdq', …}) 0 / 0 yes (escaped 4000, snap line 4021)
E18a 4000 ' CREATE (:USR {id: 'USR_d4ksq', …}) 0 / 0 yes (escaped 4000, snap line 4021)
E18a 4000 CJK 1333 × U+6D4B (测, 3-byte UTF-8) + 1 ASCII X; USR_e4kcj 0 / 0 yes (1334 chars, utf8 4000, snap line 4021)
E18a 4096 (a–e) same shapes; CJK is 1365 × 测 + 1 X 0 / 0 all five yes (\/| snap line 8215; quotes/CJK 4119)
E18b tab mid/end CREATE (:USR {id: 'USR_tabm', … value: 'ab\tcd'}) / 'ab\t' 0 / 0 yes (byte-exact)
E18b CR mid/end … value: 'ab\rcd' / 'ab\r' save 0 / load 1 no — @ERR: FIELD_COUNT|Expected 4 fields for USR got 3 (same as newline: str.splitlines splits on CR before newline_in_value)
E18c SCHEMA 64/128 SCHEMA WIDE ; fields=id p000 … (64 / 128 names) open 1 @ERR: limit_exceeded|fields 64/32 and 128/32
E18c SCHEMA 32 SCHEMA PRT ; fields=id p00 … p30 save 0 / load 0 yes
E18c 64/128 extras on USR CREATE 60 / 124 keys beyond 4-field SCHEMA save 0 / load 0 RAM extras yes; load drops them
E18c 8 × 4000 ASCII CREATE (:FAT {id: 'FAT_8x4000', p000: <4000 A>, … p007: <4000 A>}) 0 / 0 yes (snap line 32024 < 32768; all eight fields exact)

Gaps (do not fix here)

  • Snapshot cannot round-trip raw newlines or CR (FIELD_COUNT via splitlines).
  • 16 KiB properties refuse leftover snapshot load (value_bytes).
  • No write-once snapshot; no per-session RSS; bind not enforced on serve.
  • save / load / close do not who-check.
  • Grouped count() / RETURN and ISO INSERT are not product mutate.
  • List IN and GQL CONTAINS / STARTS WITH / =~ are not product filters.
  • Default 5000 cannot hold a 7500-row fulldoc. Hub pin_map M=4000 and leftover glob of fat USR hit the 4 MiB serve frame.
  • No native delete-refused-while-referenced.
  • SCHEMA max_fields=32; extra RAM properties drop on snapshot emit.

Proof

Sid-free logs: /opt/cursor/artifacts/doc-gate-readiness-proof.log (items 2–7 + E11–E14 + fat RSS); /opt/cursor/artifacts/doc-gate-readiness-e12-e16.log; /opt/cursor/artifacts/doc-gate-readiness-e17.log; /opt/cursor/artifacts/doc-gate-readiness-e18.log.

python scripts/probe_doc_gate_readiness.py --out /opt/cursor/artifacts/doc-gate-readiness-proof.log
python scripts/probe_doc_gate_readiness.py --churn 0 --rss-samples 0 --expire-wait 0 --fat-churn 0 --fat-rss-samples 0 --load-nodes 0 --fulldoc-nodes 0 --nodes 40 --out /opt/cursor/artifacts/doc-gate-readiness-e18.log

Do not merge.

Open in Web Open in Cursor 

cursoragent and others added 15 commits October 8, 2026 11:43
Add a sid-free probe and live-serve tests for churn, snapshot round-trip,
save-on-expire, ACL, GQL coverage, and process RSS. Document the gate loop.
Do not change engine behaviour or cap defaults.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Classify ACL lifecycle skips separately from authorised success. Populate
synthetic USR text as a single line so 1800-part round-trip can be exact.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Report-only: session_load vs ingest budget, MEMNET_MAX_ROWS node+edge
count, 16 KiB string round-trip, list IN membership. No engine change.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Snapshot emit escapes |; 16 KiB still refuses value_bytes 16384/4096.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Report-only: 3000 nodes plus 4500 cites/refersTo/inSection edges through
mutate; session row cap on write/read/load; reverse-lookup then DELETE.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Record 4 MiB serve-frame failures instead of aborting the probe, and treat
a successful 7500-row load under MEMNET_MAX_ROWS=5000 as a no.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Documented MATCH ()-[r {id}]-() DELETE r lowers as an empty-id node DROP
and refuses not_found. Use MATCH (n WHERE true)-[r {id}]->() DELETE r for
the atomic batch; keep the documented refuse as a measured finding.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
MEMNET_MAX_ROWS counts nodes plus edges on write and session_load
(5001/5000 at default; 7500 fits 10000). pin_map hub M=4000 hits the
4 MiB serve frame. E16 p95 is under 300 ms; no native referenced-delete.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
CONTAINS / STARTS WITH / ENDS WITH / =~ are not product filters (RETURN is
product_gate; SET drops WHERE). Record needle escaping and measure
find --keyword as the working substring on the fulldoc fixture.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Do not read pin4000.stderr before the hub pin_map call runs.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Hub neighbourhood hid the updated SEC row. leftover glob of 1500 fat
USR values is recorded even when the 4 MiB serve frame refuses it.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
CONTAINS is not a product filter. find --keyword p95 is under 85 ms on
the fulldoc fixture; leftover glob of fat USR hits the 4 MiB frame.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Prove leftover value_bytes is decoded after split_payload (not escaped
join_payload size), then round-trip 4000/4096-byte \, |, quotes, and CJK
through mutate CREATE / save / load. No engine change.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
Fill the one-session-per-document E18 table from the loopback serve probe.
Leave short-blob wire shapes intact so USR_w64 is not tokenised.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
@chouswei
chouswei marked this pull request as ready for review October 8, 2026 14:03
@chouswei
chouswei merged commit 8a634d3 into master Oct 8, 2026
2 checks passed
cursor Bot pushed a commit that referenced this pull request Oct 8, 2026
…HERE, and ACL who.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>
chouswei added a commit that referenced this pull request Oct 8, 2026
* Add MN-REQ-01.9, 01.10, 03.4 and 05.3 for snapshot, WHERE and ACL.

SysML-first honesty cut: lossless snapshot round-trip, honour-or-refuse
WHERE, consistent decoded value_bytes cap, and who-check on session
save/load/close. Verify cases MN-VER-01-S04, 01-S05, 03-S01, 05-S01.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Fix snapshot round-trip, GQL WHERE honour-or-refuse, and ACL who-check.

Snapshot emit escapes LF/CR/pipe/backslash; load decodes; save fails closed
with snapshot_unsaveable. GQL mutate shares MEMNET_MAX_VALUE_BYTES on decoded
UTF-8. MATCH WHERE SET/DELETE honours the predicate or refuses
unsupported_predicate with nothing applied. Session save/load/close accept
--caller when ACL is enabled; MCP passes caller through.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Test snapshot round-trip, WHERE, ACL who-check; update cap-contract.

Pytest covers lossless specials, 0.19.18 fixture load, GQL value_bytes,
unsupported_predicate, fail-closed save, and session save/load/close ACL.
Cap-contract probe hits the new wires; bug 4 (GQL mutate skipping the
value cap) is removed.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Extend MN-REQ-01.9 and 05.3 for splitlines seps, extras, and line_bytes.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Escape splitlines separators, persist undeclared snapshot properties, check line_bytes at save.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Test snapshot CR/seps round-trip, undeclared property persist, and write-time line_bytes.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Format E18 snapshot tests and shorten cap-contract line_bytes knob.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Flip #201 doc-gate probe and tests onto lossless snapshot, honoured WHERE, and ACL who.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Pass --caller on ServeProc.close; honour documented relationship DELETE in E16.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Keep sessions live when expire-save cannot write a snapshot.

Expire-save failure (snapshot_unsaveable or OSError) no longer drops RAM.
The session still counts against MEMNET_MAX_SESSIONS, retries warn
expire_snapshot_failed, and access after TTL is session_expired|overdue
until an explicit successful save or close. Save-on-expire off is unchanged.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Fix live expire-keep cap assert to match wire sessions 2/1.

Serve @err replaces inner pipes with spaces; the hold still counts against MEMNET_MAX_SESSIONS.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

* Test Endleaf MATCH (n WHERE true)-[r {id}]->() DELETE r.

Trivial WHERE true stays honoured, not unsupported_predicate. In-process and live serve loopback cover the exact no-space id map spelling the gate sends.

Co-authored-by: chouswei <chouswei@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: chouswei <chouswei@users.noreply.github.com>
chouswei added a commit that referenced this pull request Oct 8, 2026
Hatch, project.toml, changelog, and the version map name the #201 doc-gate readiness probe and the #202 snapshot / value cap / WHERE / ACL who / expire-save fixes. Not 0.20.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants