Skip to content

server: Write registry files atomically and ignore non-JSON entries - #687

Closed
Nacl-HCl wants to merge 1 commit into
eclipse-basyx:mainfrom
Nacl-HCl:fix-registry-file-store-666
Closed

Nacl-HCl wants to merge 1 commit into
eclipse-basyx:mainfrom
Nacl-HCl:fix-registry-file-store-666

Conversation

@Nacl-HCl

@Nacl-HCl Nacl-HCl commented Nov 5, 2026

Copy link
Copy Markdown

LocalFileDescriptorStore (the registry storage used with
STORAGE_PERSISTENCY) is a copy of the SDK's LocalFileIdentifiableStore,
but two fixes made to the SDK store were never carried over:

  1. add() and commit() opened the target file with "w" and then
    serialized into it. A crash or a full disk left an empty or truncated
    JSON file behind.
  2. __len__() and __iter__() used os.listdir() without a .json
    filter, and __iter__() stripped the suffix with rstrip(".json").
    Any other entry (lost+found at the root of an ext4 volume,
    .DS_Store) made GET /shell-descriptors fail with a 500.

This ports both fixes (option 1 from the issue), mirroring the SDK store:

  • New _write_atomic(): serialize to a temp file in the store directory
    and os.replace() it over the final file, so readers only ever see a
    complete file; the temp file is removed again if the write fails.
    add() and commit() now use it.
  • __len__() and __iter__() only consider .json entries, and the
    suffix is removed with name[:-5] instead of rstrip(".json").

Fixes #666

How to verify

  • New regression tests in server/test/backend/test_local_file.py:
    test_len_and_iter_ignore_non_json_entries,
    test_failed_commit_keeps_previous_file,
    test_failed_add_leaves_no_file.
  • Reproduction (before the fix): len() returned 3 with 1 descriptor +
    lost+found + .DS_Store; list() raised KeyError on .DS_Store;
    a failing commit() left the stored file empty. After the fix all
    three cases behave correctly and no .tmp file is left behind.
  • pytest test/backend/test_local_file.py test/model/: 10 passed.
  • ruff check and ruff format --check on both changed files: clean.
  • mypy app test (from server/): no issues in 32 source files.

LocalFileDescriptorStore wrote descriptors in place (open with "w",
then serialize), so a crash or a full disk left an empty or truncated
JSON file behind. __len__() and __iter__() also looked at every entry
in the storage directory, so foreign entries (lost+found, .DS_Store)
broke GET /shell-descriptors with a 500.

The store now writes through a temp file plus os.replace(), like the
SDK's LocalFileIdentifiableStore does since eclipse-basyx#553, and len() and
iteration only consider .json files, like the SDK store since eclipse-basyx#507.

Fixes eclipse-basyx#666
@s-heppner

Copy link
Copy Markdown
Member

Thanks for your interest. We're closing this PR because it doesn't follow our contribution guidelines.

@s-heppner s-heppner closed this Nov 13, 2026
@Nacl-HCl

Copy link
Copy Markdown
Author

Hi, could you clarify whether #687/#688 were closed because they targeted main instead of develop? ECA is signed and the titles follow the guide — if develop is the expected base, I'll retarget there. Thanks!

@s-heppner

Copy link
Copy Markdown
Member

Hi, thanks for asking. Changing the base branch alone would not change the outcome. Both #687 and #688 were closed for these reasons:

  • Neither issue was claimed or assigned to you. We only review pull requests for issues that a maintainer has assigned (see Claim an Issue). Pull requests for unassigned issues are closed.
  • We believe these pull requests were made with AI, and the description does not name the model (see Merging Pull Requests). Under our guidelines we reject such pull requests.
  • They target main instead of develop.

Please don't reopen or retarget these pull requests. If you want to work on an issue, comment on it with your use case and wait until a maintainer assigns it to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

server: Registry file store truncates files and breaks on non-JSON files

2 participants