Skip to content

feat(id-docs): add occ command to recover identification documents orphaned in appdata - #9535

Open
maia-andre wants to merge 2 commits into
LibreSign:mainfrom
maia-andre:fix/9168-recover-orphaned-id-docs
Open

maia-andre wants to merge 2 commits into
LibreSign:mainfrom
maia-andre:fix/9168-recover-orphaned-id-docs

Conversation

@maia-andre

@maia-andre maia-andre commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Resolves: #9168

📝 Summary

Adds occ libresign:id-docs:recover, which finds identification documents of external signers that were written to the unauthenticated appdata folder before #8368 and copies them into the storage of their owner, so they open again in Documents Validation.

Your answer on the questions in my claim comment hadn't arrived, so I went with the defaults proposed there. Each one is easy to change in review:

  1. Name: libresign:id-docs:recover.
  2. Originals: never deleted by the command. The admin removes them after verifying. The admin docs say so.
  3. Signed copies: rows with a signed_node_id are reported as ambiguous and skipped.
  4. Backport: main only for now. Stables 32–35 lack FolderService::getUnauthenticatedFolder(), so a backport needs adaptation. Happy to do it if you want it.

How it decides

Only id_docs rows with a null user_id (the external-signer path) are checked. Each one is reported with a reason:

Status When
affected node found in unauthenticated appdata, owner exists and matches the owner of the signed document, node referenced by exactly one file record
already_correct the owner can already read the node (this includes rows recovered by an earlier run)
ambiguous owner differs from the signed document's owner, node shared by several file records, or a signed copy exists
unrecoverable missing file record, owner, owner account, sign request, signed document or source file

What --execute does for an affected row

  1. Creates a new folder <date>_recovered-id-doc-<fileId>_<uid> in the owner's LibreSign folder, using the same getFolderForFile() resolution as the fixed upload path. It stops if that folder is not inside the owner's files (see below).
  2. Streams the file into it.
  3. Compares SHA-256 of copy and original, and checks the owner can read the copy.
  4. Updates libresign_file.node_id in a transaction, using getByIdForUpdate(), only if node_id is unchanged and there is still no signed copy.

File IDs, UUIDs, id_docs and sign request rows are not touched. Any failure removes the new folder and leaves the record as it was, so the next run sees the row as affected again. If that cleanup fails, the leftover node ID is printed. The output contains record and node IDs only: no file names, contents or signer data. Exceptions go to the server log.

Found while testing manually

In a CLI process, if a user whose home was never initialized is looked up by node ID first, a later write to that home throws NotFoundException, and FolderService::getContainerFolder() falls back to appdata. The copy would then become a new orphan. The command now checks that the target folder is inside the owner's files before copying, and reports owner storage not available otherwise. I left FolderService itself alone. In practice an owner who requested a signature already has a home, and the docs tell the admin to have the owner log in once.

Reading order

lib/Service/IdDocsRecoveryService.php first, then lib/Command/IdDocs/Recover.php. Composition: about 360 lines of code (service 201, command 73, mapper and FolderService helpers 86) and 529 lines of tests. Two commits: service, then command.

Admin documentation: LibreSign/documentation#176.

🧪 How to test

On a disposable instance with synthetic data:

occ libresign:id-docs:recover            # dry-run, nothing changes
occ libresign:id-docs:recover --execute  # recover
occ libresign:id-docs:recover --execute  # re-run: already_correct

Then open the document in Documents Validation.

Output from my run (isolated sqlite instance, synthetic rows: one owner with an initialized home, one without):

### dry-run
| 73 | 326 | 2407 | affected | stored in unauthenticated appdata |      |
| 74 | 328 | 2413 | affected | stored in unauthenticated appdata |      |
### --execute
| 73 | 326 | 2407 | failed    | owner storage not available            |      |   (exit 1, nothing left in appdata)
| 74 | 328 | 2413 | recovered | copied to owner storage, original kept | 2425 |
### after the first owner's home was initialized, --execute
| 73 | 326 | 2407 | recovered       | copied to owner storage, original kept | 2435 |
| 74 | 328 | 2425 | already_correct | readable in owner storage              |      |

After that, AccountFileService::getPdfByUuid() (the Documents Validation path) returned the original content for both documents.

Validation evidence

  • Focused regression/acceptance test
  • Relevant lint/type/static-analysis checks
  • Broader suite required by the changed surface
  • CI reviewed

All runs were in the devcontainer, against an isolated sqlite instance (NEXTCLOUD_CONFIG_DIR).

  • tests/php/Unit/Service/IdDocsRecoveryServiceTest.php and tests/php/Unit/Command/IdDocs/RecoverTest.php: 22 tests, OK. They cover each skip reason, dry-run without copying, recovery, content mismatch, unreadable copy, record changed or signed concurrently, appdata fallback, rollback with failed cleanup, and command output and exit codes.
  • tests/php/Integration/Service/IdDocsRecoveryServiceTest.php: 2 tests, OK, with real storage and database:
    • dry-run classifies an orphan, an already-correct document, a missing source, a deleted owner and a broken sign request, without changing records or creating folders;
    • --execute makes getPdfByUuid() return the content (it throws before recovery), keeps the original in appdata, keeps the UUID, owner and id_docs/sign request link, doesn't expose the node to another user, and is idempotent on re-run.
  • Full api,integration suite: 517 tests, OK (1 warning from DnsPinMiddleware, network). An earlier run showed the new test failing only inside the full suite, because of stale appdata nodes left by earlier classes. It now extends AppDataTestCase, like the other appdata tests.
  • composer test:unit: 5014 tests. The only error is TsaValidationServiceTest reaching freetsa.org (no network here).
  • composer cs:check: clean. composer psalm: no issues in the changed files. The 2 errors reported are in lib/Service/Install/SignSetupService.php, which this PR doesn't touch, so I expect they come from my local vendor-bin.

Not verified: MySQL/PostgreSQL. The integration run used sqlite, where getByIdForUpdate() does not add FOR UPDATE.

⚙️ API / Back-end changes

  • New occ command libresign:id-docs:recover (no HTTP API change, no OpenAPI change)
  • Unit and/or integration tests added or updated
  • Authorization and negative paths checked when applicable
  • Documentation updated when applicable

✅ Checklist

  • I have read and followed the contribution guide.
  • The diff does not contain unrelated refactors or generated changes.
  • Commits follow Conventional Commits where applicable.
  • All commits include the required DCO sign-off.

🤖 AI (if applicable)

  • The content of this PR was partially or fully generated using AI

Before LibreSign#8368, identification documents uploaded by external signers were
written to the unauthenticated appdata folder while their file record
pointed to the owner of the signed document, so they never opened in
Documents Validation.

IdDocsRecoveryService classifies every id_docs row without a user as
affected, already correct, ambiguous or unrecoverable, with a reason.
Only rows whose owner matches the owner of the signed document, whose
node is not shared by another record and that have no signed copy are
recovered.

Recovery copies the file into a new folder of the owner, compares the
SHA-256 of both, checks that the owner can read the copy and then
updates node_id under a row lock, only if the record did not change.
The original stays in appdata. A failed step removes the new folder and
leaves the record untouched, so a later run starts over.

Signed-off-by: André Maia <andrefnkmm@gmail.com>
Expose the recovery as an occ command. It runs as a dry-run by default
and only changes data with --execute. The table lists record and node
IDs only, without file names or signer data, and the command exits with
a failure code when a recovery fails.

Signed-off-by: André Maia <andrefnkmm@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recover orphaned identification documents from legacy unauthenticated uploads

1 participant