Read Kindle, text and comic formats with cancellable preparation

README.md

@@ -1,7 +1,8 @@
 # Papyrus Reader
 
 `papyrus_reader` is a self-contained Flutter library for reading reflowable
-EPUB 2/3 and PDF documents on Android, iOS, Web, Linux, macOS, and Windows.
+EPUB 2/3, PDF, MOBI, AZW3, TXT, CBZ and CBR documents on Android, iOS, Web,
+Linux, macOS, and Windows.
 
 It provides versioned EPUB content locations, PDF page locations, TOC/outline navigation,
 scroll and paginated layouts, configurable appearance, and a responsive
@@ -67,16 +68,27 @@ flutter run
 ```
 
 The six-platform example includes deterministic long EPUB and three-page PDF
-assets, a local EPUB/PDF file picker, light/dark themes, and in-memory
+assets, a local book file picker, light/dark themes, and in-memory
 position/preference restoration. Example positions reset when the app restarts;
 production persistence belongs to the host application.
 
 ## Current scope
 
-The initial release supports reflowable EPUB 2/3 and PDF. Fixed-layout EPUBs
-return `ReaderErrorCode.unsupportedFixedLayout`. MOBI/AZW3, TXT, comic
-archives, search, bookmarks, highlights, and notes are planned extensions.
-Encrypted/DRM documents are unsupported. EPUB uses semantic reflow: publisher
+Supported inputs are reflowable EPUB 2/3, PDF, DRM-free MOBI/AZW3 (KF8), plain
+text (UTF-8, BOM-marked UTF-16 and Windows-1252), CBZ and CBR (RAR4/RAR5).
+Comic pages use natural filename order and support JPEG, PNG, GIF, WebP and BMP;
+animations display as still pages. Password-protected archives, multi-volume
+comics, Kindle Print Replica, Topaz and encrypted/DRM books are unsupported.
+Fixed-layout EPUBs return `ReaderErrorCode.unsupportedFixedLayout`.
+
+Text and Kindle inputs use the reflow engine. Comics use a transient in-memory
+page document for zoom, scrolling and facing pages; original library files are
+never changed. Conversion runs in a cancellable isolate/browser worker before
+opening; large comics may take longer and require memory for the page document.
+Reflow uses EPUB-style content locators; comics use PDF-style page locators, so
+saved positions, bookmarks and reading activity use the existing contracts.
+
+EPUB and Kindle use semantic reflow: publisher
 stylesheets, embedded fonts, advanced SVG/math, RTL progression and footnote
 interaction are incomplete; complex tables render as readable rows.
 

docs/ARCHITECTURE.md

@@ -42,6 +42,25 @@ Custom HTML rendering restores approximate progression rather than the default
 renderer’s precise content offset. Supported-format limits are in the
 [README](../README.md#current-scope).
 
+## Additional input formats
+
+The shared document worker prepares TXT and MOBI/AZW3 as transient EPUB content,
+then applies the same sanitizer, semantic offsets and layout pipeline. Text
+chapters use deterministic bounded chunks. The vendored Kindle decoder records
+its upstream version, license and browser adjustments in
+`third_party/kindle_unpack/UPSTREAM.md`.
+
+CBZ and CBR are decoded in the worker into a transient PDF page document. Natural
+path ordering determines page indices. Images preserve their aspect ratios and
+JPEG data; archive readers release their resources after preparation. Binary
+responses transfer typed byte buffers in browsers instead of serializing them
+as large JSON arrays. Cancelling a load terminates its conversion worker.
+
+Input format remains on `ReaderDocument`; reflow inputs use `EpubReaderLocator`
+and fixed-page inputs use `PdfReaderLocator`. The host retains the original file,
+and no conversion cache or second persisted copy is introduced. Comic colors
+are preserved in dark mode.
+
 ## PDF resources
 
 `PdfReaderEngine` uses a `PdfFacade` so engine tests do not depend on PDFium.

docs/VALIDATION.md

@@ -13,7 +13,11 @@ python3 tool/check_worker_build.py
 
 Widget tests inject an in-process EPUB transport because fake clocks do not model
 resident-isolate scheduling. Native transport tests and the compiled browser suite
-exercise the real transports. Browser checks require the built example and its
+exercise the real transports. Additional-format tests decode real Gutenberg MOBI/KF8
+text and covers, compare rendered comic pixels for natural page order, restore
+serialized positions, and exercise cancellation. Browser checks also open every
+format through the example file picker and record animation-frame gaps during
+worker processing. Browser checks require the built example and its
 packaged worker. `PAPYRUS_CHROME_EXECUTABLE` selects an installed Chrome;
 `PAPYRUS_SCREENSHOT_DIR` saves review images outside source control.
 
@@ -53,3 +57,44 @@ The driver writes screenshots and timings under `client/app/build/`. Keep run
 results with the relevant PR, including the tested revisions, platform and unrun
 checks. Debug turn timings include deliberate observation frames and test overhead;
 they are not production latency benchmarks.
+
+## Large-book stress checks
+
+Keep large downloads outside Git. The opt-in native corpus expects Webster's
+Unabridged Dictionary (Gutenberg 29765) as `websters-dictionary.epub`, `.mobi`,
+`.azw3`, and `.txt`, plus `alice-high-resolution.pdf`, `whiz-272-pages.cbz`, and
+`whiz-272-pages.cbr`. Record source URLs, checksums and any generated/repeated
+content alongside the files. Never present a repeated-page fixture as a newly
+published book.
+
+```sh
+PAPYRUS_LARGE_BOOKS=/absolute/path/to/corpus \
+  ../tools/flutter test test/stress/large_books_test.dart --concurrency=1
+```
+
+This checks native loading, seeking and locator restoration, and reports event-loop
+gaps and sampled test-process RSS. It does not render Flutter viewports or establish
+device memory limits. The corpus is skipped in ordinary test runs.
+
+For actual browser viewports, build the example, serve `example/build/web` on
+localhost port 8791, and run:
+
+```sh
+PAPYRUS_CHROME_EXECUTABLE=/path/to/chrome \
+PAPYRUS_STRESS_OUTPUT=/absolute/path/to/results \
+  node tool/browser/stress.mjs /absolute/path/to/corpus/*.epub
+```
+
+Pass any supported book files as arguments. `PAPYRUS_STRESS_ORIGIN` overrides the
+server URL. `PAPYRUS_STRESS_CANCEL_ONLY=1` checks cancellation during preparation
+without waiting for full reading. The runner records idle and loading frame gaps,
+long tasks, page-turn screenshots, worker closure and Back response. macOS/Linux
+RSS sampling covers all processes belonging to the test browser, including GPU
+and workers; it is not the reader's isolated heap. Each file gets a fresh browser.
+Screenshots are excluded from the navigation frame measurements.
+
+Functional success does not mean acceptable performance: review load duration,
+long tasks and memory separately. Compare frame gaps with the idle baseline, avoid
+concurrent builds, and record other desktop load. Browser page turns and native
+headless seeks do not certify long-distance Flutter pagination, phone behavior,
+or every page in a large publication.

example/lib/main.dart

@@ -290,27 +290,33 @@ final class _DemoLibrary extends StatelessWidget {
         acceptedTypeGroups: const [
           XTypeGroup(
             label: 'Books',
-            extensions: ['epub', 'pdf'],
-            mimeTypes: ['application/epub+zip', 'application/pdf'],
+            extensions: ['epub', 'pdf', 'mobi', 'azw3', 'txt', 'cbz', 'cbr'],
+
             uniformTypeIdentifiers: [
               'org.idpf.epub-container',
               'com.adobe.pdf',
+              'public.data',
             ],
           ),
         ],
       );
       if (file == null || !context.mounted) return;
       final extension = file.name.split('.').last.toLowerCase();
-      if (extension != 'epub' && extension != 'pdf') {
-        throw const FormatException('Choose an EPUB or PDF document.');
+      final format = ReaderFormat.values
+          .where((format) => format.name == extension)
+          .firstOrNull;
+      if (format == null) {
+        throw const FormatException(
+          'Choose an EPUB, PDF, MOBI, AZW3, TXT, CBZ or CBR document.',
+        );
       }
       final id =
           'file:${file.name}:${await file.length()}:${await file.lastModified()}';
       if (!context.mounted) return;
       final document = ReaderDocument(
         id: id,
         title: file.name,
-        format: extension == 'epub' ? ReaderFormat.epub : ReaderFormat.pdf,
+        format: format,
         loadBytes: file.readAsBytes,
       );
       Navigator.of(context).push(

example/pubspec.lock

@@ -25,6 +25,22 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "2.13.1"
+  barcode:
+    dependency: transitive
+    description:
+      name: barcode
+      sha256: "7b6729c37e3b7f34233e2318d866e8c48ddb46c1f7ad01ff7bb2a8de1da2b9f4"
+      url: "https://pub.dev"
+    source: hosted
+    version: "2.2.9"
+  bidi:
+    dependency: transitive
+    description:
+      name: bidi
+      sha256: "77f475165e94b261745cf1032c751e2032b8ed92ccb2bf5716036db79320637d"
+      url: "https://pub.dev"
+    source: hosted
+    version: "2.0.13"
   boolean_selector:
     dependency: transitive
     description:
@@ -272,6 +288,37 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "1.0.1"
+  kindle_unpack:
+    dependency: transitive
+    description:
+      path: "../third_party/kindle_unpack"
+      relative: true
+    source: path
+    version: "0.2.0"
+  koni_archive_core:
+    dependency: transitive
+    description:
+      name: koni_archive_core
+      sha256: f323296866d6224725839e2b8ef25dfeba5e0c2476877fa7726f4b966a4c7a84
+      url: "https://pub.dev"
+    source: hosted
+    version: "0.9.0"
+  koni_codecs:
+    dependency: transitive
+    description:
+      name: koni_codecs
+      sha256: "0a0ffa23f62f627bc50db6de79e88a6a41e019d3a21c6a455ac5ae7c1677a8bd"
+      url: "https://pub.dev"
+    source: hosted
+    version: "0.9.0"
+  koni_rar:
+    dependency: transitive
+    description:
+      name: koni_rar
+      sha256: c8f1a5b2f1b9d8f6cfd981c67e8c34298f722806f7288f06faabb40004362f56
+      url: "https://pub.dev"
+    source: hosted
+    version: "0.9.0"
   leak_tracker:
     dependency: transitive
     description:
@@ -375,6 +422,14 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "1.9.1"
+  path_parsing:
+    dependency: transitive
+    description:
+      name: path_parsing
+      sha256: "883402936929eac138ee0a45da5b0f2c80f89913e6dc3bf77eb65b84b409c6ca"
+      url: "https://pub.dev"
+    source: hosted
+    version: "1.1.0"
   path_provider:
     dependency: transitive
     description:
@@ -423,6 +478,14 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "2.3.0"
+  pdf:
+    dependency: transitive
+    description:
+      name: pdf
+      sha256: e47a275b267873d5944ad5f5ff0dcc7ac2e36c02b3046a0ffac9b72fd362c44b
+      url: "https://pub.dev"
+    source: hosted
+    version: "3.12.0"
   pdfium_dart:
     dependency: transitive
     description:
@@ -495,6 +558,14 @@ packages:
       url: "https://pub.dev"
     source: hosted
     version: "2.2.0"
+  qr:
+    dependency: transitive
+    description:
+      name: qr
+      sha256: "2c0189dca64a3ab08dda7acd7911c89af2374f08eb4eaded3e8ef4609f3b9c8b"
+      url: "https://pub.dev"
+    source: hosted
+    version: "3.0.3"
   record_use:
     dependency: transitive
     description:

lib/src/domain/reader_types.dart

@@ -1,4 +1,17 @@
-enum ReaderFormat { epub, pdf }
+enum ReaderFormat {
+  epub,
+  pdf,
+  mobi,
+  azw3,
+  txt,
+  cbr,
+  cbz;
+
+  bool get isReflowable => switch (this) {
+    epub || mobi || azw3 || txt => true,
+    pdf || cbr || cbz => false,
+  };
+}
 
 enum ReaderLayoutMode { paginated, scroll }
 

lib/src/engine/epub/epub_reader_engine.dart

@@ -85,7 +85,12 @@ final class EpubReaderEngine extends ReaderEngine {
   ReaderPreferences get preferences => _preferences;
 
   @override
-  Set<ReaderFormat> get supportedFormats => const {ReaderFormat.epub};
+  Set<ReaderFormat> get supportedFormats => const {
+    ReaderFormat.epub,
+    ReaderFormat.mobi,
+    ReaderFormat.azw3,
+    ReaderFormat.txt,
+  };
   @override
   ReaderCapabilities get capabilities => const ReaderCapabilities(
     supportsPagination: true,
@@ -103,11 +108,11 @@ final class EpubReaderEngine extends ReaderEngine {
     ReaderLocator? initialLocator,
     required ReaderPreferences preferences,
   }) async {
-    if (document.format != ReaderFormat.epub ||
+    if (!supportedFormats.contains(document.format) ||
         (initialLocator != null && initialLocator is! EpubReaderLocator)) {
       throw const ReaderException(
         ReaderErrorCode.invalidDocument,
-        'The EPUB document or initial locator has the wrong format.',
+        'The text document or initial locator has the wrong format.',
       );
     }
     cancelPendingSeek();
@@ -119,7 +124,12 @@ final class EpubReaderEngine extends ReaderEngine {
       candidate = await _workerFactory();
       _requireCurrent(generation);
       _openingWorker = candidate;
-      final metadata = await candidate.open(bytes);
+      final metadata = await candidate.request(
+        document.format == ReaderFormat.epub
+            ? 'open'
+            : 'open_${document.format.name}',
+        bytes,
+      );
       _requireCurrent(generation);
       final count = metadata['count'] as int;
       final restored = initialLocator as EpubReaderLocator?;
@@ -162,7 +172,7 @@ final class EpubReaderEngine extends ReaderEngine {
       if (!identical(candidate, _worker)) candidate?.dispose();
       throw ReaderException(
         ReaderErrorCode.invalidDocument,
-        'The EPUB document is invalid or malformed.',
+        'The text document is invalid or malformed.',
         cause: error,
       );
     } finally {