Serve the build tool and a library bundle per content type at /bundles/

README.md

@@ -8,6 +8,7 @@ The player origin of Embed My: what a snippet points at. The site people use to
 | `/h5p?src=<package url>` | The player page the snippet frames: the player alone, driven by the query string. Upward it speaks the resizer protocol, and posts one `report` on what it learnt about the package, or the `error` that stopped it |
 | `/h5p-resizer.js` | The sizing script the snippet's second line names |
 | `/build` | The build page: a source an AI chat wrote with the [h5p-author skill](https://github.com/missing-elements/h5p-offline-player/tree/main/skills/h5p-author) becomes a `.h5p` file, built in the browser, played on the page and downloaded from it. `/build#source=<base64url>` carries a source in the link |
+| `/bundles/` | For a chat that runs Node and fetches files but reaches no npm registry: the build tool as one file, a bundle of libraries per content type, every library part on its own, their licences and an `index.json` (see *The bundles*) |
 | `/samples/*.h5p` | The three demo packages the site offers (CC0 and CC BY), and `quiz-without-libraries.h5p`, the quiz as H5P.com would export it, which the browser tests play |
 
 It has no custom domain, on purpose. A GitHub Pages address has nothing to renew and nothing anyone
@@ -72,8 +73,8 @@ than English. The pack's licence list is emitted as
 | `h5p.html`, `src/h5p-page.ts` | The player page: `@missing-elements/h5p-embed`'s embed page, started with this origin's options |
 | `build.html`, `src/build-page.ts`, `src/build-source.ts`, `src/build.css` | The build page: the page's wiring, and what it does with a source with no DOM (the link's fragment, the files a source names, the text for the chat) |
 | `public/samples/` | The sample packages, copied into the build as they are |
-| `e2e/`, `playwright.config.ts` | The browser tests: the page plays a sample, with and without its libraries, a framing page gets its height and the report, a bad link gets the error; the build page builds, plays and downloads a pasted source, one from a link and one with a file of its own, and lists a source's problems for the chat |
-| `vite.config.ts` | The page, its Content-Security-Policy, `h5p-resizer.js` and the licence files, copied from their packages |
+| `e2e/`, `playwright.config.ts` | The browser tests: the page plays a sample, with and without its libraries, a framing page gets its height and the report, a bad link gets the error; the build page builds, plays and downloads a pasted source, one from a link and one with a file of its own, and lists a source's problems for the chat; `/bundles/` lists and serves every bundle and part, and its tool and a bundle build a quiz in an empty folder, a part named by the build fetched for math |
+| `vite.config.ts` | The page, its Content-Security-Policy, `h5p-resizer.js` and the licence files, copied from their packages, and `/bundles/`, written from them |
 
 The page gets a `Content-Security-Policy` in a `<meta>` tag at build: GitHub Pages sends no headers, so the tag is
 the only policy there is. The dev server gets none, because it injects scripts and styles of its own. The browser
@@ -102,6 +103,28 @@ web address gets it only where that host allows CORS.
 The page's address has no "H5P" in it, as H5P Group's trademark policy asks of a name that does
 not say who made it; its heading uses the word descriptively.
 
+## The bundles
+
+`/bundles/` is for a chat that runs Node and can fetch a file but reaches no npm registry, as
+ChatGPT's sandbox can: its download tool takes a file the browsing tool has opened. There the
+h5p-author skill fetches two files, the build tool and the bundle of the source's content type,
+unzips the bundle and builds with `node h5p-author.mjs build source.json course.h5p --libraries
+parts`. The chat then sees every problem itself, fixes its source and hands over the package; the
+build page stays for a chat that can fetch nothing.
+
+| Path | What it holds |
+|---|---|
+| `/bundles/<type>.zip` | One per content type in the pack, `H5P.QuestionSet.zip`: `parts/`, the pack's whole index with the parts a build of that type can read, and `libraries.txt`. Written by `h5p-author bundle --all` |
+| `/bundles/parts/<part>.h5p` | Every part of the pack under its own name, and `parts/index.json`. No bundle holds an addon: a source whose text calls for one, H5P.MathDisplay for math, stops the build, which names the part to fetch from here |
+| `/bundles/h5p-author.mjs` | The author package's command line as one file, everything it imports bundled in by Vite, so it runs with nothing installed. It needs `--libraries`, and has no `--version`: the npm package's own files are not with it |
+| `/bundles/index.json` | The author package's and the pack's versions, and each bundle's file, size and number of parts |
+| `/bundles/libraries.txt` | The pack's licence list, which asks to travel with its libraries |
+
+`vite.config.ts` writes them after the pages, from the packages the lockfile pins, so a
+Dependabot update of `@missing-elements/h5p-author` or `@missing-elements/h5p-libraries` moves
+them, and they live in `dist/` only, not in git: 113 MB with pack 1.1.0, well within GitHub Pages'
+1 GB. A question type's bundle is about 1 MB, a Question Set's 2.6 MB, a container's up to 8.4 MB.
+
 ## Deploy
 
 A push to `main` runs `.github/workflows/deploy.yml`: build, browser tests, then `dist/` goes to GitHub Pages.

e2e/bundles.spec.ts

@@ -0,0 +1,88 @@
+import { execFile } from 'node:child_process'
+import { mkdtemp, rm, writeFile } from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { promisify } from 'node:util'
+import { expect, test, type APIRequestContext } from '@playwright/test'
+
+/*
+ * /bundles/ as a chat's sandbox uses it (#14), following the h5p-author skill's steps: the command
+ * line and one content type's bundle fetched into an empty folder outside this project, so nothing
+ * installed here is found, unzipped and built with Node alone; and an addon the text calls for,
+ * fetched as its own part once the build has named it.
+ */
+
+const exec = promisify(execFile)
+
+type Listing = {
+  author: string, pack: string, tool: string, parts: string, licences: string,
+  bundles: Array<{ type: string, version: string, file: string, size: number, parts: number }>
+}
+
+const META = { language: 'en', license: 'CC BY', authors: [{ name: 'Ann Author', role: 'Author' }] }
+
+/** A file of the site saved into the folder, as a chat's download tool saves it. */
+async function fetchTo(request: APIRequestContext, path: string, file: string): Promise<void> {
+  const response = await request.get(path)
+  expect(response.ok(), path).toBe(true)
+  await writeFile(file, await response.body())
+}
+
+/** The command line run in the folder, with the Node running the tests. */
+const tool = (cwd: string, ...args: string[]) => exec(process.execPath, ['h5p-author.mjs', ...args], { cwd })
+
+test('lists a bundle per content type, and serves each, every part on its own, the tool and the licences', async ({ request }) => {
+  const listing = await (await request.get('/bundles/index.json')).json() as Listing
+  expect(listing).toMatchObject({ author: expect.stringMatching(/^\d+\.\d+\.\d+$/), pack: expect.stringMatching(/^\d+\.\d+\.\d+$/), tool: 'h5p-author.mjs' })
+  expect(listing.bundles.length).toBeGreaterThan(40)
+  expect(listing.bundles.map(({ type }) => type)).toEqual(expect.arrayContaining(['H5P.QuestionSet', 'H5P.Blanks', 'H5P.CoursePresentation']))
+  for (const { file, size } of listing.bundles) {
+    const response = await request.head(`/bundles/${file}`)
+    expect(response.ok(), file).toBe(true)
+    expect(Number(response.headers()['content-length']), file).toBe(size)
+  }
+
+  const index = await (await request.get(`/bundles/${listing.parts}index.json`)).json() as { libraries: Array<{ folder: string }> }
+  expect(index.libraries.length).toBeGreaterThan(100)
+  for (const { folder } of index.libraries) expect((await request.head(`/bundles/${listing.parts}${folder}.h5p`)).ok(), folder).toBe(true)
+  expect(await (await request.get(`/bundles/${listing.licences}`)).text()).toContain('H5P.QuestionSet-')
+  expect(await (await request.get(`/bundles/${listing.tool}`)).text()).toMatch(/^#!\/usr\/bin\/env node\n/)
+})
+
+test('builds a quiz in an empty folder from the tool and its bundle, and takes an addon as its own part', async ({ request }) => {
+  const dir = await mkdtemp(join(tmpdir(), 'embed-my-bundles-'))
+  try {
+    await fetchTo(request, '/bundles/h5p-author.mjs', join(dir, 'h5p-author.mjs'))
+    await fetchTo(request, '/bundles/H5P.QuestionSet.zip', join(dir, 'H5P.QuestionSet.zip'))
+    await exec('unzip', ['-q', 'H5P.QuestionSet.zip'], { cwd: dir })
+
+    const quiz = {
+      ...META,
+      type: 'H5P.QuestionSet',
+      title: 'Rivers',
+      params: {
+        questions: [
+          { library: 'H5P.MultiChoice', params: { question: 'Which river is the longest in Europe?', answers: [{ text: 'Volga', correct: true }, { text: 'Danube', correct: false }] } },
+          { library: 'H5P.TrueFalse', params: { question: 'The Rhine flows into the North Sea.', correct: 'true' } }
+        ]
+      }
+    }
+    await writeFile(join(dir, 'quiz.json'), JSON.stringify(quiz))
+    const { stdout } = await tool(dir, 'build', 'quiz.json', 'quiz.h5p', '--libraries', 'parts')
+    expect(stdout).toContain('Built ')
+    const manifest = JSON.parse((await exec('unzip', ['-p', 'quiz.h5p', 'h5p.json'], { cwd: dir })).stdout) as { mainLibrary: string }
+    expect(manifest.mainLibrary).toBe('H5P.QuestionSet')
+
+    // Math in the text calls for H5P.MathDisplay, which no bundle holds: the build names its part.
+    const squares = { ...quiz, title: 'Squares', params: { questions: [{ library: 'H5P.Blanks', params: { questions: ['The square of 3, $$3^2$$, is *9*.'] } }] } }
+    await writeFile(join(dir, 'squares.json'), JSON.stringify(squares))
+    const failed = await tool(dir, 'build', 'squares.json', 'squares.h5p', '--libraries', 'parts').then(() => null, (error: { code?: number, stderr?: string }) => error)
+    expect(failed?.code).toBe(1)
+    const part = /has no (\S+\.h5p)/.exec(failed?.stderr ?? '')?.[1]
+    expect(part, failed?.stderr).toMatch(/^H5P\.MathDisplay-\d+\.\d+\.h5p$/)
+    await fetchTo(request, `/bundles/parts/${part}`, join(dir, 'parts', part!))
+    expect((await tool(dir, 'build', 'squares.json', 'squares.h5p', '--libraries', 'parts')).stdout).toContain('Built ')
+  } finally {
+    await rm(dir, { recursive: true, force: true })
+  }
+})

package.json

@@ -20,7 +20,7 @@
     "vite": "^8.3.5"
   },
   "dependencies": {
-    "@missing-elements/h5p-author": "^0.5.1",
+    "@missing-elements/h5p-author": "^0.6.0",
     "@missing-elements/h5p-embed": "^1.7.0",
     "@missing-elements/h5p-libraries": "^1.1.0",
     "@missing-elements/h5p-offline-player": "^1.5.1",

vite.config.ts

@@ -1,6 +1,10 @@
 import { createHash } from 'node:crypto'
 import { readFileSync } from 'node:fs'
-import { defineConfig, type Plugin } from 'vite'
+import { copyFile, mkdir, readFile, writeFile } from 'node:fs/promises'
+import { basename, join, resolve } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { writeBundles } from '@missing-elements/h5p-author'
+import { build, defineConfig, type Plugin } from 'vite'
 
 /*
  * The player origin, embed-my.github.io: the page the snippet frames (`/h5p`), the sizing script
@@ -82,8 +86,78 @@ function packageFiles(): Plugin {
   }
 }
 
+/**
+ * `/bundles/`, for a chat that runs Node and can fetch a file but reaches no npm registry, as in
+ * ChatGPT's sandbox (#14). The h5p-author skill builds there from two files fetched from here: the
+ * command line, and the bundle of the source's content type.
+ *
+ * - `<type>.zip`, one per content type in the pack (`h5p-author bundle --all`): `parts/`, the
+ *   pack's whole index with the parts a build of that type can read, and `libraries.txt`.
+ * - `parts/`: every part of the pack under its own name, and its index. A bundle holds no addon,
+ *   so a build whose text calls for one stops and names its part, which the chat fetches from here.
+ * - `h5p-author.mjs`: the command line as one file, everything it imports bundled in by Vite, so it
+ *   runs with nothing installed. It takes `--libraries`: the npm package's pinned pack is not with
+ *   it, nor the package.json `--version` reads, so `index.json` says the versions.
+ * - `index.json`: the author's and the pack's versions, and each bundle's file, size and parts.
+ * - `libraries.txt`: the pack's licence list, which asks to travel with its libraries.
+ *
+ * Written after the pages, from the packages the lockfile pins: `dist/` holds them, git does not.
+ */
+function bundles(): Plugin {
+  const root = fileURLToPath(new URL('.', import.meta.url))
+  const author = new URL('./node_modules/@missing-elements/h5p-author/', import.meta.url)
+  const libraries = new URL('./node_modules/@missing-elements/h5p-libraries/', import.meta.url)
+  const version = async (from: URL) => (JSON.parse(await readFile(new URL('package.json', from), 'utf8')) as { version: string }).version
+  let outDir = ''
+  return {
+    name: 'embed-my:bundles',
+    apply: 'build',
+    configResolved(config) {
+      outDir = resolve(config.root, config.build.outDir)
+    },
+    async writeBundle() {
+      const folder = join(outDir, 'bundles')
+      const written = await writeBundles(folder)
+      const parts = join(folder, 'parts')
+      await mkdir(parts)
+      const index = JSON.parse(await readFile(new URL('parts/index.json', libraries), 'utf8')) as { libraries: Array<{ folder: string }> }
+      for (const { folder: part } of index.libraries) await copyFile(new URL(`parts/${part}.h5p`, libraries), join(parts, `${part}.h5p`))
+      await copyFile(new URL('parts/index.json', libraries), join(parts, 'index.json'))
+      await copyFile(new URL('libraries.txt', libraries), join(folder, 'libraries.txt'))
+      await build({
+        configFile: false,
+        root,
+        logLevel: 'warn',
+        publicDir: false,
+        build: {
+          ssr: fileURLToPath(new URL('bin/h5p-author.mjs', author)),
+          outDir: folder,
+          emptyOutDir: false,
+          copyPublicDir: false,
+          minify: false,
+          reportCompressedSize: false,
+          rolldownOptions: { output: { entryFileNames: 'h5p-author.mjs', format: 'es', codeSplitting: false } }
+        },
+        ssr: { noExternal: true, target: 'node' }
+      })
+      const listing = {
+        author: await version(author),
+        pack: await version(libraries),
+        tool: 'h5p-author.mjs',
+        parts: 'parts/',
+        licences: 'libraries.txt',
+        bundles: written.map(({ output, type, parts, size }) => {
+          const [machineName, typeVersion] = type.split(' ')
+          return { type: machineName, version: typeVersion, file: basename(output), size, parts: parts.length }
+        })
+      }
+      await writeFile(join(folder, 'index.json'), `${JSON.stringify(listing, null, 2)}\n`)
+    }
+  }
+}
+
 export default defineConfig({
-  plugins: [cspMeta(), packageFiles()],
+  plugins: [cspMeta(), packageFiles(), bundles()],
   build: {
     rollupOptions: {
       input: { h5p: 'h5p.html', build: 'build.html' }