Testing extensions

Every bug worth catching in an extension hides behind a successful HTTP request. The endpoint answers, the JSON parses, your mapping code runs, the array comes back full, and the source is still broken. This page is about testing the behaviour rather than the status code, plus the two harness options and their limits.

Test only against sites you are authorised to access, and keep your request volume low while you iterate. A test loop that fires a search fifty times a minute is a load test of someone else's server, and it can get your connection blocked.

A 200 response proves nothing #

A 200 means the server was willing to answer. It says nothing about whether it answered the question you asked. Every one of the following was found in a real extension that returned status 200 on every call:

• The page parameter was accepted and ignored. The API paged on offset, not page. Requesting ?page=2 returned exactly the same twenty items as ?page=1, with a 200 each time. Browse looked fine until you scrolled and the same covers appeared again.
• has_more was true forever. The listing endpoint reported another page on every response, including page 90 of an 89-page catalogue. Because the extension trusted that flag to build its metadata, infinite scroll never terminated and kept re-requesting the same empty tail.
• Sort values were accepted but changed nothing. Four Browse feeds were advertised, each passing a different sort= value. All four returned 200 and all four returned identical results in identical order. The site only supported one ordering, and silently ignored the parameter.
• The chapter list was capped at 100. No limit parameter was sent, so the API applied its default page size. A 300-chapter series showed 100 chapters, the newest ones, with nothing to indicate that the rest existed.
• A table of contents became 135 fake chapters. A book's plain text contained a contents listing whose lines matched the same "CHAPTER n" pattern as the real chapter headings, so the splitter found the sequence 1 to 135 twice and produced 270 chapters, half of them a single line long.

None of these produce an exception, a rejected promise, or a non-2xx status. They are only visible if you assert on the content.

The behavioural checklist #

Run these against a real source before publishing. Each is a claim about data, not about transport.

• Page 2 does not overlap page 1. Collect the ids from both and intersect them. The intersection must be empty. This is the single highest-value assertion in the list.
• Pagination terminates. Walk pages until metadata comes back undefined. Cap the loop at, say, 30 pages, and fail if you hit the cap. A source that never stops is a source that loops forever in the app.
• Every advertised feed genuinely reorders results. Compare feeds against each other, not against nothing. Fetch page 1 of each feed from getSourceFeeds() and assert that the id lists are not identical. If two feeds return the same order, either drop one or fix the parameter.
• Covers are absolute URLs. Assert every image starts with http. Relative paths render as an empty card.
• Chapter times are epoch milliseconds. Assert typeof time === 'number' and that the value converts to a plausible date. A Date object or an ISO string is silently discarded by the host.
• Status maps to a known value. getMangaDetails must return one of ONGOING, COMPLETED, ENDED, HIATUS, ABANDONED, CANCELLED. Anything else displays as Unknown.
• getChapters returns the whole list. Cross-check the count against the series page on the site itself. Round numbers like exactly 100, 50 or 20 are the tell that a default page size is in play.
• Ids round-trip. Take a mangaId from search results, feed it to getMangaDetails and getChapters, then feed a returned chapter id to getChapterDetails. Extensions that build ids from one endpoint's shape and parse them for another break exactly here.
• Run the whole suite twice. The second run is what exposes rate limiting, IP throttling and Promise.all fragility. A source that fans out twenty concurrent requests usually passes cold and fails warm, and a user browsing for two minutes is the warm case.

A quick Node harness #

For a JSON API, a Node script that stubs the two host globals is fast and good enough. Node 18 or newer has fetch built in.

// harness.js  ->  node harness.js ./gutenberg/index.js
globalThis.App = {
  createRequest: (o) => o,
  createRequestManager: () => ({
    async schedule(request) {
      const res = await fetch(request.url, {
        method: request.method || 'GET',
        headers: request.headers || {},
        body: request.data,
      });
      return { status: res.status, headers: Object.fromEntries(res.headers), data: await res.text() };
    },
  }),
  createSourceStateManager: () => ({ retrieve: () => undefined, store: () => {} }),
};

const { Source } = require(require('node:path').resolve(process.argv[2]));
const src = new Source();
const browse = (metadata) => src.getSearchResults({ title: '', feed: 'popular', includedTags: [] }, metadata);

(async () => {
  const p1 = await browse(null);
  const p2 = await browse({ page: 2 });
  const ids1 = p1.results.map((m) => m.mangaId);
  const ids2 = p2.results.map((m) => m.mangaId);
  console.log('page 1:', ids1.length, 'page 2:', ids2.length);
  console.log('OVERLAP (must be 0):', ids2.filter((id) => ids1.includes(id)).length);
  console.log('relative covers (must be 0):', p1.results.filter((m) => !/^https?:/.test(m.image || '')).length);

  const details = await src.getMangaDetails(ids1[0]);
  const info = details.mangaInfo || details;
  console.log('status:', info.status, 'summary chars:', (info.desc || '').length);

  const chapters = await src.getChapters(ids1[0]);
  console.log('chapters:', chapters.length, 'first time:', typeof chapters[0].time, new Date(chapters[0].time));

  const body = await src.getChapterDetails(ids1[0], chapters[0].id);
  console.log('pages:', (body.pages || []).length, 'text chars:', (body.text || '').length);
})();

Extend it with the feed comparison and the pagination walk, and you have a real suite in under a hundred lines.

Why Node is not a faithful substitute #

Node is a different JavaScript host with a much larger set of globals than the app's sandbox. Code that passes here can still throw a ReferenceError on the first line that runs on device. URLSearchParams is the recurring one: it exists in Node, but Boundless does not shim it in, so it fails on device exactly like URL, TextDecoder, Buffer, atob, btoa, DOMParser, structuredClone, setTimeout, and every node: module, all of which work in the harness and all fail in the app.

For a JSON API extension where you can visually confirm the file has no browser or Node globals in it, the Node harness is fine. For an extension with long parsing code, where the temptation to reach for a convenience global is constant, run it inside the app before publishing instead, on a real device or an emulator for every platform you claim to support. That proves the file runs where users will run it, and a quirk that only exists on one platform only shows up on that platform.

curl is not the app #

If curl https://example.com returns 403 or an interstitial page, that is not on its own evidence the source is unusable. curl sends a different set of headers and a different TLS handshake than the app's networking stack, so a site can answer the two differently. Test the request the way the app will make it before writing a source off.

A challenge page, a block page or a login wall means the site's operator is controlling who gets in. Do not build a source that tries to get around one. If a site does not want automated access, that is the answer: ask the operator for permission, or choose a different catalogue.

A challenge page can also arrive with status 200 and an HTML body, so status checks alone will not detect one. If you suspect a challenge, assert on something structural in the response body, such as the number of items your selector found being greater than zero.