Testing extensions

Every bug worth catching in an extension hides behind a successful HTTP request. The endpoint answers, the JSON parses, your mapping code runs, the array comes back full, and the source is still broken. This page is about testing the behaviour rather than the status code, plus the two harness options and their limits.

A 200 response proves nothing #

A 200 means the server was willing to answer. It says nothing about whether it answered the question you asked. Every one of the following was found in a real extension that returned status 200 on every call:

• The page parameter was accepted and ignored. The API paged on offset, not page. Requesting ?page=2 returned exactly the same twenty items as ?page=1, with a 200 each time. Browse looked fine until you scrolled and the same covers appeared again.
• has_more was true forever. The listing endpoint reported another page on every response, including page 90 of an 89-page catalogue. Because the extension trusted that flag to build its metadata, infinite scroll never terminated and kept re-requesting the same empty tail.
• Sort values were accepted but changed nothing. Four Browse feeds were advertised, each passing a different sort= value. All four returned 200 and all four returned identical results in identical order. The site only supported one ordering, and silently ignored the parameter.
• The chapter list was capped at 100. No limit parameter was sent, so the API applied its default page size. A 300-chapter series showed 100 chapters, the newest ones, with nothing to indicate that the rest existed.
• A table of contents became 135 fake chapters. A book's plain text contained a contents listing whose lines matched the same "CHAPTER n" pattern as the real chapter headings, so the splitter found the sequence 1 to 135 twice and produced 270 chapters, half of them a single line long.

None of these produce an exception, a rejected promise, or a non-2xx status. They are only visible if you assert on the content.

The behavioural checklist #

Run these against a real source before publishing. Each is a claim about data, not about transport.

• Page 2 does not overlap page 1. Collect the ids from both and intersect them. The intersection must be empty. This is the single highest-value assertion in the list.
• Pagination terminates. Walk pages until metadata comes back undefined. Cap the loop at, say, 30 pages, and fail if you hit the cap. A source that never stops is a source that loops forever in the app.
• Every advertised feed genuinely reorders results. Compare feeds against each other, not against nothing. Fetch page 1 of each feed from getSourceFeeds() and assert that the id lists are not identical. If two feeds return the same order, either drop one or fix the parameter.
• Covers are absolute URLs. Assert every image starts with http. Relative paths render as an empty card.
• Chapter times are epoch milliseconds. Assert typeof time === 'number' and that the value converts to a plausible date. A Date object or an ISO string is silently discarded by the host.
• Status maps to a known value. getMangaDetails must return one of ONGOING, COMPLETED, ENDED, HIATUS, ABANDONED, CANCELLED. Anything else displays as Unknown.
• getChapters returns the whole list. Cross-check the count against the series page on the site itself. Round numbers like exactly 100, 50 or 20 are the tell that a default page size is in play.
• Ids round-trip. Take a mangaId from search results, feed it to getMangaDetails and getChapters, then feed a returned chapter id to getChapterDetails. Extensions that build ids from one endpoint's shape and parse them for another break exactly here.
• Run the whole suite twice. The second run is what exposes rate limiting, IP throttling and Promise.all fragility. A source that fans out twenty concurrent requests usually passes cold and fails warm, and a user browsing for two minutes is the warm case.

A quick Node harness #

For a JSON API, a Node script that stubs the two host globals is fast and good enough. Node 18 or newer has fetch built in.

// harness.js  ->  node harness.js ./gutenberg/index.js
globalThis.App = {
  createRequest: (o) => o,
  createRequestManager: () => ({
    async schedule(request) {
      const res = await fetch(request.url, {
        method: request.method || 'GET',
        headers: request.headers || {},
        body: request.data,
      });
      return { status: res.status, headers: Object.fromEntries(res.headers), data: await res.text() };
    },
  }),
  createSourceStateManager: () => ({ retrieve: () => undefined, store: () => {} }),
};

const { Source } = require(require('node:path').resolve(process.argv[2]));
const src = new Source();
const browse = (metadata) => src.getSearchResults({ title: '', feed: 'popular', includedTags: [] }, metadata);

(async () => {
  const p1 = await browse(null);
  const p2 = await browse({ page: 2 });
  const ids1 = p1.results.map((m) => m.mangaId);
  const ids2 = p2.results.map((m) => m.mangaId);
  console.log('page 1:', ids1.length, 'page 2:', ids2.length);
  console.log('OVERLAP (must be 0):', ids2.filter((id) => ids1.includes(id)).length);
  console.log('relative covers (must be 0):', p1.results.filter((m) => !/^https?:/.test(m.image || '')).length);

  const details = await src.getMangaDetails(ids1[0]);
  const info = details.mangaInfo || details;
  console.log('status:', info.status, 'summary chars:', (info.desc || '').length);

  const chapters = await src.getChapters(ids1[0]);
  console.log('chapters:', chapters.length, 'first time:', typeof chapters[0].time, new Date(chapters[0].time));

  const body = await src.getChapterDetails(ids1[0], chapters[0].id);
  console.log('pages:', (body.pages || []).length, 'text chars:', (body.text || '').length);
})();

Extend it with the feed comparison and the pagination walk, and you have a real suite in under a hundred lines.

Why Node is not a faithful substitute #

Node is a different JavaScript host with a much larger set of globals than JavaScriptCore. Code that passes here can still throw a ReferenceError on the first line that runs on device. URLSearchParams is the recurring one: it exists in Node, does not exist in bare JavaScriptCore, and Boundless installs a shim for it. Anything else in that family does not get a shim. URL, TextDecoder, Buffer, atob, btoa, DOMParser, structuredClone, setTimeout and every node: module all work in the harness and all fail in the app.

For a JSON API extension where you can visually confirm the file has no browser or Node globals in it, the Node harness is fine. For an HTML-scraping extension, where the parsing code is long and the temptation to reach for a convenience global is constant, run it in a real JavaScriptCore context before publishing: an Xcode unit test target, or a small Swift command line tool that creates a JSContext, installs the same host shim the app installs (the request manager bridged to URLSession, plus the App factories), evaluates your file, and calls the methods. That is the only environment that proves the file will run on a device.

curl is not the app #

If curl https://example.com returns 403 or an interstitial challenge page, that is not evidence the source is unusable. curl has a distinctive TLS fingerprint that bot-detection vendors flag on sight, and it sends no browser-like header set by default. The app's requests go out through the platform's native networking stack, which produces a fingerprint indistinguishable from ordinary iOS traffic and is frequently allowed straight through.

Test the request the way the app will make it before writing a source off. Note also that a challenge page usually arrives with status 200 and an HTML body, so status checks alone will not detect one. If you suspect a challenge, assert on something structural in the response body, such as the number of items your selector found being greater than zero.