I'm building a personal desktop app that collects download links from several websites and presents them through one shared interface. What backend architecture would keep a change to one site's layout from breaking the rest of the application? I'm especially interested in modular parser designs, reusable templates, and testing strategies for this kind of project.
2 Answers
A plugin-style architecture works well here: put each domain in its own file or package and have it implement a common base class. The core application should know nothing about CSS selectors or site-specific quirks; it should only load plugins, ask which one supports a URL, and process the normalized results. Add clear error messages when expected elements are missing, plus fixture-based parser tests for saved pages. That way, a layout change usually means updating one plugin and its tests rather than changing the downloader, UI, and queue logic.
Use one adapter or extractor per site, with every adapter exposing the same small interface, such as `canHandle(url)` and `extractLinks(url)`. Each module should contain that site’s selectors, parsing rules, and validation, while the UI, download queue, retries, and progress handling stay in shared code. A simple shape could look like `class SiteAdapter { canHandle(url) {} async extractLinks(url) {} }`. Have a registry select the first adapter that can handle a URL, and return a consistent result format such as `{ title, url, filename }`. Keep timeouts and errors scoped to individual jobs so one broken extractor does not stop the entire queue. Saving representative HTML samples and testing each adapter against them also makes site changes much easier to diagnose.
That adapter pattern makes sense. I was mainly unsure how much of the base class and registry to standardize, so the minimal interface and shared result format are exactly what I needed.