I'm designing a B2C ecommerce platform where products are linked to specific pages in large PDF catalogs stored in Amazon S3. For example, Product ABC123 might use pages 42–43 of catalog_2026.pdf, while Product XYZ999 uses page 120. I want customers to view or download only the pages relevant to a product instead of transferring the entire catalog, while keeping AWS costs low.
I've looked into linearized PDFs and Fast Web View. They can support progressive loading and HTTP Range requests, but they don't appear to solve the problem completely because a PDF page may depend on objects, fonts, images, shared resources, cross-references, and other data located throughout the file. A page therefore may not map cleanly to one continuous byte range.
What architecture would you recommend for this on AWS? Should I pre-extract individual pages or page groups and store them in S3, generate them on demand with Lambda or another service, or use some other AWS-native approach? I'd appreciate advice on the trade-offs involving storage, request costs, latency, caching, and catalog updates.
5 Answers
Before optimizing around partial downloads, measure the actual catalog size, traffic, and cache hit rate. If the complete PDFs are only a few megabytes and users commonly view the whole catalog, serving the original through CloudFront may be cheaper and much simpler. If the catalogs are large or product pages are requested independently, pre-extracted page or product PDFs are the safer design.
The number of S3 GET requests is not necessarily reduced by a complicated range-based design, and the extra development and operational work can outweigh modest storage savings.
The simplest and usually most practical approach is to preprocess the catalog and create a smaller PDF for each product, or for each commonly requested group of pages. Store those generated PDFs in S3 and keep the object key in your product database. Put CloudFront in front of the bucket for caching and delivery.
S3 storage is inexpensive compared with the engineering complexity of trying to retrieve arbitrary PDF pages directly. You can run the extraction as a batch job when a catalog changes, using a PDF library, a container task, or Lambda if the files and processing time fit Lambda’s limits. This also gives you predictable latency when customers request a document.
A reasonable AWS layout would be: keep the original catalogs in S3, run a PDF-splitting pipeline whenever a new catalog is published, store extracted page or product PDFs in a separate S3 prefix, and record the generated object keys and catalog version in your product database. Deliver the generated objects through CloudFront, using signed URLs or cookies if access must be restricted.
For a small number of requests, generating a file synchronously may work, but avoid making every customer request download and process the full catalog. If you use Lambda, check the package size, temporary storage, memory, execution-time, and concurrency requirements; for larger catalogs, a container-based batch worker may be a better fit.
An S3 Range request only retrieves a specified byte interval; it does not understand PDF pages. PDF cross-reference data and page resources can be distributed throughout the file, so you generally cannot make a reliable page-number-to-byte-offset map and render the result with one range request. Linearization improves startup and progressive viewing, but it still requires a compatible viewer and may require several requests for page data and shared resources.
Unless you have a very specialized requirement, treating the original PDF as a random-access page store will be more complicated than extracting the pages ahead of time.
You can store each page as its own single-page PDF, while retaining the original full catalog as well. Your existing product-to-page mapping can point to keys such as catalog_2026/page_042.pdf. If a customer needs multiple pages, either serve the individual files or generate and cache a combined PDF for that product.
A variation is lazy generation: when a product’s page PDF does not exist, an event-driven worker or Lambda downloads the source catalog, extracts the requested pages, writes the result to S3, and then serves it. This avoids preprocessing rarely viewed products, but you need to handle concurrent requests, retries, cold starts, and the latency of the first request.
Whether eager or lazy generation is better depends mostly on access frequency and how often catalogs change. For popular or predictable products, prebuilding is simpler; for a huge catalog with sparse access, lazy generation plus caching can reduce unnecessary processing.
That approach is especially reasonable when the catalog is large but product-page access is frequent. Even if the extracted pages increase total storage somewhat, the monthly cost is typically tiny compared with building and maintaining a custom PDF range reader.