How to Scrape Data from Modern JavaScript E-commerce Sites?

I'm a relatively new developer and I'm working on building a tool to extract historical product data from a client's e-commerce site. The main goal is to retrieve information like price, availability, variants, and descriptions from their product pages to update older records accurately.

However, I'm running into a challenge: the data displayed in my browser is vastly different from what my scraper receives. When I load the page in a normal browser session, everything appears fine—the JavaScript executes, components mount, and the API calls resolve, resulting in a fully populated page.

But my scraper doesn't function like a browser; it only processes the initial HTML response and what I'm getting back is mostly an empty shell. I'm missing key data like price, variants, and availability, which are only present after the JavaScript executes or user interactions occur.

Here are some specific issues I'm facing:
- Price and inventory are stored in JavaScript state only.
- Variants only load after user interactions.
- Descriptions can be injected after component mount.
- Important relationships exist visually but not in the markup.

Now, I'm exploring various solutions:
- Should I run a headless browser despite the performance hit?
- Is it better to intercept underlying API calls rather than parsing the HTML?
- Should I search for embedded JSON or data hydration scripts?
- Could I push for server-rendered or pre-rendered endpoints where feasible?

Before I over-engineer my approach, how have others successfully tackled similar challenges extracting structured data from modern JS-heavy e-commerce sites?