I need to collect data from a confidential, password-protected website for a work project. Reaching the information requires several selections, such as region → team → month → week → day. I need to visit every day-level page across all regions and copy the page text into a spreadsheet.
I'm wondering whether there's a simple local-only tool, browser extension, or spreadsheet feature that could automate this, or whether learning enough code to do it would take longer than entering everything manually. I'm not very familiar with the technical terminology, so suggestions for what to search for would also help.
3 Answers
Look into web scraping or browser automation tools. Depending on how the site works, a browser extension that extracts table or page data may be enough. Since the information is confidential, choose a tool that runs locally and doesn’t upload the data to an online service.
Microsoft Excel’s Power Query is worth trying. Search for tutorials on “Excel Power Query web scraping.” It can import data from web pages and often lets you repeat the process across multiple URLs or parameters, although deeply nested menus may require some setup.
Before automating it, make sure the site owner or your employer explicitly allows scraping or automated collection. Even with authorized access, a large number of requests could trigger rate limits or get the account blocked. Once permission is confirmed, a small local script or browser automation workflow should be safer than copying everything by hand.
The site is password-protected, and I have access specifically to collect this data. I’ll confirm with the site owner or administrator before running any automation.
That’s helpful—having the right search terms makes this much easier. I’ll compare a few local extensions and automation options.