I've been trying to access a local government webpage (the West Chester, OH site) using PowerShell's `Invoke-WebRequest`, but I keep running into an "Access Denied" error, even when using common switches like `-UseBasicParsing`. Since there's no login required for this public page, I suspect a cookie or session issue. I've found a temporary solution by using Chrome's Developer Tools to copy the session information, which I'm manually adding to my PowerShell script. However, I'm looking for a more automated way to retrieve these cookie values directly from Chrome for my scripts. Is there a way to do this with PowerShell?
2 Answers
Linking to the website you want to scrape can help others give you better advice. It's tough to diagnose the issue without knowing exactly what the site is doing. That said, if you add the site to your question, others with similar experiences might chime in!
It sounds like you might be dealing with a CSRF issue, even if the page is public. Sometimes websites implement measures that can block automated requests without proper session tokens. Have you considered checking your request headers or any additional anti-bot measures? You might want to inspect the Network tab in Chrome’s developer tools when you access the site directly.
Related Questions
Set Wordpress Featured Image Using Javascript
How To Fix PHP Random Being The Same
Why no WebP Support with Wordpress
Replace Wordpress Cron With Linux Cron
Customize Yoast Canonical URL Programmatically
[Centos] Delete All Files And Folders That Contain a String