I've been experimenting with Perplexity AI and trying to scrape its responses for various queries. No matter what approach I use, like headless browsers or rotating proxies, I keep running into obstacles. Does anyone have a solid workflow or tips for scraping Perplexity AI effectively?
3 Answers
Have you considered just using their API? It's way easier and more reliable than scraping. Check out their documentation for a smooth start!
If you're looking for something more specific in how Perplexity displays its results, you might need a specialized scraper like the one from Oxylabs that can mimic real user queries.
Thanks for the tip! I’ll definitely look into that Oxylabs scraper.
Many AI companies want genuine research to be done and might provide access or useful data for that purpose. It might be worth reaching out to them instead of scraping.
I agree, the API is the way to go. Scraping could get your IP banned, and with the API, you get structured data directly.