I'm currently working on a project where I scrape Google Maps to find businesses that might be interested in my products. I built a script that takes a keyword, like "Pizzeria," and lists relevant businesses, storing them in a JSON format. Here's the issue: my initial run did 12,000 API calls just for that one keyword, and I'm concerned about the costs as I plan to search for other types of businesses too. I've already used up €200 of my €250 in free credits!
To give you more context, I break the country down into a grid of 64km² and make a call for each grid area. The next step is to do one call for every business in order to find specific keywords, which could easily lead to around 30,000 calls.
I've seen pricing options on Google Cloud Platform ranging from 10k a month (unofficial sources) to Google's official plan at 2M a month for free, but neither of those are activating for me. Any advice would be appreciated!
5 Answers
Have you checked if your country provides public datasets for active businesses? Your script might be over-complicating things.
Honestly, Google's pricing can be overwhelming for a large-scale operation like yours. I switched to using Qoest's scraping API for Google Maps data because their pay-per-use structure is so much more manageable for handling lots of listings.
Your grid division strategy is clever, but making a call for each grid square adds up quickly. You should look into other business databases or local directory APIs that might offer more affordable solutions for bulk data scraping. Sometimes it’s not about optimizing your API usage but rather finding a different, more cost-effective data source.
It sounds like your approach might be causing too many repeated requests for the same data. Instead of calling for each grid, consider narrowing your search or just fetching specific data for businesses that really catch your eye; 12k calls for pizzerias seems excessive!
I'm currently breaking France into grids of 64km² for each call. I’m considering adjusting the grid size or finding new ways to optimize.
Just a heads up, scraping Google Maps like that might violate their Terms of Service. You could end up with your account banned, so make sure you're using a Google account that doesn't have any important data.
That's concerning! I really hope my account stays safe if I proceed with this.
We do have some public data, but not specifically for food licensed businesses. I've seen lists, but they're costly, similar to what I’m doing.