I'm new to web scraping and I'm curious about how difficult it is to create a scraper that can handle job data from potentially hundreds of different websites. Given my lack of experience, I'd love to hear from anyone who has tackled this before—how feasible is this project really?
1 Answer
Web scraping has become pretty accessible with the libraries available today. However, the tricky part is customizing the scraper for each specific site, since their layouts can change unexpectedly. You'll have to maintain them regularly, which can be a hassle, but starting with the right tools makes it much easier.
I didn't realize I might need to customize for every site! That makes me think this might be more complicated than I thought.