Hey everyone! I'm working on a university project where I want to analyze different subreddits and how users interact over time. I'm specifically looking to see if users who start in one subreddit eventually migrate to another. I want to focus on posts rather than comments (but would consider comments if they're easy to include). I'm interested in pulling a list of every post from the last 5 years, including the title, username, and exact posting date.
I've tried using PRAW and ChatGPT, but I'm only able to retrieve the last 1000 posts, which seems to be a limit with PRAW. I also found something called "easy-reddit-downloader" on GitHub, but it also cuts off at around 800-1000 posts. I read that Reddit heavily limits API access, so I'm wondering if it's possible to collect more than the latest 1000 posts. Any ideas on tools or methods I could use? Thanks in advance!
1 Answer
Are you looking at specific subreddits or all of them in general? That might affect how you can gather your data. Also, if you’re focusing on a couple of subreddits, setting them as your input should help refine your search results. You might want to check if there are any paid APIs that can give you access to more extensive data.
Yeah, narrowing it down to specific subreddits should help, and there might be some APIs that charge for more extensive data access. Definitely worth researching!