I've been developing a tool to evaluate various companies, which involves a detailed process. First, I scrape the company website, and some sites have over 500 pages. Next, I extract features using large language models (LLMs) from the data in JSON format. I often narrow down the search using keyword or embedding matching, and in some cases, I have to input batches of these JSONs (sometimes all 500+) to gather the necessary features since the information is usually spread out throughout the webpage. Finally, I use LLMs to evaluate the companies based on certain criteria. Looking back, I see why my bill climbed to $28,000 over three months — I was using this method on over 200 companies. I'm wondering if there's a more efficient way to do this that would allow me to stay within a budget of around $1,000 per month without compromising the quality of the output.
3 Answers
Wow, that's quite a hefty bill! So, just to clarify, was the 28K primarily due to LLM token usage?
Considering you have about 100,000 pages to analyze, summarizing content using smaller models might significantly cut costs. For instance, using a smaller model like GPT-4o-mini for summarization could reduce expenses to just a couple hundred bucks!
I doubt that would help much since the main cost comes from input tokens, which would still be billions even if I summarize.
Just a heads up, your post is kind of vague when it comes to solving your issue, which might suggest it's time to consider hiring a skilled engineer to help. Or you could try reaching out to ChatGPT for more specific advice!
Yes, that's correct.