I'm building a small application with mostly static data, although new records will be added over time. For each verification request, I only need to check whether a requested value exists. Would it be a good idea to load every record into memory when the server starts so requests avoid querying the database? The application may run on a single VPS, possibly with Redis or Memcached in Docker. If a new record is added, I could refresh or invalidate the cache. If a value is missing from the cache, I would treat it as not found rather than querying the database.
4 Answers
If the data is truly suitable for static delivery, you could periodically generate files and serve them through object storage or a CDN, while keeping the server only for operations that write new records. That avoids managing an in-memory cache, though it may be more infrastructure than you need for a small VPS-based application.
A Redis or Valkey-backed set can be a good fit for existence checks: add a key when a record is created and test membership during verification. Just make sure writes update both the database and the cache reliably, decide how to handle failures, and remember that a cache-only lookup can return a false negative if the cache is stale or has been lost.
It can work if the complete dataset comfortably fits in memory and you only run one server instance, but cache invalidation is the difficult part. Every write must update or invalidate the relevant cached data, and a restart requires rebuilding it. Also measure before adding this complexity—an indexed database lookup may already be extremely fast because the database caches frequently used pages.
Rather than preloading everything at startup, use a cache only if profiling shows the database is a bottleneck. Keep the database as the source of truth and update or invalidate the relevant cache entry whenever a record changes. A startup-warming process is extra code to maintain and usually isn’t worthwhile unless loading the data is genuinely expensive.
Would refreshing the cache whenever a new record is added be enough? In this case, a cache miss should mean “not found,” so I don’t want to query the database on every miss.