Does an OpenSearch Ingestion pipeline keep Serverless search capacity from scaling to zero?

0
0
Asked By MellowPine7! On

I'm planning to use RDS for PostgreSQL as the source of truth and OpenSearch Serverless NextGen for search. I'm deciding whether to synchronize the two with an OpenSearch Ingestion pipeline or write to both systems from my API layer.

My understanding is that OpenSearch Ingestion OCUs and Serverless OCUs are billed separately, with no shared capacity. OpenSearch Ingestion has a one-OCU minimum and does not scale to zero, while NextGen indexing and search capacity can independently scale down after about 10 minutes of inactivity.

What I can't determine is whether an idle ingestion pipeline keeps the Serverless indexing capacity active. If the source database has no changes and no bulk requests are sent, should indexing still scale to zero, or does the pipeline generate heartbeats, health checks, index checks, or other background requests that reset the idle timer?

I'd also appreciate confirmation of the billing model and advice on whether a continuously running pipeline makes sense for a low-volume synchronization workload.

3 Answers

Answered By CopperMeadow3 On

For occasional or low-volume synchronization, writing to both systems from the application may be cheaper and simpler than paying for a permanently running pipeline. Use retries and idempotent document updates so a temporary OpenSearch failure can be recovered safely. OpenSearch Ingestion is more compelling when you need transformations, dead-letter handling, continuous database capture, or multiple destinations.

VelvetOrbit6 -

It’s also worth checking whether PostgreSQL can handle the search workload by itself. Full-text search with tsvector and a GIN index covers many keyword use cases, while pg_trgm can help with fuzzy or typo-tolerant matching. Keeping one datastore may be preferable if the workload does not truly require a separate search engine.

Answered By QuirkyLantern8 On

There’s some uncertainty around exactly what background requests an idle pipeline makes, so I wouldn’t rely on the collection scaling to zero while the pipeline is running. A pipeline may maintain connections or perform periodic checks, and any target traffic could affect the collection’s idle timer. In practice, the always-on ingestion OCU is the predictable cost, while the Serverless side should be tested with your exact pipeline and monitored for indexing capacity changes.

Answered By BrightCedar42 On

The ingestion service and the Serverless collection use separate OCU pools and are billed independently. OpenSearch Ingestion has its own minimum running capacity, so a continuously active pipeline does not scale to zero even when there are no records to process. That means you should expect the ingestion floor in addition to any Serverless usage. I’d verify the exact regional pricing, but the two services are not pooled together.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.