Best Async-Native Alternative to Celery for I/O-Heavy Workloads?

I'm working on Rhesis.ai, which tests LLM applications using a FastAPI backend along with Celery for task processing. Our workload is quite I/O-heavy as we perform numerous external API calls followed by LLM API queries, like those from OpenAI, to assess outputs.

Currently, we handle parallelization within a Celery task by utilizing asyncio. For example, if we have a test set of 50 tests, we send all requests at once and also evaluate them concurrently. However, we're limited by the number of Celery worker processes, which increases RAM usage.

We're looking for a solution that fully utilizes async capabilities—namely, a setup where tasks can be continuously scheduled onto an event loop without being restricted by a set number of worker processes or threads. FastAPI handles many concurrent requests effectively, but it lacks task queuing.

We considered Dramatiq, but it seems to have similar constraints as Celery, where tasks are executed sequentially by the workers despite being internally async. Ideally, we want a stable and mature library or architecture that can handle this proactive scheduling of tasks on an event loop without waiting for previous tasks to complete. We're wary of using experimental solutions for a core part of our infrastructure.