I have a basic C11 thread pool that works well for fire-and-forget jobs, but I now want callers to wait for a particular task to finish. Tasks are currently removed and freed after execution, so a waiter can find a task, then race with a worker that completes and frees it before the waiter has initialized or used its completion data. I also worry that the task may finish before the waiter calls cnd_wait(), causing the notification to be missed and the waiter to block forever.
What is the correct synchronization pattern for waiting on an individual task? The current design uses a condition variable in TaskCompletionData, but I am unsure how to safely manage the task's lifetime and avoid missed signals. The task's completion state, result, and any associated synchronization data may need to remain valid until the waiter is finished.
3 Answers
A condition variable is not the state; it only wakes threads so they can recheck state. Give each task a completion record containing a `finished` flag (and optionally the result), protect that record with a mutex, and always wait in a loop:
`mtx_lock(&completion->mutex);`
`while (!completion->finished) cnd_wait(&completion->condition, &completion->mutex);`
`mtx_unlock(&completion->mutex);`
The worker should lock the same mutex, write the result and set `finished = true`, then signal or broadcast, and unlock it. If the task finished before the waiter arrived, the waiter sees the already-true flag and does not sleep. The loop also handles spurious wakeups.
Treat the task handle or completion record as a separately owned object. Do not free it immediately when the worker removes the task from the queue. The worker can mark the record complete and remove/free only the private execution data, while the completion record remains alive until the caller releases its task handle. This prevents both use-after-free and destruction of a mutex or condition variable while another thread may still access it.
Also perform all accesses to the task status and completion flag under the appropriate mutex. Signal while holding that mutex after changing the predicate. A condition variable notification is not queued, so relying on the notification alone is unsafe.
You can also use one pool-wide condition variable if every waiter checks its own task's completion state while holding the task-list mutex. The worker marks the specific task finished under that mutex and broadcasts; each waiter wakes, searches for its task again, and continues waiting until that task is finished. Per-task completion records are usually simpler and avoid repeatedly scanning the whole list, but either design must use a protected predicate and a `while` loop.
There are some additional races in the sample: the worker unlocks the task-list mutex before changing status and executing the task, and the list node is destroyed after the task is freed. Those operations need a clearly defined ownership scheme and consistent locking as well.
The difficulty is that the flag has to identify one task, since the pool can finish unrelated tasks while I am waiting. I also need to keep the completion record alive until the waiter is done using it.