This article is Part II of the C++ Concurrency and Parallel Programming series. Part I covered the execution model, thread lifecycles, mutexes, condition variables, and futures as a result channel; this installment continues with atomics and memory order, C++20’s cancellation and coordination facilities (stop_token, semaphore, latch, barrier), the engineering practices that take code from “correct” to “maintainable and measurable,” and finally a shutdown-capable concurrent task processor that computes squares.
To keep cross-references consistent, chapter and example numbering continues from Part I: this installment begins with Chapter 8 and Example 15, and references to Part I material are marked “Part I.”
8. Atomics: Establish the Boundaries Before Discussing Memory Order
8.1 A Single Atomic Operation Is Not an Atomic Business Transaction
std::atomic<T> provides operations that cannot be subdivided for eligible types. Integer atomics commonly use load, store, exchange, and fetch_add. fetch_add returns the value prior to modification and performs the read, add, and write as a single atomic read-modify-write.
The commonly used operations are listed below. The corresponding overloads of load, store, exchange, fetch_*, and CAS let you specify a memory order explicitly; when it is omitted, the default is seq_cst. Operator forms always use seq_cst and accept no memory-order argument. Lock-free capability queries do not involve memory order (§8.3):
| Operation | Semantics | Returns |
|---|---|---|
load() | Atomically read the current value | The value read |
store(v) | Atomically write a value | None |
exchange(v) | Atomically replace the value | The previous value |
fetch_add(n) / fetch_sub(n) | Atomic addition / subtraction (eligible integer and pointer types; C++20 also supports floating-point specializations) | The value before modification |
operator++ / operator+= etc. | Shorthand for the corresponding fetch operation | Prefix ++ returns the value after modification; postfix ++, like fetch_add, returns the value before modification |
compare_exchange_weak/strong(expected, desired) | Replace with desired only if the value still equals expected (§8.7) | Whether it succeeded; on failure expected is updated to the observed value |
is_lock_free() / is_always_lock_free | Query whether this object / type has a lock-free implementation | The answer at runtime / compile time |
The assignment operator performs a store with the default memory order and returns the value written, while implicit conversion performs a load with the default memory order; consequently count = count + 1 still involves two separate operations, a read and a write, with the same problem as the faulty snippet below. C++20 also offers atomic wait / notify_one / notify_all; their absence in older TDM-GCC environments is noted in Part I §2.2, and this article avoids them to keep the examples coherent. The specification for types and operations appears in the C++20 draft’s atomic generic types.
Example 15: Atomic counting with the default memory order.
| |
Faulty example—do not copy it (snippet; count is atomic):
| |
There is no data race on count here, yet updates can be lost: both threads load the same value and then store the same new value. Use fetch_add or ++count instead.
Multiple atomics likewise do not automatically form a transaction. If two account balances are each atomic and you debit A before crediting B, an observer may see the total decrease between the two steps—even when every operation uses the default memory order. Guarding the shared invariant with a lock is usually easier to maintain than designing a multi-variable lock-free protocol.
Atomic objects, too, must be constructed before publication and destroyed only after no one accesses them. atomic<T> is not available for arbitrary T and does not guarantee lock-free operation for every type; you can check is_lock_free() or is_always_lock_free. Some types and targets require linking libatomic explicitly; the small-integer examples here have no such requirement.
8.2 happens-before: The Links in a Correctness Proof
The memory model is not a programming interface for “flushing some level of cache.” It specifies which executions and observations are permitted; the compiler and hardware are responsible for realizing them.
First, distinguish three relationships:
- sequenced-before: Within a single thread, the ordering established by the rules of evaluation; you cannot infer the order of just any expression from the way it is written.
- synchronizes-with: A cross-thread synchronization edge designated by the standard, such as from an unlock to a subsequent lock, or from thread completion to a join returning.
- happens-before: The ordering formed by joining intra-thread sequencing with cross-thread synchronization. This article does not use
consume, so you can reason along these links and their transitive closure.

Figure 8-1: A complete synchronization chain (using the publication in Example 16 as an illustration). In thread A, sequenced-before links “write payload” to the release store; the release and the acquire load that reads that value form a synchronizes-with edge; adding the sequenced-before within thread B yields “write payload happens-before read payload.” The synchronization edge between an unlock and a subsequent lock works the same way.
Modification order is the total order formed by all modifications to a given atomic object; every thread must respect that object’s atomicity and coherence constraints. Distinct atomic objects each have their own modification order, and these do not automatically combine into a single global order—a property that holds even with relaxed.
Seeing an operation “appear to have happened first” in real time does not establish happens-before; sleep_for(), log output, and the number of CPU cores are none of them the proof you need. Lock-free code in particular cannot infer cross-platform correctness from the observed behavior of a single x86 machine.
8.3 What Each Memory Order Is Responsible For
| Memory order | Main role | Good starting point |
|---|---|---|
seq_cst | Retains the corresponding acquire/release effects and provides a single total order over all seq_cst operations, subject to the standard’s constraints | The default choice |
release | Publishes prior operations within a matching synchronization relationship | A store that publishes a ready state |
acquire | Receives published operations within a matching synchronization relationship | A load that observes a ready state |
acq_rel | Both acquires and releases | A read-modify-write that needs synchronization in both directions |
relaxed | Guarantees this atomic access and the object’s modification order, without publishing other data | Standalone counters with no publication duty |
consume | Propagates publication along data dependencies, with rules more intricate than acquire | Discouraged by the standard since C++17 (formally deprecated in C++26); ordinary code should avoid it |
C++20 defines six memory orders in all, whose full names are std::memory_order_relaxed, std::memory_order_consume, std::memory_order_acquire, std::memory_order_release, std::memory_order_acq_rel, and std::memory_order_seq_cst. consume has long been difficult to implement and use correctly; mainstream implementations usually treat it as acquire, and it is formally deprecated in C++26, so this article avoids it. load cannot use release or acq_rel; store cannot use consume, acquire, or acq_rel.
Release and acquire synchronize only when they actually connect: they must operate on the same atomic object, and the acquire must read the value written by the corresponding release—or a value from the release sequence that begins with it. Any two operations bearing those two words do not automatically pair up.
This article uses a single publication to avoid such complexity. In C++20, a release sequence is the longest contiguous subsequence in an object’s modification order that begins with a release operation and is followed entirely by atomic read-modify-writes; inserting an ordinary atomic store truncates it, whereas a plain load is not a modification and does not truncate it. The older textbook claim that “an ordinary atomic store by the same thread can extend the sequence” no longer applies (proposal P0982R1 deliberately narrowed this definition—a breaking change to the old rule, so code that relied on the removed extension loses its synchronization guarantee). The precise rules appear in C++20 memory order.
8.4 Proving acquire/release with a Single Publication
Example 16: Publishing ordinary data after initialization.
| |
The proof has three steps:
- The write to
payloadis sequenced-before the release store within the thread. - The initial value is false and only the producer writes true, so an acquire that reads true must have observed that publication.
- That acquire is sequenced-before the read of
payload, linking the whole into a happens-before chain, so the ordinary integer’s write and read do not race.
Faulty example—do not copy it (a snippet showing an incorrect replacement for Example 16):
| |
The atomic flag is safe in itself, but it no longer establishes the synchronization that publishes the data; the conflicting accesses to the ordinary payload lack happens-before and produce undefined behavior. “Working most of the time” on some processor cannot repair an error at the C++ level.
The weaker memory orders here are used to expose the synchronization edge, not as measured performance advice; keeping the default seq_cst is equally correct. When real waiting times may be long, use a condition variable; busy-waiting and yield promise neither fairness nor bounded response time. C++20 also offers atomic wait/notify, whose absence in older toolchains is recorded in Part I §2.2; this article continues to use condition variables for a uniform waiting protocol.
You cannot turn a one-shot publication protocol into repeated transfers simply by clearing ready back to false: the producer might overwrite the data while the consumer is still reading the old value. Multi-round handoffs need an additional acknowledgement, mutual exclusion, or a queue protocol.
8.5 What seq_cst Guarantees—and What It Does Not
If a program has no data races and all relevant atomic operations use seq_cst, you can reason as if all atomic operations were interleaved in a single global order consistent with each thread’s sequencing. This total order is not a lock around the whole function, does not guarantee that several loads form a snapshot at one instant, and says nothing about real-time scheduling or about reading the “latest” value at some wall-clock moment.
For example, with x and y initially 0 and both threads using seq_cst:
| |
You cannot get rA == 0 and rB == 0 at the same time: that would require each load to be ordered before the other thread’s store, forming an impossible cycle together with the intra-thread ordering. With relaxed, such an outcome is permitted; and even with release stores and acquire loads, reading the initial 0 fails to connect to the other thread’s publication, so the outcome still cannot be ruled out.
Once weak memory orders are mixed with seq_cst, you can no longer apply the “global interleaving” picture to every operation; the formal constraints are finer. Without a concrete performance requirement, keeping the default order is easier to prove correct than weakening operations piecemeal. seq_cst will not repair unprotected ordinary data accesses, dangling references, or a business operation that has been split apart.
8.6 When relaxed Is Already Enough
If Example 15 needs only an independent count whose final value is read after all threads join, you can change its fetch_add and final load to relaxed: the atomic read-modify-write prevents lost updates, join handles synchronization after thread completion, and the counter itself publishes no other business data.
This is a correctness argument, not a promise of speed; whether the change is worthwhile requires measurement. Do not generalize it into “once the count reaches N, you can directly read N ordinary results”: if you treat the counter as a readiness signal, data publication needs a separate proof.
8.7 CAS: Replace It If It Has Not Changed
CAS stands for compare-and-swap and is provided in C++ by compare_exchange_weak/strong. Understanding expected is the key: it is both an input and a value that may be modified.
| Outcome | Atomic object | expected |
|---|---|---|
| Success, returns true | Write desired | Retains the value used for this comparison |
| Failure, returns false | No write performed | Updated to the atomic value just observed |
weak permits spurious failure: it may report failure even when the values compare equal, which is why it is usually placed in a loop. strong has no such spurious failures, but it still fails when another thread changes the value. A CAS comparison does not call a custom operator==; this article uses simple integers to avoid complications with exotic representations.
Example 17: Decrement stock only when it is greater than zero.
| |
The output is 100; how much each thread gets is nondeterministic. A successful CAS is the only moment at which a decrement takes effect: the write of expected minus one happens only while the value still equals a positive expected, so the stock never goes negative.
Faulty example—do not copy it (snippet):
| |
This can write a stale result after contention, producing a logical error—usually not a data race on the atomic stock. The fix is to recheck the precondition and recompute desired on every round, as in Example 17. The retry body must not perform non-repeatable side effects such as payments or notifications; those belong after the operation succeeds.
When you supply explicit memory orders, CAS has a success order and a failure order: success is a read-modify-write, failure is only a read, so the failure order cannot be release or acq_rel. The old precondition that “the failure order must not be stronger than the success order” was removed in C++17 and should not be treated as a C++20 rule; the success order applies to the read-modify-write, while the failure order must still be a valid order for a read. In practice, use a combination you know and have proved correct, such as acq_rel/acquire or release/relaxed; here the defaults suffice and no manual pairing is needed. The single-order overload adjusts the failure order automatically: acq_rel maps to acquire and release maps to relaxed. The precise rules appear in C++20 atomic operations.
CAS does not imply that the whole algorithm is lock-free, still less that a given thread cannot starve. If a value goes A→B→A, a CAS may not notice the intervening change—the ABA problem. With pointers you must also handle object reclamation and dangling accesses. Do not jump from this integer example straight to writing your own lock-free linked list.
A common remedy for ABA is the tagged pointer: compare and update the pointer together with a version number as a single atomic object. When the value goes A → B → A, CAS detects the change as long as the version has not wrapped. Tags have finite bits and can wrap, so you must prove that a tag is not reused while an older observation is still in use. Updating a plain counter and an atomic pointer separately does not provide this guarantee; the availability of wide CAS, whether it is lock-free, and pointer-bit encoding all depend on the implementation, so do not assume 64-bit machines support it automatically.
Detecting ABA and managing object lifetime are separate matters. Ownership through shared_ptr can extend an object’s lifetime, but the shared-pointer variables themselves still require synchronization, and reference counting cannot be generalized into a cure for every logical ABA. Hazard pointers prevent premature reclamation by publishing the node being accessed and rechecking it; epoch reclamation waits until old visitors have passed a safe epoch before freeing nodes. Both require a complete reclamation protocol and neither automatically resolves A → B → A in arbitrary business state. This section only delineates the boundaries; it does not implement a lock-free list.
8.8 std::atomic_flag and the Limits of Spin Locks
std::atomic_flag provides flag operations guaranteed to be lock-free. test_and_set() atomically sets the flag to true and returns its previous value; clear() sets it to false. The following correct snippet shows only the locking protocol (with <atomic>, <mutex>, and <thread> already included):
| |
In use, manage it with a named std::lock_guard<SpinLock> so that exceptional paths cannot forget to unlock. While the holder is preempted, waiters keep burning CPU, and heavy contention also generates cache-coherence traffic. Single-core systems, excessive threads, slow I/O, long critical sections, and blocking operations all make this kind of spin lock a poor fit; a plain std::mutex is usually better. C++20’s atomic_flag::test() can observe the flag without changing it; atomic waiting is a different implementation of waiting, whose support should be probed on the target toolchain.
9. C++20 Cancellation and Coordination Facilities
9.1 Cooperative Cancellation: Request, Check, Exit Safely
Cancellation does not mean forcibly cutting a thread short from the outside. A thread may be holding a lock, updating several fields, or managing a file resource; killing it abruptly would leave a half-finished state.
C++20 provides a set of handles over a shared stop state:
stop_source: owns the ability to issue a stop request, viarequest_stop().stop_token: observes the request viastop_requested(); can be copied to related tasks.stop_callback: registers a callback to run when a request occurs.
A jthread carries its own stop source. If the thread function can accept an injected token, the jthread passes it as the first argument. A request is one-shot: you cannot reset the same stop state back to “no request”; a fresh round of work normally needs a fresh state.

Figure 9-1: The data flow of cooperative cancellation. A request is written into the shared stop state; a task polls the token at safe points or responds passively through a stop_callback. Both paths require the thread to do its own cleanup before exiting.
Long computations should check the token at sensible safe points, for example after processing each batch. Checking too rarely hurts responsiveness; checking too often can add overhead. Plain sleep_for(), blocking I/O, and condition_variable::wait() do not wake up merely because the token has been requested.
Stop-handle interfaces at a glance:
| Interface | Purpose | Notes |
|---|---|---|
stop_source::request_stop() | Issue a stop request to the shared stop state | Returns true only if this call is the first to issue the request; false if no state is associated or a request was already made |
stop_source::get_token() | Obtain a stop_token that observes the state | Can be copied and distributed to any number of tasks or callbacks |
stop_token::stop_requested() | Query whether the associated state has received a stop request | A request_stop() that returns true synchronizes with a subsequent associated query returning true; a false query establishes no such edge and does not guarantee no request follows |
stop_token::stop_possible() | Whether a request has been made, or could still arrive via an associated source | False when there is no state, or when no request has been made and no stop_source remains associated; merely holding a state via a token is not enough |
stop_callback<Fn>(token, fn) | Execute fn once when the state is requested | If already requested, runs immediately in the registering thread; otherwise may run synchronously in the requesting thread, see the caveats in §9.2 |
The synchronization rules for stop queries appear in C++20 stop state, and the possibility rules in stop_token members. None of this means a query protects an entire business transaction, or that the thread has exited once a request is made; reaping the thread still requires join.
9.2 Waiting Threads Must Be Able to Exit Too
condition_variable_any works with lock types that satisfy the BasicLockable requirements; it is used here mainly so that a wait can also respond to a stop request. C++20 provides three interruptible wait overloads, listed below; in each, the stop_token immediately follows the lock argument:
| Overload | Wait boundary |
|---|---|
wait(lock, token, pred) | The condition holds or a stop request arrives |
wait_for(lock, token, rel_time, pred) | Adds a relative timeout budget |
wait_until(lock, token, abs_time, pred) | Adds an absolute deadline |
All three require a predicate; there is no wait(lock, token) overload without one. Each returns the latest evaluation of the predicate before returning (a bool), not an enumeration of the wake-up reason, and reacquires the lock on return. Example 18 uses business state to decide whether to keep working or exit.
Example 18: Cancelling a thread that is waiting—or has not yet begun to wait.
| |
The output is cancelled. work_ready stays false throughout; only the worker writes cancelled, and the main thread reads it after join. When a stop request and the predicate are both true, the wait result is still decided by the predicate; a real queue must specify whether to “drain pending work first” or give “stop priority”—you cannot infer business policy from a single bool. The definition of interruptible waits appears in C++20 condition_variable_any.
A stop_callback may run synchronously in the thread that calls request_stop(); if a stop was already requested at registration time, the callback runs immediately in the registering thread before registration returns. Destroying a callback can also block waiting for that callback to finish executing in another thread, and destroying it while holding a lock the callback needs can deadlock. Keep callbacks short and never let them throw (an exception escaping a callback calls std::terminate()), and do not request a stop while holding a lock the callback requires, or you may deadlock. Prefer the standard wait overloads above, which avoid hand-assembled callback protocols prone to lost notifications.
A jthread’s destructor requests a stop and then joins, but it can still hang when: the function never checks the token; the thread is blocked on an operation that does not support cancellation; or exiting requires a lock currently held by the destructor. With multiple interdependent jthreads, you should generally request a stop from every thread first, then join them one by one, rather than relying on sequential destructors to untangle mutual waits.
9.3 semaphore: Holding a Count of Permits
Condition-variable notifications do not accumulate; a semaphore holds a permit count. acquire() waits while the count is zero, otherwise decrementing it; release() adds permits and gives waiters a chance to proceed. binary_semaphore suits handoffs with at most one permit, while counting_semaphore<N> handles multiple permits—N is a lower bound on the required maximum, and should not be misread as an exact ceiling for every implementation.
A semaphore has no thread ownership as a mutex does, so one thread may acquire and another release. It suits limiting how many tasks use a resource concurrently or signalling how many items are available; it does not automatically protect an arbitrary container structure. You must not release beyond the allowed maximum count, waiters are not guaranteed fairness, and try_acquire() permits spurious failure.
counting_semaphore<N> interfaces at a glance (binary_semaphore is counting_semaphore<1>):
| Interface | Purpose | Notes |
|---|---|---|
Constructor counting_semaphore(n) | Create with n initial permits | Requires 0 <= n <= max(); neither copyable nor movable |
acquire() | Decrement the count by one; block until available if the count is zero | Synchronizes with the corresponding release when the acquire observes its effect; initial permits correspond to no release, so the data handoff in Example 19 relies on a subsequent release |
try_acquire() | Try to decrement without blocking, returning success | Spurious failures allowed: may return false even when the count is positive; on false no permit was taken |
try_acquire_for(d) / try_acquire_until(t) | Wait for a permit for a bounded duration / until a time point | Returns false on timeout; a timeout is not a real-time return guarantee |
release(update = 1) | Add update permits | Precondition: update >= 0 and the count after release does not exceed max(); violating this is undefined behavior; any thread may release |
max() (static) | Return the maximum count supported by the implementation | At least the template parameter N; N is only a “minimum ceiling” and the actual maximum may be larger |
Example 19: Handing off three values through a single slot. This example requires a C++20 standard library that implements <semaphore>.
| |
The expected output is 6. The two directions of handoff prevent “reading before the write completes” and “overwriting before the read completes,” respectively, keeping the count between 0 and 1. One semaphore cannot express both ownership transfers. Multi-producer, multi-consumer queues also involve shared state such as slot indices, which should be guarded by a separate lock.
For real resource permits, use a scope guard to ensure they are returned even on exceptions; here the slot holds only integer operations that throw no business exceptions. The standard rules appear in C++20 semaphore.
9.4 latch: A One-Shot Completion Set
A latch is a countdown latch: count_down() signals that one piece of work is done without waiting, wait() blocks until the count reaches zero without decrementing it, and arrive_and_wait() decrements first and then waits for zero. The count cannot be reset, and once it reaches zero waiters pass. You must not decrement below zero.
latch interfaces at a glance:
| Interface | Purpose | Notes |
|---|---|---|
Constructor latch(n) | Create with count n | Requires 0 <= n <= max(); one-shot, not resettable or copyable |
count_down(update = 1) | Decrement by update without waiting | Writes a thread completed before its own count_down are visible to threads unblocked when the count reaches zero; precondition update >= 0 and not greater than the current count, otherwise undefined behavior |
wait() | Block until the count reaches zero | Does not consume the count; passes immediately if already zero |
try_wait() | Non-blocking query of whether the count is zero | Returns a bool without waiting; the standard permits false with very low probability (spurious failure) |
arrive_and_wait(update = 1) | Decrement, then wait for zero | Equivalent to count_down(update) followed by wait() |
max() (static) | Return the maximum count supported by the implementation | A capability hint only |
Example 20: Waiting for two results to be ready. This example requires a C++20 standard library that implements <latch>.
| |
The expected output is 42. A latch waits for a progress point, not necessarily thread exit, so threads still need reaping. This example does not modify the results after decrementing. If a task exits with an exception before calling count_down, the wait may continue forever; failure handling must be part of the application protocol.
9.5 barrier: Repeatable Phase Boundaries
A barrier suits computations with a fixed set of participants working in rounds. Each participant calls arrive_and_wait(), and all proceed only once every participant for that phase has arrived. You may supply a non-throwing completion function to perform reductions or similar work at phase boundaries; do not wait inside it for work that can only complete in the next phase.
barrier interfaces at a glance:
| Interface | Purpose | Notes |
|---|---|---|
Constructor barrier(n, f) | Create with n expected participants and completion function f | Requires 0 <= n <= max(); the completion type must satisfy is_nothrow_invocable_v<CompletionFunction&>, so a lambda should be explicitly noexcept; omitting means a no-op; a zero-participant object may only be destroyed |
arrive() | Report arrival and decrement the expected count for this phase, without waiting | Requires 0 < update <= remaining count for this phase; returns an arrival token and is marked [[nodiscard]] |
wait(token) | Wait for completion of the phase associated with the token | The token must come from the current or immediately preceding phase of the same barrier |
arrive_and_wait() | Arrive and wait for this phase to complete | Equivalent to wait(arrive()), the most common rendezvous form |
arrive_and_drop() | Arrive and reduce the expected count for current and subsequent phases | Formally exit subsequent phases, replacing “leaving without telling anyone” |
max() (static) | Return the maximum expected count supported by the implementation | A capability hint only |
Example 21: Two participants, three rounds of filling and reading. This example requires a C++20 standard library that implements <barrier>.
| |
The expected output is 18. The second barrier cannot be dropped casually: otherwise the worker might already be writing the next round’s values[1] while the main thread still reads the previous one, creating a data race. The barrier’s phase synchronization and participant constraints appear in C++20 coordinating types.
arrive() only reports arrival, decrements this phase’s count, and returns an arrival token without waiting for other participants; you can then do work that does not depend on the phase’s completion and later call wait(std::move(token)) to await that phase. The token must come from the current or immediately preceding phase of the same barrier. arrive_and_wait() combines the two steps. The completion function runs in a participating thread chosen as the standard permits—you cannot assume it is always the main thread; the end of the completion step strongly happens-before the return of any wait call unblocked by that phase’s completion, so phase results can be delivered safely.
Participants cannot leave unannounced; when a participant will not take part in later phases, use arrive_and_drop() as the protocol requires to reduce the expected count for the current and subsequent phases. A stop request does not automatically remove a participant from a barrier, so cancellation and phase exit must be designed together.
9.6 Choosing Among Them
| Need | First consideration |
|---|---|
| A shared rule across several fields | A mutex and RAII locks |
| Wait for dynamic conditions such as “queue non-empty or closed” | A condition_variable and a shared predicate |
| Deliver one result or exception per task | future with promise / packaged_task / async |
| Limit how many tasks use a resource at once | semaphore |
| Wait for a group of work to reach a single completion point | latch |
| Wait for all participants each round | barrier |
| Ask a task to finish at safe points | stop_token and an explicit exit protocol |

Figure 9-2: Comparing the semantics of semaphore / latch / barrier—permit counting, one-shot countdown, and repeatable phase barriers.
These facilities solve different problems; there is no need to use every one in every project.
10. From a Correct Program to a Maintainable, Measurable One
10.1 Task Decomposition Matters More Than Thread Count
Identify independent work first, then choose the execution resources. Prefer tasks that only read their inputs and write outputs they own, merging at the end. Multiple threads modifying different elements of a plain array is usually fine, but they must not change the container’s structure at the same time; the bit-packed representation of vector<bool> is an exception that deserves special vigilance.
std::thread::hardware_concurrency() returns a hint about the number of hardware threads that can run concurrently—possibly the logical core count, possibly 0—and says nothing about the current quota or the optimal thread count. Provide a sensible fallback for 0 and cap the thread count at the number of meaningful tasks. Empty jobs should simply return.
When you need per-thread state, thread_local gives each thread its own instance of a variable, eliminating sharing at the source: typical uses are per-thread accumulators, caches, or random-number engines. Instances are destroyed as their threads exit. It is a tool for avoiding sharing, not for making shared data safe—handing a pointer or reference to a thread_local object to another thread still requires you to handle synchronization and the lifetime of the owning thread’s exit. In a thread pool, its lifetime follows the thread rather than the task, so successive tasks reuse the same state. For one-shot tasks like Part I’s Example 5, an ordinary local variable inside the lambda is enough; if you use per-thread accumulators, you must explicitly hand a copy to the reducer before the thread exits—the main thread cannot read another thread’s instance through a thread_local variable of the same name.
For CPU-bound work, start measuring with a number of worker threads close to the available hardware resources; heavy blocking I/O may call for a different strategy. The C++20 standard library offers no general-purpose asynchronous network I/O or ready-made thread pool, and you cannot solve every scenario by creating threads without limit.
10.2 Why More Threads Can Be Slower
| Cost or limit | Effect | Common remedy |
|---|---|---|
| Thread creation and reaping | Small tasks never amortize the management cost | Reuse a small pool of workers; submit in batches |
| Context switching | Scheduling, stack, and cache-state switches have a cost | Cap the thread count; avoid waves of useless wake-ups |
| Lock contention | Threads queue for the same critical section | Shorten critical sections; accumulate locally then merge; reduce sharing |
| Tasks too fine-grained | Enqueue and synchronization overhead dominates | Coalesce adjacent small tasks |
| Tasks too coarse-grained | A few long tasks hold up the tail | Chunk sensibly or use a dynamic task queue |
| Memory bandwidth | Concurrent reads/writes across cores can hit a bandwidth ceiling | Improve access patterns; avoid redundant copying |
| Oversubscription | More active threads than the hardware suits, intensifying contention | Avoid each nested parallel layer creating its own full set of threads |
False sharing occurs when different threads modify different variables that happen to share a cache line, increasing cache-coherence traffic. It may involve no data race yet still hurt performance. Per-thread local accumulation, less frequent writes, and padding hot data apart when necessary can all help.
Cache locality refers to the tendency to reuse recent or adjacent data. Giving each thread a contiguous range usually helps locality more than chasing pointers at random, but the actual gain must be measured. Blindly padding every object inflates memory usage and can damage locality; likewise, do not assume the cache line size is always 64 bytes.
10.3 Common Design Patterns
| Pattern | How data and execution are organized | Fits and limits |
|---|---|---|
| Partition then reduce | Each thread handles an independent range; results are merged after reaping | Inputs must be splittable and results combinable; watch for load imbalance |
| Producer–consumer | A queue connects submitters and processors | Needs a close protocol; a bounded queue also provides backpressure |
| Fixed worker pool | A small set of threads loops over tasks | Avoids per-task thread creation; design the close and wait dependencies carefully |
| Pipeline | Stages hand off through queues | Stages run at different rates; buffer growth must be bounded |
| Single owner / message passing | One path of execution modifies state; others send messages | Reduces shared writes; watch the serial stage becoming a bottleneck |
| Immutable snapshot | Read-only after publication; updates create a new version | Simple read path; still needs safe publication and lifetime management |
A resource-exhaustion deadlock occurs when every worker in a pool is waiting on subtasks queued in that same pool that have not started; no mutex is required for this. Prefer waiting in a coordinator thread, or avoid such nested dependencies.
Backpressure means limiting the submission rate or queue capacity when processing cannot keep up, rather than accumulating tasks without bound. The task processor in §11 has a finite input; it uses an unbounded queue first to keep the core protocol clear, then discusses bounded extensions.
10.4 The Boundaries of std::execution
Since C++17, some standard algorithms accept execution policies such as std::execution::par, but a policy is not a command to “use N cores.” An implementation may choose to execute serially; parallel implementations may also need an extra back end—some libstdc++ configurations use TBB—so the default compile command cannot be promised to yield parallel execution. This article does not cover back-end configuration or demonstrate parallel speedup. See libstdc++’s execution-policy linkage requirements.
| Policy | Execution relationship of user functions |
|---|---|
std::execution::seq | Executes in the calling thread, with no interleaving among invocations; promises no particular element order |
std::execution::par | Allows multiple threads; invocations on the same thread do not interleave, and everything may still run serially |
std::execution::par_unseq | Allows multiple threads, and invocations on the same thread may also be unordered and interleaved |
std::execution::unseq | C++20: allows unordered execution in the calling thread, e.g. vectorization |
Callbacks in parallel invocations must not modify shared state without protection. Under par you can guard shared access with a correct mutex protocol, but you cannot wait for “another iteration to run concurrently,” since the implementation may serialize. par_unseq and unseq forbid calling standard-library functions that are unsafe for vectorization, such as locking a mutex; interleaving on the same thread can also deadlock. Atomic operations must still meet vectorization-safety requirements, so an arbitrary atomic is not a drop-in replacement for blocking synchronization. Execution-policy specification
For the standard-defined execution policies, if an exception escapes a user function called by the algorithm, the standard requires std::terminate(); this includes overloads passed seq explicitly, which is not an exception-propagation model like async. An allocation failure inside the algorithm may throw std::bad_alloc, which is distinct from an exception escaping a user function.
10.5 How to Verify and Measure
Correctness checks come before performance testing: first write down ownership, lock-protected ranges, and exit conditions, then run small deterministic result checks, vary the task count, exercise error paths, and repeat runs. Logging changes scheduling, so it can only help locate problems—it cannot prove an ordering.
On supported platforms, ThreadSanitizer can detect some data races. It cannot prove the absence of logical races, deadlocks, or dangling-lifetime problems; you can build separately in a supported GCC/Clang environment, for example:
| |
This diagnostic command is not part of the default compilation checks run here, nor is it supported on every platform; the official TSan manual recommends passing -fsanitize=thread to both compilation and linking, with optimization enabled (-O1 or -O2) together with -g, balancing runtime speed against reports with line numbers. Do not compare performance using speeds measured under the sanitizer.
When measuring performance, use identical inputs and optimization levels, compare against a serial baseline first; either include or explicitly exclude the costs of thread creation, enqueueing, and merging; use the monotonic steady_clock for time intervals; measure repeatedly and inspect the distribution, recording hardware, thread count, and load. Do not print logs inside the timed hot path. The examples in this article do no timing and claim no real speedup; more threads are not automatically faster.
10.6 Concurrent Testing Is a Set of Protocols
Check deterministic results and invariants first, then vary the number of workers, producers, consumers, and tasks. Shared statistics in tests must also be synchronized—let each thread write an exclusive slot, then aggregate after join. Schedule test progress with promise/future, latch, or a lock-protected predicate; sleep_for() only perturbs scheduling and cannot tell you another thread has finished.
Stress tests should record the random seed and inputs: the same seed reproduces the inputs but not necessarily the thread scheduling. Running 100, 500, or more iterations only raises the chance of finding a bug; a program with a data race need not produce a wrong value, and a faulty relaxed publication need not fail on x86. A happens-before proof must be written separately for memory ordering.
Deadlocks, unsatisfied futures, and close failures should be guarded by timeouts in a separate process; do not leave a thread that may block forever inside the grader’s process. Timeout reports should distinguish a wait failure from a build failure. Cover task exceptions, empty work, invalid inputs, repeated closes, repeated construction and destruction, and the acceptance boundary when shutdown and submit race. CMake handles portable builds, CTest registers behavior tests with timeouts, and Python orchestrates compilation, repeated runs, seeds, and grading; assertions are not a complete grading system.
Tool-based detection is supplementary evidence. TSan finds some data races; AddressSanitizer checks some out-of-bounds and use-after-invalid accesses. Build them separately, first probing whether compilation, linking, and running actually work, and report SKIP explicitly when unsupported. Passing tests, clean sanitizer runs, and a correctness proof are three distinct kinds of evidence. Official TSan documentation
11. Complete Example: A Shutdown-Capable Concurrent Squaring Task Processor
11.1 Requirements and Design Constraints
This section implements a small batch processor: the main thread submits integer tasks, a fixed number of workers pull tasks from a shared queue, and compute their squares. Each task returns its own future, and the caller retrieves results in submission order.
It must meet the following requirements:
- Every accepted task is taken exactly once, and computation runs outside the queue lock.
- Inputs are restricted to 0 through 1000; invalid inputs propagate
invalid_argumentthrough the future without killing the worker. - Once submission ends, the main thread closes the queue; no new tasks are accepted, accepted tasks are all processed, and waiters on the empty queue exit.
- Workers are managed by
jthread. If thread creation or submission fails, the queue must still be closed so already-started threads can exit before the exception is propagated. - Test empty input, a single thread, multiple threads, valid and invalid tasks; reject zero workers.
This design uses drain-on-close: closing forbids new tasks but still processes accepted ones. Discarding queued tasks on shutdown is a different contract, requiring an explicit decision about how the futures of discarded tasks are satisfied; it should not be folded into the same ambiguous switch.
11.2 Complete Implementation
Example 22: The task queue and asynchronous results.
| |
The output is all batch checks passed. run_batch is a blocking batch interface: the internal tasks run concurrently, but the function waits for the whole batch before returning, so the returned futures are already ready. The futures are kept to show that each task stores its own success or exception; you cannot call this interface an “asynchronous submission service that returns immediately.”
11.3 Design and Thread-Safety Rationale

Figure 11-1: Components and data flow of the task processor. Submission, task retrieval, and closing all pass through the same queue lock; computation and result delivery happen outside the lock.
Ownership: the main thread holds the result handles; the queue holds unexecuted tasks; a worker gains exclusive execution rights by moving a task out. The lambda captures the integer by value, never referring to a temporary in the caller. packaged_task is non-copyable, preventing a task from being accidentally duplicated.
Shared state: only the task queue and the close flag need a mutex. Every access to these two pieces of state happens under the same lock. The result state is synchronized by the packaged_task/future protocol, so the queue lock need not protect results.
Waiting and synchronization: submission enqueues under the lock and notifies after unlocking. Workers check the predicate under the lock, release it atomically while waiting, and can take a task only after reacquiring it. The close flag is persistent state, so even if close happens before a thread prepares to wait, the thread observes the closed state when it checks under the lock; it then drains remaining tasks, confirms the queue is empty, and exits. Task execution happens outside the critical section, so a slow task cannot hold the queue lock indefinitely.
Draining: a true predicate does not mean exit immediately. Only an empty queue yields false; if closed but tasks remain, the worker keeps taking them. The move and pop under the lock form one protected removal, so each task is handed to exactly one worker.
Lifetime: the normal path joins explicitly; the error path closes first, then lets the destruction of workers reap the threads, and only finally destroys the queue. No join is performed while holding the lock. Here workers do not accept a stop_token, so the stop request issued by a jthread’s destructor does not affect the drain contract; thread exit depends on the queue’s closed state.
Exceptions: invalid inputs are captured automatically by packaged_task, so later tasks still run. Allocation failures during thread creation or submission enter the cleanup path. This example does not try to recover from severe runtime failures such as the synchronization primitives themselves becoming invalid, nor can it guarantee shutdown of arbitrary user tasks that never return; its computation tasks are integer operations with a finite number of steps.
Result order: completion order is nondeterministic, but the main thread stores and reads the futures in submission order, so the output interface does not depend on scheduling. The internal shared state survives queue destruction and is held by the futures until get retrieves the result.
11.4 Directions for Improvement
Keep these correctness foundations in place, then extend according to real requirements:
- Turn the unbounded queue into a bounded one: add a “not full or closed” predicate with matching notifications; closing must wake both producers and consumers.
- For a service that returns immediately, put the queue and workers into a long-lived object with explicit
submit,close, andjoincontracts; never return threads that reference a destroyed local queue. - When adding cooperative cancellation, distinguish “stop accepting and drain” from “cancel unexecuted tasks.” The latter must define the exception or cancellation result for each unexecuted future and have long computations check the stop state at safe points.
- Measure throughput and latency at task granularity before deciding whether to enqueue in batches, reduce wake-ups, or reuse result buffers.
A long-lived service also needs a clear contract: submit enqueueing and the close-state transition use the same lock to fix the acceptance boundary; tasks enqueued before close must complete, while submissions after close are rejected synchronously. shutdown() closes, wakes, and waits for all workers; repeated and concurrent shutdowns require separately serialized reaping—you cannot join the same jthread concurrently. Never join while holding the queue lock.
A worker calling a shutdown that would wait for that same worker is self-waiting; reject such calls and document that pool tasks must not wait on subtasks queued in the same pool. Destruction belongs to a single owner, and external submit/shutdown calls must have finished first; concurrent destruction and member calls are outside the service’s protocol. General tasks can use packaged_task<int()> to stay within this article’s scope: movable callables are allowed, with values and exceptions delivered through the future; drain-on-close requires user tasks to eventually return, while cooperative cancellation is a separate contract.
Do not add priorities, work stealing, and lock-free queues all at once; each increases the state and exit paths you must prove.
12. Summary
Looking back over the whole series, here are engineering practices to check against when writing concurrent code:
- Minimize shared mutable state first; read inputs only, keep outputs independent, merge at the end.
- Default to RAII, use jthread where it fits; never let a thread outlive an object it references.
- For every group of shared data, document the lock that guards it and the business invariant; reads must follow the protocol too.
- Give lock-managing objects a name; keep critical sections short but complete; use a safe locking scheme for multiple locks; do not hold a business lock while waiting on tasks.
- Design condition variables around a shared predicate; on close, change state first, then notify, and reap last.
- Make explicit who satisfies each asynchronous state, who calls get, and when the last reference is released; exceptions must have a recipient too.
- Use seq_cst as the atomic default; before weakening a memory order, write the synchronization proof and confirm a real need to optimize.
- Cancellation must cover both compute and wait paths; do not expect destructors, timeouts, or stop requests to forcibly interrupt work.
- Bound concurrent resources and queue growth; avoid unbounded thread creation and nested waits.
References
When a rule is easy to confuse, consult the C++20 draft directly:
- Lambda capture: by-value, by-reference, init-capture, and capture of the current object.
- scoped_lock: acquiring and releasing multiple locks; do not rely on the unlock order.
- Timeout specification: relative timeouts, clock recommendations, and return delays.
- Interruptible waits: the position of the stop_token parameter and the predicate’s return value.
- async: policies, results, exceptions, and synchronization with completion of the associated thread.
