Testing concurrent code

Key points

• One passing run proves nothing. Repeat the scenario dozens of times before you believe a concurrency fix.
• Use a CountDownLatch as a starting gun, so every thread reaches the shared code at the same instant.
• Never use Thread.sleep to wait for a result. Await a latch or poll with a timeout, then fail loudly.
• Assert an invariant, such as a final total, not the order of prints. Ordering is not something you can promise.
• Read the result after join or await. Those handoffs give the happens-before edge that makes the value visible.
• Turn the pressure up: more threads than cores, tiny queues, and run the suite on the build machine too.

Example

import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;

public class Main {
    static int broken;                                   // deliberately unsafe
    static final AtomicInteger fixed = new AtomicInteger();

    // Release every thread at the same instant, so the interleaving really happens.
    static void round(int threads, int perThread) throws InterruptedException {
        broken = 0;
        fixed.set(0);
        CountDownLatch start = new CountDownLatch(1);
        CountDownLatch done = new CountDownLatch(threads);
        ExecutorService pool = Executors.newFixedThreadPool(threads);
        for (int t = 0; t < threads; t++) {
            pool.execute(() -> {
                try {
                    start.await();
                    for (int i = 0; i < perThread; i++) { broken++; fixed.incrementAndGet(); }
                } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
                finally { done.countDown(); }
            });
        }
        start.countDown();                               // go
        done.await(5, TimeUnit.SECONDS);                 // happens-before: results are visible
        pool.shutdown();
    }

    public static void main(String[] args) throws InterruptedException {
        int threads = 4, perThread = 20_000, expected = threads * perThread;
        int flaky = 0;
        for (int run = 1; run <= 20; run++) {            // one green run proves nothing
            round(threads, perThread);
            if (broken != expected) flaky++;
            if (fixed.get() != expected) throw new AssertionError("atomic counter lost an update");
        }
        System.out.println("runs          : 20");
        System.out.println("expected      : " + expected);
        System.out.println("unsafe wrong  : " + flaky + " of 20 runs");
        System.out.println("atomic wrong  : 0 of 20 runs");
    }
}

Start together, repeat often, assert the invariant.