False sharing and memory layout

Key points

• CPUs move memory in cache lines, typically 64 bytes - never single fields. Two unrelated variables in the same line are, to the hardware, one thing.
• False sharing: two threads write two different fields that happen to share a line. Each write invalidates the other core copy, so they ping-pong the line between cores at enormous cost.
• The code looks perfectly parallel and correct. The only symptom is that adding threads makes it slower, which is why this is so hard to find by reading source.
• The fix is padding, so hot fields land in different lines. @Contended does it without hand-written padding, though it needs -XX:-RestrictContended outside the JDK.
• This is why LongAdder beats AtomicLong under contention: it spreads counting across padded cells and sums them only when read.
• Object layout matters more generally. The JVM reorders fields to minimise gaps, and compressed oops keep references at 4 bytes below a 32GB heap - which is why a heap just over 32GB can use more memory than one just under.

Example

// FALSE SHARING - a and b almost certainly share one 64-byte line.
class Counters {
    volatile long a;      // thread 1 writes this
    volatile long b;      // thread 2 writes this
}                         // adding the second thread makes it SLOWER

// FIXED by padding them into separate lines.
class PaddedCounters {
    volatile long a;
    long p1, p2, p3, p4, p5, p6, p7;   // 56 bytes of padding
    volatile long b;
}

// Or let the JVM do it.
class Annotated {
    @jdk.internal.vm.annotation.Contended volatile long a;
    @jdk.internal.vm.annotation.Contended volatile long b;
}
// Requires -XX:-RestrictContended for application code.

// The standard-library answer to the same problem:
//   AtomicLong  - one field, every thread CASes it, contention destroys it
//   LongAdder   - padded cells per thread, summed on read
LongAdder hits = new LongAdder();
hits.increment();          // no shared line under contention
long total = hits.sum();   // only here do the cells get combined

/*
 * Inspect real layout rather than guessing - JOL prints it:
 *
 *   java -jar jol-cli.jar internals com.example.Counters
 *
 *   OFFSET  SIZE   TYPE DESCRIPTION
 *        0    12        (object header)
 *       12     4        (alignment gap)
 *       16     8   long Counters.a      <-- same 64-byte line
 *       24     8   long Counters.b      <-- as this one
 */

If adding threads makes a correct parallel program slower, suspect false sharing before you suspect the algorithm.