Multi-Threaded Memory Consistency Models: Sequential Consistency vs. Weak Ordering

MULTI-THREADED MEMORY CONSISTENCY MODELS: SEQUENTIAL CONSISTENCY VS. WEAK ORDERING In multi-threaded systems running on multi-core processors, threads execute concurrently across different CPU sockets. To optimize performance, modern hardware and compilers perform out-of-order execution, which can cause threads to see memory updates in different orders. Managing this behavior requires understanding memory consistency models. SEQUENTIAL CONSISTENCY (SC) Defined by Leslie Lamport, Sequential Consistency requires that the result of any execution is the same as if the operations of all processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program. Under SC, if Thread A writes to variable X and then writes to Y: Thread A: X = 1; Y = 1; Thread B reading these values must never see Y = 1 while X still holds its old value (0). SC maintains strict program order. However, enforcing SC requires disabling write buffers and compiler optimizations, causing significant memory latency penalties. WEAK ORDERING AND INSTRUCTION REORDERING To maximize throughput, modern CPUs (such as x86 and ARM) implement Weak Ordering. The hardware allows memory reads to bypass pending memory writes, and the compiler reorders instructions if it determines there is no local data dependency. For example, on a weakly ordered CPU: Thread A: X = 1; // Write Y = 1; // Write Thread B: if (Y == 1) { print(X); // Might print 0! } Because there is no dependency between X and Y on Thread A, the CPU or compiler may reorder the writes, or Thread B's CPU may read X from its local cache before the update propagates across the interconnect bus. MEMORY BARRIERS AND SYNCHRONIZATION To prevent invalid reordering, systems programmers must insert Memory Barriers (fences). A memory barrier is a hardware instruction that forces the CPU to complete all pending memory read/write operations before executing subsequent instructions, ensuring state consistency across cores. Reference: E. Balagurusamy, "Fundamentals of Computers", Chapter 2: Central Processing Unit & Internal Communications.