When the mmap() system call is made, the kernel generates a structure to represent this allocated memory in the Virtual Memory Area (VMA). The structure vm_area_struct contains various items for flags and properties for the memory section.
VMAs were previously managed by red-black trees. Now, they use
Maple Trees (a B-tree data type for storing non-overlapping ranges). The reason for this change is that the Maple Trees are
RCU safe.
Read Copy Update (RCU) is a pattern to make reads memory safe even if they are being copied or updated. This allows for read to occur concurrently to writes. Within a maple tree, a maple node either represents a VMA or a gap; there shouldn't be a gap between two intervals.
When during concurrent modification, there are restrictions put in place; an exclusive lock is required by all writers. For reading, the MM read-write lock can be taken, but results in slower times. Instead, a read can be performed on the section without taking the lock, which is done in performance critical situations.
On a stack expansion, the memory automatically grows. This requires editing the VMA maple tree to add this change. In some situations, the operation can be performed atomically, meaning that it's not a problem. But, when the neighboring node has the MAP_GROWSDOWN flag, this operation CANNOT be performed atomically. When the gap needs to be removed (aka removal of the node), a new node must be created instead of simply altering the old one. This results in the old node being destroyed in a RCU callback.
The RCU callback is invoked only after all pre-existing RCU critical sections have concluded. When accessing VMAs with the MM read lock held, it does not enter the critical RCU section. Why is this bad? The callback can be invoked at any time, resulting in a use after free when attempting to access it once again.
According to the author, this was exploited in the locked down KCTF environment. This means it's exploitable on nearly all systems. Additionally, this claims to be the first use after free by RCU bug that's happened. Super interesting post and an extremely deep bug.