This is a continuation of a previous blog post on Solana fuzzing. They found two bugs in the node software via fuzzing and this post is about triaging the bugs.
The first bug they found was an intricate memory leak. In Solana, the Instruction Meter is used to keep track of the amount of gas a program has left remaining. The meter is only checked at the end of a block, which should be fine.
However, the ordering of operations matters here. During a previous security fix, the ordering was changed.
If an out of gas exception occurred at the same time that a unknown symbol error happened, then the latter error was simply overwritten. As a result, the string was left along on the heap without a reference to it. Although Rust provides memory safety, it doesn't guarantee that all memory is cleaned up. By default, this was only seven bytes, which was not a huge deal.
The string that gets allocated comes from the loaded ELF file - it's the name of the function. Since this is controllable, providing a very large name. The most we can do is 1M bytes being allocated. Using this in a loop on Solana over and over again will lead to the system being running out of RAM and crashing.
The second bug was easy to trigger but hard to diagnose. The JIT terminated with Ok(0) but the interpreter returned with an access violation error. Juicy! Looks like there's an out-of-bounds write in the JIT somewhere.
The violation was at 0x100000000 specifically. This memory address stores some read-only data present in the ELF provided. In order to find the bug, they used GDB to see when and why this was being written to. The program has access and bounds checks in it. Why is this failing?
The instruction was being executed was cmp DWORD PTR [rax+0x19], 0x0. This is surprising because it's not an 8-bit operand!? Why!? The x86 instruction uses the opcode 0x81 but only for 16, 32, and 64-bit register operands. If you want to compare the 8-bit version, you must use the 0x80 opcode instead. This leads to the CPU performing an incorrect comparison and using unintended values around it. Neat!
By breaking the comparison for the write using an 8-bit value, we can write a single byte to the read-only memory section. At first, the author thought this could be used to compromise the execution of a program. However, this only allows for modifying your own program and not others. Instead of the 1M they were hoping for, they got 100K for this bug (and the last one).
Overall, it is a fantastic series! I enjoyed the Oracle approach to finding bugs that weren't the typical "fuzzier crashed" situation. Both bugs were relatively complicated and deep, yet not too many lines of code. Good strategy!