Race conditions are issues that stem from the timing of requests being made. Historically, most race conditions were time of check vs. time of use (TOCTOU). For instance, redeeming a gift card multiple times by submitting the requests simultaneously. The author calls these limit overrun attacks. However, there are other TOCTOU issues besides limit overruns.
James Kettle introduces a new class of race condition bugs beyond limit overrun attacks: state machine races. Best quote of the article: "Every pentester knows that multi-step sequences are a hotbed for vulnerabilities, but with race conditions, everything is multi-step." By considering every operation that the backend performs a state change, new requests between these changes could cause major havoc by accessing unintended substates.
The author was testing an application where the flow was Login->Assign Role to token. So, what happens prior to the role being assigned to the user? It's an admin user! So, by making a request while the role assignment process was occurring, the user could briefly become an administrator. This is a perfect example of a sub state causing problems.
These race conditions can be extremely tight though. How do we hit these? Network jitter between two requests can cause particular conditions to not work. A common technique is the
last byte sync, where the final byte is not send until the very end.
HTTP/2 allows for
multiple requests to go over a single packet. This was abused for
Timing Attacks in the past. Now, James Kettle was abusing this to get the network jitter to be extremely low. On top of this, they combined the
last byte sync from before in order to get 20-30 requests interpreted all at the same time. This is implemented into Burp Intruder but they have more details on how to do this yourself as well.
Finding these types of vulnerabilities is extremely difficult to do. So, the author came up with a methodology for finding these bugs. First, we need to identify objects, such as users and sessions, with security controls that that we want to bypass. Then, identify functionality that reads or writes to this state, such as profile edits, password resets and more. Finally, race these various types of requests to find a problematic sub-state.
The big questions to be ask are:
- How is the data stored? Persistent server side state data is what should be tested here.
- What's the operation keyed on?. Finding weird sub-states may only be possible in situations where two items share the same key or the state has various operations that change in a single request.
- Editing or appending. Editing is much more prune to race conditions.
Of course, Kettle went through some case studies. While looking through email invitations functionality, sending the same invitation request 6 times turned up 5 successes and 1 "this has already been taken" message and two emails instead of 1. This was different than doing this sequentially. So, the weirdness of this call indicates that a weird sub-state has been hit.
Initially, this did not gather much. However, they came up with a crazy attack scenario. Doing the email resolution process, what if you replaced the email? The email resolution would verify it while the replacement would swap the email. Sadly, the code being sent was for an already verified account and not the new one trying to be added.
During the Gitlab process of changing an email, the author noticed that it was incredibly fast; much faster than sending an email is. So, this indicating that another thread was likely being used to send this. The author decided to change their email address twice at the same time. While doing this, they noticed that the To and body did not always match. Sometimes, the confirmation code for one user was included in the other users email! What a crazy bug!
Testing for these bugs is complicated via blackbox but impossible to spot via whitebox testing. There are many many many bugs like this waiting to be discovered in the wild. I hope to find some of these in the near future via blackbox testing. Race conditions thrive on complexity.