Resources

People often ask me "How did you learn how to hack?" The answer: by reading. This page is a collection of the blog posts and other articles that I have accumulated over the years of my journey. Enjoy!

How we found and fixed a rare race condition in our session handling - 427

Dirkjan Bussink - GithubPosted 5 Years Ago
  • Recently, Github had a security report that a couple of users had been logged in as the wrong users. This sounds absurd but this is what happened.
  • After reviewing all infrastructure and code changes, they came to the conclusion that only the response of the request for the session cookie was wrong. How could this be though?
  • One of the recent code changes moved the logic to check a user's enabled features into a background thread. Additionally, error reporting from the background thread to the main thread would show information about the data from both threads. Although this was not a direct issue, it's still very odd.
  • In Unicorn, the underlying Rack HTTP server on the Rails application, the ENV object is the same for reach request (global variable). This global variable is changed between threads via the Hash#clear function. This, combined with the error reporting led to the sharing of data across requests.
  • If an exception occurred at just the right time and if concurrent request processing happened in just the right sequence across multiple requests, Github ended up replacing the session in one response with a session from an earlier response.
  • Github worked with Unicorn to have new requests use their own ENV hashes. Additionally, they removed the threading issue that had existed in the code.
  • I really enjoy the transparency of Github to explain an issue that occurred! It makes me trust them more now that they take security seriously.