By design, the IEEE Floating Point format has adjacent floats have adjacent integer representations. This seems obvious but is only possible because of the implied 1 in the mantissa and the ordering of the fields. Super cool that the integer representation is fairly similar to the float!
When you increment the mantissa, it increases the value of the float. When the mantissa wraps to zero, this increments the exponent. P
This awesome quirk does have its limitations though. The representation of positive zero is next to the smallest representable positive number, but the representation for negative zero is 2 billion-ish values away. This also means that tiny positive and negative numbers have integer representations which are about two billion apart.
The difference between two floats is at most 1.00000012 times larger than the number, which is a modest and predictable radio. This does not apply to number between FLT_MAX and infinity and FLT_MIN and zero.
Incrementing floats with INF, -INF and NaN causes very strange issues. Be careful with these values.
For what it's worth, on my system the values are around to the closest value in either direction. Unlike with Integers in C, where the value is imply truncated. For instance, 1e10 10000000000 is on one end while 10000001024 is the next number. If the number is 10000000001, then the value is sent backwards to 10000000000. If the value is 10000001023, then it is sent forwards.