NOTE: The info page for sort does explain this behavior, though the man has no mention of it:

Numeric sort uses what might be considered an unconventional method to compare strings representing floating point numbers.

Rather than first converting each string to the C `double' type and then comparing those values, `sort' aligns the decimal-point characters in the two strings and compares the strings a character at a time.

One benefit of using this approach is its speed. In practice this is much more efficient than performing the two corresponding string-to-double (or even string-to-integer) conversions and then comparing doubles.

In addition, there is no corresponding loss of precision.

Converting each string to `double' before comparison would limit precision to about 16 digits on most systems.