Skip to content

seq/printf: fix OOM abort in %a formatter for huge negative exponents#13370

Open
koopatroopa787 wants to merge 2 commits into
uutils:mainfrom
koopatroopa787:fix-seq-format-a-oom-on-large-exponent
Open

seq/printf: fix OOM abort in %a formatter for huge negative exponents#13370
koopatroopa787 wants to merge 2 commits into
uutils:mainfrom
koopatroopa787:fix-seq-format-a-oom-on-large-exponent

Conversation

@koopatroopa787

Copy link
Copy Markdown
Contributor

Summary

seq -f %a 1E-1000000000000000000 1 aborts with an out-of-memory error:

memory allocation of 375000000000000024 bytes failed
Aborted (core dumped)

The crash is in format_float_hexadecimal inside uucore. When the decimal exponent exp10 is very negative, the formula:

let margin =
    ((max_precision + 1) as i64 * 4 - frac10.bits() as i64).max(0) + -exp10 * 3 + 1;

produces margin ≈ 3 × 10¹⁸ for exp10 = -10¹⁸. The subsequent frac10 << margin asks num-bigint to build an integer of that many bits (~375 petabytes), which the allocator rejects and the process aborts (exit 134).

Root cause

GNU seq uses a long double (80-bit on x86-64, exponent range ≈ ±4932). Any decimal value with magnitude smaller than ~10⁻⁴⁹³² underflows to 0 and is formatted as 0x0p+0 cheaply. The uutils bignum path has no such guard and tries to compute the exact binary representation regardless of the exponent magnitude.

The existing code even has a TODO comment noting the hazard:

TODO: this is most accurate, but frac2 will grow a lot for large precision or exponent, and formatting will get very slow.

Fix

Add an early-exit before the bignum path in format_float_hexadecimal: if exp10 < -5000, the value underflows to zero in any long-double implementation, so return the zero hex-float representation immediately.

The threshold 5000 is safely beyond the ±4932 decade range of 80-bit and 128-bit long double, so no representable nonzero value is incorrectly rounded to zero.

Tests

  • Unit test hexadecimal_float_huge_negative_exponent_does_not_oom in num_format.rs — verifies 1E-10000000000000000000x0p+0 and 1E-50010x0p+0 without panic/abort.
  • Integration test test_format_a_huge_negative_exponent_does_not_oom in test_seq.rs — exercises the exact invocation from the bug report.

Fixes #13222

`seq -f %a 1E-1000000000000000000 1` aborted with:

  memory allocation of 375000000000000024 bytes failed
  Aborted (core dumped)

The root cause is in `format_float_hexadecimal` (uucore): when the
decimal exponent is very negative the formula

  margin ≈ -exp10 * 3 + (precision+1)*4

overflows to ~3×10¹⁸, so `frac10 << margin` asks num-bigint to build
an integer of that many bits, exhausting memory.

GNU `seq` uses a `long double` (≈15-bit exponent, ±4932 decade range),
so values below ~10⁻⁴⁹³² naturally underflow to zero and are formatted
as `0x0p+0`.  Mirror that behaviour: if the decimal exponent is below
−5000, return the zero representation immediately without entering the
bignum path.

Fixes uutils#13222
Copilot AI review requested due to automatic review settings July 12, 2026 13:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@sylvestre

Copy link
Copy Markdown
Contributor

Please fix the failing jobs

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/seq/seq-epipe is now being skipped but was previously passing.
Congrats! The gnu test tests/printf/printf-surprise is now passing!

Two CI failures on uutils#13370:

1. cspell flagged 'bignum' as unknown — add it to the existing
   spell-checker:ignore directive in num_format.rs and inline in test_seq.rs.

2. test_format_a_huge_negative_exponent_does_not_oom timed out (>30s)
   because seq's BigDecimal arithmetic on a 10^18-scale number hangs
   before the formatter is even reached.  Swap to 1E-5001: this value
   still exercises the early-exit guard (*num < -5000) but keeps the
   BigDecimal scale manageable (~5001-digit integers).  The unit test in
   num_format.rs still covers the 1E-1000000000000000000 extreme case
   directly.
@koopatroopa787

Copy link
Copy Markdown
Contributor Author

Fixed — two issues addressed in the latest push:

  1. Spelling: added bignum to the spell-checker:ignore directive in num_format.rs (and inline in the test).
  2. Timeout: the integration test was timing out because seq's BigDecimal arithmetic on a 10^18-scale number hangs before the formatter is even reached. Changed the test to use 1E-5001 (just past the guard threshold, but with manageable ~5001-digit BigDecimal arithmetic). The unit test in num_format.rs still covers 1E-1000000000000000000 directly against the formatter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

# seq -f %a aborts (out-of-memory) on a large-magnitude exponent that GNU formats fine

3 participants