whisper: validate mel filter dimensions to prevent integer overflow

[Sebasteuo]

Summary

This PR adds bounds validation to the mel filter loading code in whisper_model_load() to prevent an integer overflow that leads to an undersized heap allocation and out-of-bounds read.

The bug

In src/whisper.cpp, the original code was:

read_safe(loader, filters.n_mel);
read_safe(loader, filters.n_fft);

filters.data.resize(filters.n_mel * filters.n_fft);

Both filters.n_mel and filters.n_fft are int32_t read directly from the model file with no validation. When the model declares dimensions whose product exceeds INT32_MAX (e.g. n_mel=65537, n_fft=65536), the multiplication overflows int32_t to a small value, the buffer is allocated undersized, and subsequent reads in log_mel_spectrogram_worker_thread() (line ~3136) access data out of the allocated region.

The pre-existing assert at line ~3114 (n_fft == 1 + (frame_size / 2)) is a no-op in release builds compiled with -DNDEBUG, so it does not protect production deployments.

Reproduction (high level)

A crafted ggml model file with attacker-controlled n_mel and n_fft headers triggers the overflow during model loading. Confirmed with AddressSanitizer on aarch64 and x86_64 (Ubuntu 24.04) against whisper-cli built with:
cmake .. -DCMAKE_CXX_FLAGS="-fsanitize=address -g -O1"
-DCMAKE_C_FLAGS="-fsanitize=address -g -O1"
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address"
-DBUILD_SHARED_LIBS=OFF

ASAN reports heap-buffer-overflow READ of size 4 in log_mel_spectrogram_worker_thread (src/whisper.cpp:3136), located 0 bytes after the undersized region allocated at filters.data.resize(...).

The fix

This PR makes three changes inside the existing mel filters loading block:

1. Bounds check on the raw dimensions before any multiplication: n_mel in [1, 1024] and n_fft in [1, WHISPER_N_FFT]. These ranges comfortably accommodate all current and reasonable future Whisper variants while making the integer-overflow path unreachable.

2. Consistency check between filters.n_mel and hparams.n_mels. This invariant was already implicitly relied upon by the assert at src/whisper.cpp:2386 (mel_inp.n_mel == wctx.model.hparams.n_mels); promoting it to upstream validation makes the failure mode explicit and works in release builds.

3. size_t cast for the allocation size so that even within the validated bounds, the multiplication is performed in 64-bit precision, defending against any future changes to the bounds.

On invalid input the function logs a clear error and returns false, matching the existing error-handling pattern in whisper_model_load().

Disclosure note

This issue was reported privately to the maintainer email on 2026-03-30 and re-pinged on 2026-04-17. I am opening this PR to facilitate the fix in case the private channel is no longer being monitored. The patch itself is minimal and self-contained; no exploit details, PoC, or crafted model file are included in this PR. I am happy to coordinate further on CVE assignment / advisory wording / disclosure timing as the maintainers prefer.

Testing

• The patch is local to a small block of whisper_model_load() and only adds validation around an already-existing resize call; no behavioral change for valid model files.
• Manual review of the diff against the surrounding error-handling style in whisper_model_load() (see e.g. existing checks at src/whisper.cpp:1500 and src/whisper.cpp:1557).

Happy to add a regression test if helpful, in this PR or as a follow-up.