Skip to content

[Bug] Gemini Live native audio degenerates into a sustained non-speech tone while output transcription stalls #2715

Description

@jooneyp

Summary

In one production Korean inbound voice call, Gemini Live native audio intermittently degenerated into a loud sustained non-speech, vocal-like harmonic tone for approximately 14 seconds. The provider output transcription stopped at an incomplete phrase while audio bytes continued until the caller hung up.

This was not a simple playback stutter: the malformed signal was present in our independently recorded AI-output branch before FreeSWITCH/carrier playback, and the PCM did not contain exact repeated 20–200 ms frames.

All customer identifiers and raw conversational content are omitted from this public report.

Environment

  • SDK: google-genai==1.55.0
  • Runtime: Python 3.12, stateful telephony worker
  • API surface: Vertex AI Live API
  • Expected model from current no-override configuration: gemini-live-2.5-flash-native-audio
  • Response modality: native audio, 24 kHz PCM16 from Gemini
  • Local processing: independent streaming 24→8 kHz resampler for phone playback and 24→16 kHz resampler for the canonical stereo recording
  • Language: Korean
  • Thinking budget: disabled in the current default configuration

Important limitation: the call record did not persist the effective provider model ID, and the historical telephony task logs were no longer retained after a later task replacement. Therefore, the model ID above is the expected effective model from the instance/process defaults, not a per-call model-ID readback.

Observed behavior

The canonical 46.24-second stereo recording has caller audio on the left and AI output on the right.

  • ~3.0–19.5 s: normal Gemini speech
  • ~25.5–26.5 s: another normal short Gemini utterance
  • ~31.5/32.4 s until 46.2 s: continuous abnormal harmonic vocalization / sustained tone
  • Persisted Gemini output_transcription for the final response ends with an incomplete Korean phrase equivalent to “Yes, about the gas stove...”
  • No call error or failed handler event was recorded
  • The caller eventually hung up; the WebSocket then closed normally

Signal evidence

For the abnormal AI-output interval:

  • Continuous matching run: 14.5 s
  • Median dominant frequency bin: 187.5 Hz (approximately 195 Hz fundamental by autocorrelation)
  • Median RMS: -12.4 dBFS
  • Median spectral flatness: 0.0004
  • Stable harmonic comb consistent with a sustained vocal-like generation, not broadband static
  • Zero exact adjacent repeats at 20, 40, 100, or 200 ms chunk sizes

The same local resamplers and recording path produced normal speech earlier in the same call. Native codec/resampler tests for PCM alignment, duration, chunk continuity, per-session isolation, passband preservation, and above-Nyquist rejection all passed locally.

Transport and routing checks

Retained FreeSWITCH evidence for the exact call showed:

  • One channel answer
  • One audio-fork start
  • WebSocket connection success
  • No second fork
  • Normal caller hangup
  • Graceful audio socket close

The canonical AI-right recording branch is populated before freeswitch_ws.send_bytes(), so downstream FreeSWITCH routing, carrier encoding, or phone playback cannot originate a tone already present in that branch.

Our Gemini receive path sequentially appends each response.data block once, emits it once, and clears the small provider buffer after callback delivery. There is no retry/requeue path that could create a local frame loop. Output transcription is consumed separately from server_content.output_transcription.

Cohort check

We ran a bounded acoustic comparison over 22 unique completed canonical recordings: all available completed recordings from the incident day plus the affected instance's completed recordings from the preceding seven days.

  • This call was the only recording with a matching continuous run of 8 seconds or longer
  • The next-longest broad-signature run was 3.0 seconds
  • A later call on the same instance approximately three minutes afterward had no prolonged matching run (maximum 1.5 seconds)

This looks intermittent rather than an ongoing route-, process-, or instance-wide failure.

Expected behavior

Gemini Live should either:

  1. produce intelligible audio corresponding to the output transcription, or
  2. emit an explicit error/interruption/termination signal that allows the client to recover safely.

It should not continue streaming loud malformed native audio after semantic/output-transcription progress has stalled.

Questions

  1. Is sustained non-speech/harmonic degeneration a known failure mode of gemini-live-2.5-flash-native-audio on Vertex AI?
  2. Which response/session metadata should clients retain to correlate this with provider-side diagnostics without storing raw customer audio?
  3. Is there a supported response-scoped cancel mechanism for malformed-but-valid audio that does not terminate or replay the entire Live session?
  4. Is session replacement the recommended recovery, and if so, how should clients avoid replaying tool results or duplicating the previous response?

Related public reports

We can provide additional PII-free timing summaries and spectral plots if useful, but cannot post the raw customer recording publicly.

Metadata

Metadata

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.status:awaiting user responsetype: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions