Repository navigation
fix(bq_driver): transparently handle transient stream disconnections and resume HTAPI reads - #1692
Draft
KanchanShu wants to merge 2 commits into
Draft
KanchanShu wants to merge 2 commits into
KanchanShu wants to merge 2 commits into
Conversation
…and resume HTAPI reads
| std::string message = status.message(); | ||
| std::transform(message.begin(), message.end(), message.begin(), | ||
| [](unsigned char c) { return std::tolower(c); }); | ||
| if (message.find("rst_stream") != std::string::npos || |
Contributor
There was a problem hiding this comment.
how do we know we need to retry for these messages? some probably can be permanent errors and we are unnecessarily retrying for them
Contributor
Author
There was a problem hiding this comment.
Updated, To avoid retrying permanent errors, I removed broad terms (timeout, proxy) and kept only explicit TCP/HTTP-2 transport disconnects (rst_stream, stream removed, connection reset, broken pipe).
| } | ||
|
|
||
| int max_retries = conn_handle->GetDsn().max_retries; | ||
| if (max_retries <= 0) { |
Contributor
There was a problem hiding this comment.
don't change for max_retries = 0, as the user might not want retries
Contributor
Author
There was a problem hiding this comment.
fixed. We now use conn_handle->GetDsn().max_retries directly without overriding 0, so setting MaxRetries=0 disables retries as expected.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes an issue where long-running queries using the High-Throughput API (HTAPI / BigQuery Storage Read API) fail prematurely when intermediate network devices, proxies, or load balancers drop the connection (e.g., HTTP/2
RST_STREAM, TCP connection resets, proxy timeouts, orUNAVAILABLEerrors).Previously, any stream error in
ReadNextResultsFromStream()immediately failed the query and returned a fatal error to the user application.This PR implements transparent stream re-attachment and resumption:
IsTransientStreamError()to identify retryable status codes (UNAVAILABLE,DEADLINE_EXCEEDED,ABORTED,RESOURCE_EXHAUSTED) as well asINTERNAL/UNKNOWNerrors resulting from transport-level drops (rst_stream,connection reset,broken pipe,proxy,timeout,transport: lost connection).stream_row_offset_) as Arrow record batches are consumed. On a transient failure, the driver re-attaches to the existingReadSessionstream usingReadRowsRequest.set_offset(N)after an exponential backoff delay.read_stream_name_,stream_row_offset_, andwas_htapi_enabled_onStatementHandle, and resets them inCloseCursor()to avoid state leakage across reused statement handles.Key Changes
google/cloud/odbc/bq_driver/internal/odbc_sql_execute_utils.h:IsTransientStreamError().google/cloud/odbc/bq_driver/internal/odbc_sql_execute_utils.cc:IsTransientStreamError()for status code and error message inspection.ReadNextResultsFromStream()to execute an exponential backoff retry loop up tomax_retrieswithread_rows_request.set_offset(stream_row_offset_).FetchBQDataReadArrow().google/cloud/odbc/bq_driver/internal/odbc_stmt_handle.h:read_stream_name_,stream_row_offset_, andwas_htapi_enabled_.google/cloud/odbc/bq_driver/internal/odbc_stmt_handle.cc:CloseCursor().Verification
libgoogle_cloud_odbc_bq_driver_internal.aandlibgoogle_cloud_odbc_bq_driver.sobuild cleanly with zero errors.