Add staged connection diagnosis for opaque dial failures#1114
Draft
rossnelson wants to merge 2 commits into
Draft
Add staged connection diagnosis for opaque dial failures#1114rossnelson wants to merge 2 commits into
rossnelson wants to merge 2 commits into
Conversation
Covers classifyGRPCError, connectSummary's grep-compatibility contract, and an end-to-end case where the failing address comes from a config profile (exercising the new cliext builder metadata).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
Connection failures now produce a classified, actionable error with an inline staged diagnosis instead of a bare
context deadline exceededor an empty message.When
client.DialContextfails, the CLI probes the target address in stages (DNS → TCP → TLS) using the same TLS configuration the client would use, classifies the root cause, and suggests one concrete fix:Other classified cases: connection refused (with a
temporal server start-devhint onlocalhost:7233), DNS failure (naming the config profile if the address came from one), TLS↔plaintext mismatch in either direction (detected behaviorally, including an opportunistic anonymous TLS handshake when the client is plaintext), untrusted/mismatched server certificates, rejected API keys, unreadable TLS cert files (caught pre-dial, naming the exact path), and black-holed connections.Design notes:
--client-connect-timeoutwhen set).TEMPORAL_CLI_DISABLE_CONNECT_DIAGNOSIS=1opts out; interrupts skip it.Error(), so the defaultError:printing, JSON mode (no ANSI — rendering happens whilecolor.NoColoris set), and the test harness all work unchanged.Unwrap()preserves the original error forerrors.Is/As.connection refused,context deadline exceeded, …) so scripts grepping stderr keep matching.crypto/tlsservers.Two adjacent paper cuts fixed along the way:
context.WithTimeoutCause(command timed out after 5s) were discarded; they now survive into the error output.--command-timeoutexpiry asprogram interrupted.cliext.ClientOptionsBuildergains read-only output fields (ResolvedAddress,ResolvedAddressSource,ResolvedProfileName,HasAPIKey) so the suggestion engine knows where the address came from, following the existingPayloadCodecoutput-field pattern.Why?
Fixes #224 — a server requiring mTLS without client certs produced an empty error.
Fixes #851 — connection failures surfaced as a bare
context deadline exceededwith no indication it was a connection problem (tctl gave a useful error).Both trace to the same gap: the SDK's dial is lazy, so failures surface from the eager
GetSystemInfocheck and were returned bare fromdialClient, while SDK connection logs go to a nop logger by default (--log-leveldefaults tonever).Checklist
Closes [Feature Request] Show actual error message when mTLS certs are missing #224
Closes Temporal cli gives context deadline exceeded. Tctl does not #851
How was this tested:
connectdiag_test.go) run the classifier against real in-process listeners: aRequireAndVerifyClientCertTLS server, a plaintext server answered with TLS, a TLS server dialed in plaintext, refused ports,.invalidDNS, and an untrusted CA. These also act as canaries for the Go TLS alert strings.client_test.go) assert the full rendered error for each scenario end-to-end throughtemporalcli.Execute, including JSON mode (no ANSI, empty stdout), the disable env var, unreadable cert files, and--command-timeoutcause survival.TEMPORAL_CLI_DISABLE_CONNECT_DIAGNOSISif we document escape-hatch env vars; open to renaming or dropping it.Notes for reviewers
commands.yaml; this is all in the dial path.-o jsonmode, a standalonetemporal doctorcommand reusing the same prober, and surfacing the diagnosis inoperator cluster health.cliextmodule does not currently compile on its own atmain(itsgo.modpinsenvconfig v1.0.0, predatingClientConfigProfile.Authority); it builds through the root module'sreplace. Unrelated to this PR but worth a follow-up bump.