Fix TLS 1.3 reads with mbedtls, and errno signaling on Windows#594
Merged
Conversation
In TLS 1.3 the server sends NewSessionTicket after the handshake. With mbedtls 3.6+ mbedtls_ssl_read can surface this as MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET (same for early data), a non-fatal "read again" code. SocketMbedTLS::recv treated it as a fatal error, so the first read after the handshake failed with "Failed reading HTTP status line". Retry the read instead. Also fix would-block signaling on Windows: recv/send set the CRT errno, but Socket::getErrno() reads WSAGetLastError(), so isWaitNeeded() aborted legitimate would-block retries. Add a Socket::setErrno() helper mirroring getErrno() and use it at all TLS backend error-signaling sites. This fixes the same latent bug in the OpenSSL backend. Map ECONNRESET to WSAECONNRESET in IXNetSystem.h so it round-trips through setErrno()/getErrno() like the other errno codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #592.
Problem
Connecting to a TLS 1.3 wss/https server (e.g. behind Cloudflare/nginx) with the mbedtls backend (mbedtls 3.6+, TLS 1.3 on by default) fails on the first read after the handshake with
Failed reading HTTP status line.Two issues combine:
NewSessionTicketafter the handshake.mbedtls_ssl_readsurfaces this asMBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET(-0x7B00), a non-fatal "read again" code (same forMBEDTLS_ERR_SSL_RECEIVED_EARLY_DATA).SocketMbedTLS::recvtreated any such code as a fatal error.WANT_READ/WANT_WRITE, recv/send set the CRTerrno, butSocket::getErrno()readsWSAGetLastError()on Windows, soisWaitNeeded()aborted legitimate would-block retries. The OpenSSL backend had the same latent bug.Fix
SocketMbedTLS::recv: retry the read onMBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET/MBEDTLS_ERR_SSL_RECEIVED_EARLY_DATA(both guarded with#if defined(...)so older mbedtls still compiles).Socket::setErrno()helper mirroringgetErrno():WSASetLastError()on Windows,errnoelsewhere. Used at all TLS backend error-signaling sites (mbedtls, OpenSSL, SecureTransport), fixing the Windows would-block bug in both the mbedtls and OpenSSL backends.IXNetSystem.h: mapECONNRESET→WSAECONNRESETalongside the existing errno mappings so it round-trips throughsetErrno()/getErrno().Verification
Reproduced and verified on macOS against real TLS 1.3 servers (mbedtls 3.6.3):
mbedtls_ssl_conf_tls13_enable_signal_new_session_tickets, cf. Breakage due to enabling TLS 1.3 by default in 3.6.0 Mbed-TLS/mbedtls#9223), so to reproduce, the signal was temporarily forced on. Instrumentation confirmed-0x7b00coming out ofmbedtls_ssl_read, and an HTTPS GET to cloudflare.com failed exactly as reported (on POSIX the bug is usually masked by a staleerrno == EAGAINfrom the preceding nonblocking read, which is why it mostly bites on Windows).wss://echo.websocket.orground-trip echoes correctly.🤖 Generated with Claude Code