ZD-22438: fix TLS version checks

[padelsbach]

Description

Fix to enable wolfSSL_SetVersion value to be reflected in version advertisements.

Fixes zd 22438

Testing

New test cases

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[github-actions]

retest this please

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11456

Scan targets checked: wolfssl-src, wolfssl-bugs

Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

[wolfSSL-Fenrir-bot]

SetVersion leaves protocol downgrade enabled · TLS protocol issues

wolfSSL_SetVersion() no longer clears options.downgrade, so flexible-method sessions negotiate below the selected version. Unlike known #6545's missing downgrade sentinel, this expands the allowed range itself.

Related known finding #1810 (similar but distinct): Both can permit negotiation of older TLS versions, but #1810 is compile-time insecure context defaults in wolfSSL_CTX_new_ex, while this is stale per-session downgrade state in wolfSSL_SetVersion. The faulting operations, root causes, and required patches differ.

Suggested fix: Clear downgrade when only wolfSSL_SetVersion() is set; re-enable it only when an explicit minimum establishes a range.
Basis: wolfSSL API Reference, wolfSSL_SetVersion: the call overrides the session protocol setting with version.

[padelsbach]

Fixed. This was an intentional choice, but swapped the behavior back.

[wolfSSL-Fenrir-bot]

Handshake test does not verify negotiated version · Weak or missing assertions

test_wolfSSL_SetVersion_tls12_with_tls13_peer() checks only handshake success; an implementation that ignores the TLS 1.2 override and negotiates TLS 1.3 still passes.

Suggested fix: Assert wolfSSL_get_version() equals TLSv1.2 for both peers after the handshake.
Basis: wolfSSL API Reference, wolfSSL_SetVersion: the call overrides the session protocol setting with version.

[padelsbach]

Added

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m0plus

• FLASH: .text +76 B (+0.1%, 67,099 B / 262,144 B, total: 26% used)

gcc-arm-cortex-m3

• FLASH: .text +788 B (+0.6%, 126,679 B / 262,144 B, total: 48% used)

gcc-arm-cortex-m4

• FLASH: .text +320 B (+0.2%, 205,405 B / 262,144 B, total: 78% used)

gcc-arm-cortex-m4-baremetal

• FLASH: .text +64 B (+0.1%, 69,539 B / 262,144 B, total: 27% used)

gcc-arm-cortex-m4-crypto-only

• FLASH: .text +64 B (+0.0%, 178,840 B / 262,144 B, total: 68% used)

gcc-arm-cortex-m4-dtls13

• FLASH: .text +704 B (+0.4%, 187,964 B / 1,048,576 B, total: 18% used)

gcc-arm-cortex-m4-min-ecc

• FLASH: .text +64 B (+0.1%, 64,389 B / 262,144 B, total: 25% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .rodata +96 B, .text +1,536 B (+0.2%, 785,444 B / 1,048,576 B, total: 75% used)

gcc-arm-cortex-m4-pkcs7

• FLASH: .rodata.CSWTCH.202 -3 B, .text +128 B (+0.1%, 217,689 B / 262,144 B, total: 83% used)

gcc-arm-cortex-m4-pq

• FLASH: .text +832 B (+0.3%, 302,856 B / 1,048,576 B, total: 29% used)

gcc-arm-cortex-m4-rsa-only

• FLASH: .text +1,152 B (+0.3%, 334,048 B / 1,048,576 B, total: 32% used)

gcc-arm-cortex-m4-sp-math

• FLASH: .text +64 B (+0.1%, 64,389 B / 262,144 B, total: 25% used)

gcc-arm-cortex-m4-tls12

• FLASH: .text +832 B (+0.7%, 127,475 B / 262,144 B, total: 49% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +768 B (+0.3%, 243,279 B / 262,144 B, total: 93% used)

gcc-arm-cortex-m7

• FLASH: .text +384 B (+0.2%, 205,405 B / 262,144 B, total: 78% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +832 B (+0.3%, 303,816 B / 1,048,576 B, total: 29% used)

gcc-arm-cortex-m7-tls13

• FLASH: .text +768 B (+0.3%, 243,343 B / 262,144 B, total: 93% used)

linuxkm-pie

• Data: __patchable_function_entries +40 B (+0.1%, 27,128 B)

linuxkm-standard

• Data: __patchable_function_entries +72 B (+0.1%, 49,952 B)

stm32-sim-stm32h753

• FLASH: .text +1,456 B (+0.8%, 189,676 B / 2,097,152 B, total: 9% used)

[JacobBarthelmeh]

Please address the Fenrir report. I thumbed up the one I identified when also reviewing the PR. Once addressed then assign back to wolfssl-bot.

[padelsbach]

@JacobBarthelmeh, my mistake, I had missed a push. Updates applied. Back over to you.

[padelsbach]

retest this please