fix: stabilize test_text_query_word_weights with structural assertions - #541
fix: stabilize test_text_query_word_weights with structural assertions#541thakoreh wants to merge 3 commits into
Conversation
Replace exact-string assertion (sensitive to non-deterministic token ordering) with component-based checks that verify: - Description clause is properly delimited - Weighted terms appear inside the clause - alpha occurs twice (matching input frequency) - Unweighted terms are present - Post-query modifiers (SCORER, WITHSCORES, DIALECT, LIMIT) follow Addresses review feedback on redis#524.
There was a problem hiding this comment.
Pull request overview
This PR re-enables and stabilizes the previously flaky test_text_query_word_weights by replacing an exact full-string comparison with structural assertions that are intended to be insensitive to token ordering while still validating query shape.
Changes:
- Removed the
@pytest.mark.skipthat was disabling the flaky test. - Replaced the single exact-string assertion with component-based checks over the generated query string.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Verify both alpha occurrences are weighted (no unweighted alpha tokens) - Enforce modifier ordering: SCORER <= WITHSCORES <= DIALECT <= LIMIT
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Check query components with structural guarantees, | ||
| # not exact token ordering (which is non-deterministic). |
There was a problem hiding this comment.
The comment about token ordering being non-deterministic is likely misleading here: TextQuery._tokenize_and_escape_query() preserves the input token order and only does in-place substitutions for weighted tokens. If the goal is to avoid brittleness, consider rewording this to the actual nondeterminism/flakiness source (or just say the test avoids exact full-string matching).
| # Check query components with structural guarantees, | |
| # not exact token ordering (which is non-deterministic). | |
| # Check query components with structural guarantees rather than relying | |
| # on exact full-string or token-order matching, to avoid brittle tests. |
| # alpha appears twice and both occurrences are weighted | ||
| alpha_weighted = "alpha=>{$weight:2}" | ||
| assert desc_clause.count(alpha_weighted) == 2 | ||
| # Ensure no unweighted 'alpha' tokens slipped through | ||
| idx = 0 |
There was a problem hiding this comment.
desc_clause.count(alpha_weighted) and the subsequent find("alpha") loop rely on substring matching, which can behave unexpectedly if token escaping changes or a different token contains alpha as a substring. Since tokens are joined with " | ", consider splitting desc_clause into tokens (strip whitespace) and asserting exact token equality + frequency via a list/Counter.
| # Unweighted terms are present | ||
| for term in ["query", "string", "bravo", "tango"]: | ||
| assert term in desc_clause |
There was a problem hiding this comment.
assert term in desc_clause checks substrings, not whole tokens, so it may pass even if the tokenization/output changes in a way that shouldn’t be acceptable (or fail for escaped tokens). Consider tokenizing desc_clause (split on | and strip) and asserting the expected token set is present.
| # Unweighted terms are present | |
| for term in ["query", "string", "bravo", "tango"]: | |
| assert term in desc_clause | |
| # Unweighted terms are present as standalone tokens within the clause | |
| desc_tokens = {token.strip() for token in desc_clause.split("|") if token.strip()} | |
| for term in ["query", "string", "bravo", "tango"]: | |
| assert term in desc_tokens |
vishal-bala
left a comment
There was a problem hiding this comment.
Hi @thakoreh, thanks for the contribution. Can you take a look at the automated review comments?

Summary
Fixes the flaky
test_text_query_word_weightstest (related to #497).Replaces the exact-string assertion (sensitive to non-deterministic token ordering in older Python dicts) with structural component checks:
(...)alpha=>{$weight:2},delta=>{$weight:0.555}) appear inside the clausealphacount matches input frequency (2 occurrences)This is resistant to token ordering while still catching regressions in query structure. Addresses review feedback on #524.
Testing
Note
Low Risk
Low risk: changes are limited to a unit test, replacing a brittle exact-string assertion with order-insensitive structural checks to reduce flakiness.
Overview
Stabilizes
test_text_query_word_weightsby removing the skipped/brittle exact query-string comparison and replacing it with structural assertions that tolerate non-deterministic token ordering.The test now validates the
@description:(...)clause contents (including weighted terms and repeated weightedalpha) and ensures the expected post-query modifiers (SCORER,WITHSCORES,DIALECT,LIMIT) appear after the clause in the correct sequence.Written by Cursor Bugbot for commit 9680523. This will update automatically on new commits. Configure here.