Skip to content

Do not ignore SKIP as a new test result after re-test #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
miczyg1 wants to merge 1 commit into
base: v7.3-merged-changes
Choose a base branch
from
Open

Do not ignore SKIP as a new test result after re-test #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 18 additions & 11 deletions atest/robot/rebot/merge.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,17 +78,20 @@ Using other options
Test merge should have been successful suite name=Custom
Log should have been created with Fail keywords flattened

Merge ignores skip
Copy link
Copy Markdown

@philipanda philipanda May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test fails for me:

----------------------------------------------------------------------------------------------------
FAIL: Robot.Rebot.Merge.Merge skip result
Evaluating expression "'Test skipped using 'NOT skip' tag pattern.' != ''" failed: SyntaxError: invalid syntax (<string>, line 1)
----------------------------------------------------------------------------------------------------

Merge skip result
Create Output With Robot ${ORIGINAL} ${EMPTY} rebot/merge_statuses.robot
Create Output With Robot ${MERGE1} --skip NOTskip rebot/merge_statuses.robot
Run Merge
${prefix} = Catenate
... *HTML* Test has been re-executed and results merged.
... Latter result had <span class="skip">SKIP</span> status and was ignored. Message:
${pass msg} = Create expected merge message ${EMPTY}
... SKIP Test skipped using 'NOT skip' tag pattern. PASS ${EMPTY}
${fail msg} = Create expected merge message ${EMPTY}
... SKIP Test skipped using 'NOT skip' tag pattern. FAIL Not &lt;b&gt;HTML&lt;/b&gt; fail
${skip msg} = Create expected merge message ${EMPTY}
... SKIP <b>HTML</b> skip SKIP <b>HTML</b> skip
Should Contain Tests ${SUITE}
... Pass=PASS:${prefix}\nTest skipped using 'NOT skip' tag pattern.
... Fail=FAIL:${prefix}\nTest skipped using 'NOT skip' tag pattern.<hr>Original message:\nNot &lt;b&gt;HTML&lt;/b&gt; fail
... Skip=SKIP:${prefix}\n<b>HTML</b> skip<hr>Original message:\n<b>HTML</b> skip
... Pass=SKIP:${pass msg}
... Fail=SKIP:${fail msg}
... Skip=SKIP:${skip msg}

*** Keywords ***
Run original tests
Expand Down Expand Up @@ -277,8 +280,10 @@ Create expected merge message header

Create expected merge old message body
[Arguments] ${old status} ${old message}
${old status} = Set Variable If '${old status}' == 'PASS'
... <span class="pass">PASS</span> <span class="fail">FAIL</span>
${old status} = Set Variable If
... '${old status}' == 'PASS' <span class="pass">PASS</span>
... '${old status}' == 'FAIL' <span class="fail">FAIL</span>
... <span class="skip">SKIP</span>
${old message} = Set Variable If '${old message}' != ''
... ${old message}<br> ${EMPTY}
${old message html achor} = Set Variable If '${old message}' != ''
Expand All @@ -289,8 +294,10 @@ Create expected merge old message body

Create expected merge message body
[Arguments] ${new status} ${new message} ${old status} ${old message}
${new status} = Set Variable If '${new status}' == 'PASS'
... <span class="pass">PASS</span> <span class="fail">FAIL</span>
${new status} = Set Variable If
... '${new status}' == 'PASS' <span class="pass">PASS</span>
... '${new status}' == 'FAIL' <span class="fail">FAIL</span>
... <span class="skip">SKIP</span>
${new message html achor} = Set Variable If '${new message}' != ''
... <span class="new-message">New message:</span>${SPACE} ${EMPTY}
${new message} = Set Variable If '${new message}' != ''
Expand Down
12 changes: 0 additions & 12 deletions src/robot/result/merger.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,6 @@ def visit_test(self, test):
if old is None:
test.message = self._create_add_message(test)
self.current.tests.append(test)
elif test.skipped:
Copy link
Copy Markdown

@philipanda philipanda May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make it conditional using a flag passed to rebot?
This way there would be less friction in future rebases of the fork.

Copy link
Copy Markdown
Member Author

@miczyg1 miczyg1 May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestions to the flag name?

Copy link
Copy Markdown

@philipanda philipanda May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe --skip-overrides-fail?

old.message = self._create_skip_message(old, test)
else:
test.message = self._create_merge_message(test, old)
index = self.current.tests.index(old)
Expand Down Expand Up @@ -129,13 +127,3 @@ def _format_old_status_and_message(self, test, merge_header):
self._message_header("New"),
self._message_header("Old"),
)

def _create_skip_message(self, test, new):
msg = (
f"*HTML* {test_or_task('Test', self.rpa)} has been re-executed and "
f"results merged. Latter result had {self._status_text('SKIP')} "
f"status and was ignored. Message:\n{self._html(new.message)}"
)
if test.message:
msg += f"<hr>Original message:\n{self._html(test.message)}"
return msg
Loading