feat(testcase): annotate testcase links with results

docs/how-to/test_to_doc_links.rst

@@ -20,21 +20,44 @@ Reference Docs in Tests
 This guide explains how to annotate test cases so that
 docs-as-code automatically creates traceability links between tests and requirements.
 
-Details of implementation
--------------------------
+How to annotate tests
+---------------------
 
-The mechanism is language-agnostic:
-The ``score_source_code_linker`` extension parses ``test.xml`` files, extracts test metadata
-(name, file, line, result, and verification properties),
-and creates backlinks on the referenced requirements.
+To link a test to the requirements it verifies, add the test metadata using
+the mechanism provided by its test framework:
 
-The extension will look for ``test.xml`` files in both ``bazel-testlogs/``
-as well as a folder named ``tests-report/``. This folder can be created manually if the tests
-require some pre-run step or are matrix tests or similar.
+Python (pytest)
+^^^^^^^^^^^^^^^
+   Use the ``@add_test_properties`` decorator. The test docstring supplies
+   the ``Description`` metadata.
+
+C++ (gTest)
+^^^^^^^^^^^
+   Use ``RecordProperty``. Put shared properties in ``SetUp()`` and
+   per-test properties inside each ``TEST_F``.
+
+Rust
+^^^^
+   There is currently no provided official way to add this metadata in Rust.
+   Use the advanced JUnit XML path below until Rust support is available.
 
+See the `Verification Templates <https://eclipse-score.github.io/reference_integration/main/_collections/score_process/process/process_areas/verification/guidance/verification_templates.html>`_
+for complete examples and the required metadata.
 
-Required Properties
--------------------
+Advanced usage: JUnit XML for other languages
+----------------------------------------------
+
+This section is only relevant when your language or test framework does not
+have one of the integrations above. In that case, produce JUnit XML with the
+metadata described below. The generated test results are processed
+automatically and create GitHub links from the requirements to the testcases.
+
+The extension looks for files named ``test.xml`` under ``bazel-testlogs/`` or
+``tests-report/`` at the workspace root. Create ``tests-report/`` manually when
+the test framework needs a separate pre-run step or produces matrix results.
+
+Required properties
+^^^^^^^^^^^^^^^^^^^
 
 Every linked test must declare the following properties
 (see :need:`gd_guidl__verification_specification` for detailed values):
@@ -50,13 +73,12 @@ Every linked test must declare the following properties
 
 ``Description``
    A human-readable explanation of the test objective and expected outcome.
-   *In Python tests, a docstring takes the place of the description attribute.*
 
 What should a test.xml look like?
----------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Each testcase has file as well as it's line number as additional attributes.
-Then as described above, each testcase also has properties inside the XML.
+Each testcase must include its source file and line number as attributes,
+along with the verification properties.
 
 .. code-block:: xml
 
@@ -80,35 +102,22 @@ Then as described above, each testcase also has properties inside the XML.
      </testsuite>
    </testsuites>
 
-If you are not working in Python and using the provided pytest plugin, please ensure that the xml that is written in the end
-looks like this, otherwise the extension will not be able to parse the xml correctly.
-
-What happens when properties are missing
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+When properties or attributes are missing, testcases are still generated,
+but a testcase can only be linked to requirements if either ``PartiallyVerifies``
+or ``FullyVerifies`` is filled.
 
-When properties or attributes are missing for some or all tests inside the test.xml, testcases are still generated.
-However, testcases can only be linked to the requirements if either Partially- or FullyVerifies is filled.
-
-Language-Specific Annotations
------------------------------
-
-Each language uses a different mechanism to attach properties to test cases,
-but all produce the same JUnit XML output that the linker consumes.
-
-C++ (gTest)
-   Use ``RecordProperty`` — shared properties go in ``SetUp()``, per-test properties
-   inside each ``TEST_F``.
-   See :need:`gd_req__verification_link_tests_cpp`.
-
-Rust
-   Currently there is no provided official way to do this in Rust.
-   We are working with the rust community to figure this out.
-
-Python (pytest)
-   Use the ``@add_test_properties`` decorator; the docstring serves as ``Description``.
-   See :need:`gd_req__verification_link_tests_python`.
+Testcase result annotations
+---------------------------
 
-See :need:`gd_temp__verification_specification` for code templates.
+GitHub test links are decorated with their result, for example ``(passed)``,
+``(failed)``, ``(skipped)``, or ``(disabled)``. The annotation is applied to
+rendered links that target a testcase need, including the ``testlink`` entries
+shown on requirements. The status text inherits the surrounding theme colour
+and uses CSS classes with colours selected for the current S-CORE light and
+dark themes. These colours are not configurable for arbitrary Sphinx themes.
+Testcases without a result are left unchanged; links whose GitHub URL
+identifies multiple testcases are also left unchanged because their result
+would be ambiguous.
 
 
 Running Tests and Building Docs
@@ -120,7 +129,7 @@ Running Tests and Building Docs
 
       bazel test //...
 
-2. Build the documentation — the linker picks up ``bazel-testlogs/`` automatically:
+2. Build the documentation — the generated test results are picked up automatically:
 
    .. code-block:: bash
 
@@ -183,4 +192,4 @@ Limitations
 -----------
 
 - Tests must be executed by Bazel before building docs so ``test.xml`` files exist.
-- Not compatible with Esbonio / live preview (no ``bazel-testlogs/`` available).
+- Not compatible with Esbonio / live preview because generated test results are unavailable there.

src/extensions/docs/source_code_linker.md

@@ -381,6 +381,7 @@ score_source_code_linker/
 ├── needlinks.py                 # CodeLink dataclass & JSON encoder/decoder
 ├── testlink.py                  # DataForTestLink definition & logic
 ├── xml_parser.py                # Parses XML files into test case data
+├── testcase_annotations.py      # Adds result annotations to GitHub testcase links
 ├── tests/                       # Testsuite, containing unit & integration tests
 │   └── ...
 ```

src/extensions/score_source_code_linker/__init__.py

@@ -51,6 +51,9 @@
     load_repo_source_links_json,
     store_repo_source_links_json,
 )
+from src.extensions.score_source_code_linker.testcase_annotations import (
+    annotate_testcase_results,
+)
 from src.extensions.score_source_code_linker.testlink import (
     DataForTestLink,
     load_data_of_test_case_json,
@@ -140,15 +143,24 @@ def setup_source_code_linker(app: Sphinx, ws_root: Path | None):
         description="Skip rescanning source code files via the source code linker.",
     )
 
-    # Define need_string_links here to not have it in conf.py
-    # source_code_link and testlinks have the same schema
+    # Define need_string_links here to not have them in conf.py. Test links
+    # carry the result as an additional field in their serialized value.
     app.config.needs_string_links.setdefault(
-        "source_code_linker",
+        "source_code_linker_pure",
         {
             "regex": r"(?P<url>.+)<>(?P<name>.+)",
             "link_url": "{{url}}",
             "link_name": "{{name}}",
-            "options": ["source_code_link", "testlink"],
+            "options": ["source_code_link"],
+        },
+    )
+    app.config.needs_string_links.setdefault(
+        "test_code_linker",
+        {
+            "regex": r"(?P<url>.+)<>(?P<name>.+)<>(?P<result>.+)",
+            "link_url": "{{url}}",
+            "link_name": "{{name}} ({{result}})",
+            "options": ["testlink"],
         },
     )
 
@@ -323,6 +335,10 @@ def setup_once(app: Sphinx):
     # Priority=515 to ensure it's called after the test linker & combined connection
     app.connect("env-updated", inject_links_into_needs, priority=525)
 
+    # sphinx-needs resolves NeedIncoming and need metadata links on the same event.
+    # Run after those resolvers so the GitHub references are available to annotate.
+    app.connect("doctree-resolved", annotate_testcase_results, priority=800)
+
 
 def setup(app: Sphinx) -> dict[str, str | bool]:
     # Esbonio will execute setup() on every iteration.
@@ -460,7 +476,7 @@ def _render_test_link(
             type="score_source_code_linker",
         )
         return str(link.name)
-    return f"{base}<>{link.name}"
+    return f"{base}<>{link.name}<>{link.result}"
 
 
 def _warn_missing_need(source_code_links: object) -> None:

src/extensions/score_source_code_linker/testcase_annotations.py

@@ -0,0 +1,135 @@
+# *******************************************************************************
+# Copyright (c) 2026 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# SPDX-License-Identifier: Apache-2.0
+# *******************************************************************************
+"""Color execution results in rendered GitHub testcase links.
+
+The ``testlink`` string-link configuration renders each link as
+``<testcase name> (<result>)``. This hook replaces the plain result suffix with
+the corresponding colored HTML span after sphinx-needs has created the link.
+"""
+
+from __future__ import annotations
+
+from html import escape
+
+from docutils import nodes
+
+# Known result values get semantic classes so the stylesheet can provide
+# readable colours for the current S-CORE light and dark themes.
+RESULT_CLASSES = {
+    "passed": "score-testcase-result--passed",
+    "failed": "score-testcase-result--failed",
+    "skipped": "score-testcase-result--skipped",
+    "disabled": "score-testcase-result--disabled",
+}
+# The event handler is expected to be idempotent for a doctree. This marker
+# prevents a second invocation from appending the same status again.
+_ANNOTATED_ATTR = "score_source_code_linker_testcase_result_annotated"
+
+# The styles are inserted into a document only when that document contains an
+# annotation. Keeping them in one block avoids repeating inline style rules on
+# every testcase link and lets the same classes handle theme changes.
+_TESTCASE_STATUS_CSS = """
+<style>
+.score-testcase-result {
+  font-weight: bold;
+}
+.score-testcase-result--passed {
+  color: #146c2e;
+}
+.score-testcase-result--failed {
+  color: #b42318;
+}
+.score-testcase-result--skipped {
+  color: #8a5300;
+}
+.score-testcase-result--disabled {
+  color: #5f6368;
+}
+html[data-theme="dark"] .score-testcase-result--passed {
+  color: #7ee787;
+}
+html[data-theme="dark"] .score-testcase-result--failed {
+  color: #ff7b72;
+}
+html[data-theme="dark"] .score-testcase-result--skipped {
+  color: #d29922;
+}
+html[data-theme="dark"] .score-testcase-result--disabled {
+  color: #c4cad2;
+}
+@media (prefers-color-scheme: dark) {
+  html:not([data-theme="light"]) .score-testcase-result--passed {
+    color: #7ee787;
+  }
+  html:not([data-theme="light"]) .score-testcase-result--failed {
+    color: #ff7b72;
+  }
+  html:not([data-theme="light"]) .score-testcase-result--skipped {
+    color: #d29922;
+  }
+  html:not([data-theme="light"]) .score-testcase-result--disabled {
+    color: #c4cad2;
+  }
+}
+</style>
+"""
+
+
+def _result_node(result_text: str, result_class: str) -> nodes.raw:
+    escaped_result = escape(result_text, quote=True)
+    status_html = (
+        f'<span class="score-testcase-result {result_class}"> ({escaped_result})</span>'
+    )
+    return nodes.raw("", status_html, format="html")
+
+
+def _color_existing_result_suffix(ref: nodes.reference) -> bool:
+    """Replace a recognized ``(result)`` suffix with a colored node."""
+    if not ref.children or not isinstance(ref.children[-1], nodes.Text):
+        return False
+
+    last_text = ref.children[-1]
+    text = last_text.astext()
+    for result_text, result_class in RESULT_CLASSES.items():
+        suffix = f" ({result_text})"
+        if text.endswith(suffix):
+            prefix = text[: -len(suffix)]
+            ref.replace(last_text, nodes.Text(prefix))
+            ref.append(_result_node(result_text, result_class))
+            return True
+    return False
+
+
+def annotate_testcase_results(app, doctree, docname):
+    """Color rendered testcase result suffixes using the S-CORE theme palette.
+
+    The handler runs after sphinx-needs' own ``doctree-resolved`` handlers.
+    It therefore sees the external references generated for GitHub ``testlink``
+    metadata, whose labels already contain the result.
+    """
+    # CSS applies to the whole document, so one style block is enough even if
+    # the document contains many annotated references.
+    css_added = False
+
+    for ref in list(doctree.findall(nodes.reference)):
+        if ref.get(_ANNOTATED_ATTR):
+            # A repeated event invocation must not append another badge.
+            continue
+
+        if not _color_existing_result_suffix(ref):
+            continue
+
+        if not css_added:
+            doctree.insert(0, nodes.raw("", _TESTCASE_STATUS_CSS, format="html"))
+            css_added = True
+        ref[_ANNOTATED_ATTR] = True

src/extensions/score_source_code_linker/tests/test_source_code_link_integration.py

@@ -250,8 +250,25 @@ def basic_conf():
         color="#BFD8D2",
         style="node",
     ),
+    dict(
+        directive="testcase",
+        title="Testcase",
+        prefix="testcase__",
+        color="#D9EAD3",
+        style="node",
+    ),
+]
+needs_extra_options = [
+    "source_code_link",
+    "testlink",
+    "name",
+    "file",
+    "line",
+    "test_type",
+    "derivation_technique",
+    "result",
+    "result_text",
 ]
-needs_extra_options = ["source_code_link", "testlink"]
 needs_extra_links =  [{
       "option": "partially_verifies",
       "incoming": "paritally_verified_by",
@@ -432,7 +449,9 @@ def make_test_link(testlinks: list[DataForTestLink]):
         url="",
         hash="",
     )
-    return ", ".join(f"{get_github_link(metadata, n)}<>{n.name}" for n in testlinks)
+    return ", ".join(
+        f"{get_github_link(metadata, n)}<>{n.name}<>{n.result}" for n in testlinks
+    )
 
 
 def compare_json_files(
@@ -552,6 +571,11 @@ def test_source_link_integration_ok(
             expected_test_link = make_test_link(example_test_link_text_all_ok[treq_id])
             actual_test_code_link = treq_info.get("testlink", "no test link")
             assert expected_test_link == actual_test_code_link, treq_id
+
+        # The rendered testlink references point at the testcase external needs.
+        # Verify the annotation survives the complete Sphinx HTML rendering path.
+        rendered_html = (app.outdir / "index.html").read_text(encoding="utf-8")
+        assert "(passed)" in rendered_html
     finally:
         app.cleanup()