Skip to content

[DRAFT] refactor(tests): replace cuopt_static with CUOPT_INTERNAL_EXPORT + version script#1574

Closed
ramakrishnap-nv wants to merge 12 commits into
mainfrom
fix/internal-export-for-tests
Closed

[DRAFT] refactor(tests): replace cuopt_static with CUOPT_INTERNAL_EXPORT + version script#1574
ramakrishnap-nv wants to merge 12 commits into
mainfrom
fix/internal-export-for-tests

Conversation

@ramakrishnap-nv

Copy link
Copy Markdown
Collaborator

Builds on #1566. Eliminates cuopt_static from all test linkage by introducing CUOPT_INTERNAL_EXPORT for the small set of internal symbols that test TUs reference directly. A GNU ld version script (cmake/cuopt.map) assigns public symbols to CUOPT_1.0 and test-only internals to CUOPT_PRIVATE_1.0. Only generator::dataset_t and generate_dataset needed marking — all other routing internal types are template/inline and instantiate in-place. Removes ConfigureInternalTest entirely; all tests now use ConfigureTest (shared linkage).

arhag23 and others added 9 commits July 13, 2026 16:42
Signed-off-by: Arha Gatram <agatram@nvidia.com>
…static archive

Signed-off-by: Arha Gatram <agatram@nvidia.com>
Signed-off-by: Arha Gatram <agatram@nvidia.com>
Signed-off-by: Arha Gatram <agatram@nvidia.com>
PR #1526 switched all tests to link cuopt_static, which inflates
libcuopt-tests from ~75 MB to ~5 GB (confirmed from CI artifacts)
because each of ~36 test executables embeds the full cuopt object graph.

This commit introduces ConfigureInternalTest for tests that genuinely
need access to hidden symbols, and keeps ConfigureTest on the shared
library for tests that only use the public API.

Only routing tests are switched to ConfigureInternalTest: they all
include routing_test.cuh which pulls in cuopt::routing::detail types
(solution_t, problem_t, fleet_order_constraints_t, md_utils, etc.).

MIP, LP, QP, SOCP, dual_simplex, distance engine, CLI, and utility
tests use only public cuopt API or raft::detail inline headers (which
are compiled into the TU, not linked from cuopt.so), so they stay on
the shared library.

GRPC tests are left unchanged as they have explicit target_link_libraries
and grpc_client.cpp symbols lack CUOPT_EXPORT.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci/check_symbols.sh and conda/recipes/libcuopt/recipe.yaml were missing
the canonical suffix required by verify-copyright pre-commit hook.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ramakrishna Prabhu <ramakrishnap@nvidia.com>
…rsion script

Add CUOPT_INTERNAL_EXPORT to export.hpp for symbols that must be visible in the
shared library solely for test access, distinct from the stable public API.

Mark generator::dataset_t and generate_dataset with CUOPT_INTERNAL_EXPORT — the
only routing symbols whose compiled methods are called directly from test TUs
(all other routing internal types are template/inline and instantiate in-place).

Add cmake/cuopt.map linker version script assigning public symbols to CUOPT_1.0
and test-only internal symbols to CUOPT_PRIVATE_1.0. Wire it into the cuopt
shared library link step.

Remove ConfigureInternalTest and switch cuopttestutils + all routing tests to
link against the cuopt shared library. This eliminates cuopt_static from the
test dependency graph, keeping test binary sizes small.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ramakrishnap-nv ramakrishnap-nv requested review from a team as code owners July 14, 2026 17:57
@ramakrishnap-nv ramakrishnap-nv requested review from Bubullzz, mlubin and tmckayus and removed request for a team July 14, 2026 17:57
Comment thread cpp/cmake/cuopt.map Outdated
#
# GNU ld version script for libcuopt.so.
#
# CUOPT_1.0 — stable public API (CUOPT_EXPORT symbols)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of the C++ API is part of the stable ABI from what I understand. Only the C and Python interfaces are.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I am currently trying a way to work on a way which would solve this problem of internal API usage as a whole. So just testing as of now.

@ramakrishnap-nv ramakrishnap-nv changed the title refactor(tests): replace cuopt_static with CUOPT_INTERNAL_EXPORT + version script [DRAFT] refactor(tests): replace cuopt_static with CUOPT_INTERNAL_EXPORT + version script Jul 14, 2026
@ramakrishnap-nv ramakrishnap-nv self-assigned this Jul 14, 2026
@ramakrishnap-nv ramakrishnap-nv added non-breaking Introduces a non-breaking change improvement Improves an existing functionality labels Jul 14, 2026
…ract

Remove cmake/cuopt.map and --version-script from the cuopt shared lib link
step. The version node distinction (CUOPT_1.0 vs CUOPT_PRIVATE_1.0) implied
a C++ ABI stability commitment that does not exist: only the C and Python
interfaces are stable. CUOPT_INTERNAL_EXPORT as a source annotation is
sufficient to communicate intent without making false ABI claims.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown

CI Test Summary

✅ All 31 test job(s) passed.

@ramakrishnap-nv ramakrishnap-nv changed the base branch from fix-symbol-visibility-selective to main July 14, 2026 18:10
pdlp_solver_t template instantiations were not marked with
CUOPT_INTERNAL_EXPORT, so they were hidden in the shared library
after switching to hidden visibility. pdlp_test links against the
shared cuopt library and directly instantiates pdlp_solver_t<int,double>,
causing linker errors.
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

The change adds CUOPT visibility macros, applies them across public headers and explicit template instantiations, restructures CMake targets around an object library, adds dynamic-symbol validation, and runs that validation during the libcuopt package build.

Changes

Visibility, build, and symbol validation

Layer / File(s) Summary
Export contracts
cpp/include/cuopt/...
Adds visibility macros and applies them to public namespaces, C API declarations, and internal declarations.
Exported instantiations
cpp/src/grpc/..., cpp/src/io/..., cpp/src/pdlp/..., cpp/src/routing/..., cpp/src/mip_heuristics/...
Marks explicit template instantiations with the appropriate export macro.
Object and test library wiring
cpp/CMakeLists.txt, cpp/tests/...
Builds shared and static targets from common objects and updates test linkage selection.
Symbol checking and packaging
ci/check_symbols.sh, conda/recipes/libcuopt/recipe.yaml
Checks exported symbols for disallowed patterns and runs the check during package creation.

Estimated code review effort: 4 (Complex) | ~60 minutes

Suggested reviewers: tmckayus, bubullzz, mlubin

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the main change: moving tests off cuopt_static toward exported internals and a version script.
Description check ✅ Passed The description directly describes the linkage refactor and export/visibility changes in the pull request.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/internal-export-for-tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ci/check_symbols.sh`:
- Around line 61-64: Update the non-trailing-scope matching logic in
has_trailing_scope so the pslp_ pattern matches symbols beginning with the plain
prefix pslp_, while preserving existing exact and ::-qualified matching behavior
for other patterns. Add a regression fixture covering a symbol such as pslp_foo.

In `@cpp/include/cuopt/export.hpp`:
- Line 8: Replace `#pragma` once with a unique `#ifndef/`#define include guard and
matching closing `#endif` in cpp/include/cuopt/export.hpp (lines 8-8) and
cpp/include/cuopt/routing/assignment.hpp (lines 8-8), ensuring each header uses
a distinct guard name.

In `@cpp/tests/linear_programming/grpc/CMakeLists.txt`:
- Line 27: Replace all three cuopt_static references in the gRPC test target
definitions with cuopt, ensuring every test links against the shared target and
validates its exported symbols.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ceb19a9b-2b27-4132-bd57-a69cde59eec5

📥 Commits

Reviewing files that changed from the base of the PR and between cd8ddad and 8d7160b.

📒 Files selected for processing (73)
  • ci/check_symbols.sh
  • conda/recipes/libcuopt/recipe.yaml
  • cpp/CMakeLists.txt
  • cpp/include/cuopt/error.hpp
  • cpp/include/cuopt/export.hpp
  • cpp/include/cuopt/grpc/cython_grpc_client.hpp
  • cpp/include/cuopt/grpc/grpc_client_env.hpp
  • cpp/include/cuopt/mathematical_optimization/backend_selection.hpp
  • cpp/include/cuopt/mathematical_optimization/cpu_optimization_problem.hpp
  • cpp/include/cuopt/mathematical_optimization/cpu_optimization_problem_solution.hpp
  • cpp/include/cuopt/mathematical_optimization/cpu_pdlp_warm_start_data.hpp
  • cpp/include/cuopt/mathematical_optimization/cuopt_c.h
  • cpp/include/cuopt/mathematical_optimization/io/data_model_view.hpp
  • cpp/include/cuopt/mathematical_optimization/io/mps_data_model.hpp
  • cpp/include/cuopt/mathematical_optimization/io/mps_writer.hpp
  • cpp/include/cuopt/mathematical_optimization/io/parser.hpp
  • cpp/include/cuopt/mathematical_optimization/io/utilities/cython_parser.hpp
  • cpp/include/cuopt/mathematical_optimization/io/writer.hpp
  • cpp/include/cuopt/mathematical_optimization/mip/solver_settings.hpp
  • cpp/include/cuopt/mathematical_optimization/mip/solver_solution.hpp
  • cpp/include/cuopt/mathematical_optimization/optimization_problem.hpp
  • cpp/include/cuopt/mathematical_optimization/optimization_problem_solution.hpp
  • cpp/include/cuopt/mathematical_optimization/pdlp/pdlp_warm_start_data.hpp
  • cpp/include/cuopt/mathematical_optimization/pdlp/solver_settings.hpp
  • cpp/include/cuopt/mathematical_optimization/pdlp/solver_solution.hpp
  • cpp/include/cuopt/mathematical_optimization/solve.hpp
  • cpp/include/cuopt/mathematical_optimization/solve_remote.hpp
  • cpp/include/cuopt/mathematical_optimization/solver_settings.hpp
  • cpp/include/cuopt/mathematical_optimization/utilities/cython_solve.hpp
  • cpp/include/cuopt/mathematical_optimization/utilities/cython_types.hpp
  • cpp/include/cuopt/routing/assignment.hpp
  • cpp/include/cuopt/routing/cython/cython.hpp
  • cpp/include/cuopt/routing/data_model_view.hpp
  • cpp/include/cuopt/routing/routing_structures.hpp
  • cpp/include/cuopt/routing/solve.hpp
  • cpp/include/cuopt/routing/solver_settings.hpp
  • cpp/src/grpc/client/solve_remote.cpp
  • cpp/src/grpc/grpc_problem_mapper.cpp
  • cpp/src/grpc/grpc_settings_mapper.cpp
  • cpp/src/grpc/grpc_solution_mapper.cpp
  • cpp/src/io/data_model_view.cpp
  • cpp/src/io/lp_parser.cpp
  • cpp/src/io/mps_data_model.cpp
  • cpp/src/io/mps_writer.cpp
  • cpp/src/io/parser.cpp
  • cpp/src/io/writer.cpp
  • cpp/src/math_optimization/solution_reader.hpp
  • cpp/src/math_optimization/solver_settings.cu
  • cpp/src/mip_heuristics/solve.cu
  • cpp/src/mip_heuristics/solver_settings.cu
  • cpp/src/mip_heuristics/solver_solution.cu
  • cpp/src/pdlp/cpu_optimization_problem.cpp
  • cpp/src/pdlp/cpu_pdlp_warm_start_data.cu
  • cpp/src/pdlp/optimization_problem.cu
  • cpp/src/pdlp/pdlp.cu
  • cpp/src/pdlp/pdlp_warm_start_data.cu
  • cpp/src/pdlp/solution_conversion.cu
  • cpp/src/pdlp/solve.cu
  • cpp/src/pdlp/solver_settings.cu
  • cpp/src/pdlp/solver_solution.cu
  • cpp/src/routing/assignment.cu
  • cpp/src/routing/data_model_view.cu
  • cpp/src/routing/distance_engine/waypoint_matrix.cpp
  • cpp/src/routing/generator/generator.hpp
  • cpp/src/routing/solve.cu
  • cpp/src/routing/solver_settings.cu
  • cpp/src/routing/utilities/cython.cu
  • cpp/src/utilities/logger.hpp
  • cpp/tests/CMakeLists.txt
  • cpp/tests/examples/routing/CMakeLists.txt
  • cpp/tests/linear_programming/CMakeLists.txt
  • cpp/tests/linear_programming/grpc/CMakeLists.txt
  • cpp/tests/routing/CMakeLists.txt

Comment thread ci/check_symbols.sh
Comment on lines +61 to +64
if (has_trailing_scope) {
matched = (index(symbol, pattern) == 1)
} else {
matched = (symbol == pattern || index(symbol, pattern "::") == 1)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== ci/check_symbols.sh (relevant lines) ==\n'
cat -n ci/check_symbols.sh | sed -n '1,180p'

printf '\n== occurrences of pslp_ and pattern handling ==\n'
rg -n 'pslp_|has_trailing_scope|index\\(symbol, pattern \\|\\| index\\(symbol, pattern "::"\\) == 1' -S .

Repository: NVIDIA/cuopt

Length of output: 3260


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== ci/check_symbols.sh excerpt =="
sed -n '50,80p' ci/check_symbols.sh | cat -n

echo
echo "== pslp_ references =="
rg -n '\bpslp_' ci . || true

Repository: NVIDIA/cuopt

Length of output: 4006


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '55,70p' ci/check_symbols.sh

Repository: NVIDIA/cuopt

Length of output: 621


Make pslp_ match the prefix. pslp_ goes through the non-:: branch, so it only matches the exact symbol pslp_ or symbols starting with pslp_::; plain prefixed symbols like pslp_foo are skipped. If this entry is meant to cover the PSLP symbol prefix, add explicit prefix handling and a regression fixture.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/check_symbols.sh` around lines 61 - 64, Update the non-trailing-scope
matching logic in has_trailing_scope so the pslp_ pattern matches symbols
beginning with the plain prefix pslp_, while preserving existing exact and
::-qualified matching behavior for other patterns. Add a regression fixture
covering a symbol such as pslp_foo.

Source: Path instructions

*/
/* clang-format on */

#pragma once

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use the repository-required include-guard form.

Both changed headers use #pragma once, while the supplied coding guideline requires #define include guards.

  • cpp/include/cuopt/export.hpp#L8-L8: replace #pragma once with a unique #ifndef/#define guard and closing #endif.
  • cpp/include/cuopt/routing/assignment.hpp#L8-L8: replace #pragma once with a unique #ifndef/#define guard and closing #endif.

As per coding guidelines, C++ headers must use #define include guards.

📍 Affects 2 files
  • cpp/include/cuopt/export.hpp#L8-L8 (this comment)
  • cpp/include/cuopt/routing/assignment.hpp#L8-L8
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/include/cuopt/export.hpp` at line 8, Replace `#pragma` once with a unique
`#ifndef/`#define include guard and matching closing `#endif` in
cpp/include/cuopt/export.hpp (lines 8-8) and
cpp/include/cuopt/routing/assignment.hpp (lines 8-8), ensuring each header uses
a distinct guard name.

Source: Coding guidelines

target_link_libraries(GRPC_CLIENT_TEST
PRIVATE
cuopt
cuopt_static

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Link the gRPC tests against the shared cuopt target.

All three targets still use cuopt_static, which contradicts this PR’s shared-library test-linkage refactor and bypasses validation of the new exported symbols. Replace each occurrence with cuopt.

Proposed fix
-    cuopt_static
+    cuopt

Apply this change at all three locations.

Also applies to: 70-70, 109-109

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/tests/linear_programming/grpc/CMakeLists.txt` at line 27, Replace all
three cuopt_static references in the gRPC test target definitions with cuopt,
ensuring every test links against the shared target and validates its exported
symbols.

@ramakrishnap-nv ramakrishnap-nv added this to the 26.08 milestone Jul 14, 2026
With hidden symbol visibility on libcuopt.so, any internal class or
function directly referenced by a test binary must be explicitly
exported. Two strategies applied:

1. CUOPT_INTERNAL_EXPORT on internal symbols that tests access through
   the shared library: pdlp_solver_t, ges_solver_t, solver_t,
   csc/csr_matrix_t, sparse_vector_t, tic/toc, dual-simplex free
   functions (scaling, presolve, right_looking_ldlt, solve),
   mip_solver_t, problem_t, solution_t, various presolve/cut classes,
   and is_cusparse_runtime_mixed_precision_supported.

2. STATIC_LIB option in ConfigureTest for tests that access deeply
   internal symbols (seed_generator::seed_, clique_table_t,
   problem_checking_t, etc.) that are impractical to export: the
   six failing MIP unit tests, LP/PDLP unit tests, and
   VEHICLE_TYPES_TEST now link against cuopt_static.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
cpp/tests/CMakeLists.txt (1)

25-28: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Avoid hard-coding cuopt in cuopttestutils.
This static utility can pull cuopt onto the test link line, so STATIC_LIB tests may end up with both cuopt_static and cuopt. Link the utility against ${_cuopt_link_target} instead, or drop the dependency here so the selected target stays consistent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/tests/CMakeLists.txt` around lines 25 - 28, Update the test utility link
dependencies around the GTest entries to avoid hard-coding cuopt; use
${_cuopt_link_target} or remove the dependency so STATIC_LIB tests do not link
both cuopt_static and cuopt and the selected target remains consistent.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@cpp/tests/CMakeLists.txt`:
- Around line 25-28: Update the test utility link dependencies around the GTest
entries to avoid hard-coding cuopt; use ${_cuopt_link_target} or remove the
dependency so STATIC_LIB tests do not link both cuopt_static and cuopt and the
selected target remains consistent.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2f330ce6-98bd-4f33-a910-1bfebc743b6d

📥 Commits

Reviewing files that changed from the base of the PR and between 8d7160b and 74dfed7.

📒 Files selected for processing (28)
  • cpp/src/cuts/cuts.cpp
  • cpp/src/dual_simplex/presolve.cpp
  • cpp/src/dual_simplex/right_looking_lu.cpp
  • cpp/src/dual_simplex/scaling.cpp
  • cpp/src/dual_simplex/solve.cpp
  • cpp/src/io/mps_parser.cpp
  • cpp/src/linear_algebra/sparse_matrix.cpp
  • cpp/src/linear_algebra/sparse_vector.cpp
  • cpp/src/math_optimization/tic_toc.hpp
  • cpp/src/mip_heuristics/feasibility_jump/feasibility_jump.cu
  • cpp/src/mip_heuristics/presolve/bounds_presolve.cu
  • cpp/src/mip_heuristics/presolve/load_balanced_bounds_presolve.cu
  • cpp/src/mip_heuristics/presolve/multi_probe.cu
  • cpp/src/mip_heuristics/presolve/third_party_presolve.cpp
  • cpp/src/mip_heuristics/problem/load_balanced_problem.cu
  • cpp/src/mip_heuristics/problem/problem.cu
  • cpp/src/mip_heuristics/solution/solution.cu
  • cpp/src/mip_heuristics/solver.cu
  • cpp/src/pdlp/cusparse_view.cu
  • cpp/src/routing/ges_solver.cu
  • cpp/src/routing/local_search/local_search.cu
  • cpp/src/routing/problem/problem.cu
  • cpp/src/routing/solution/solution.cu
  • cpp/src/routing/solver.cu
  • cpp/tests/CMakeLists.txt
  • cpp/tests/linear_programming/CMakeLists.txt
  • cpp/tests/mip/CMakeLists.txt
  • cpp/tests/routing/CMakeLists.txt
🚧 Files skipped from review as they are similar to previous changes (1)
  • cpp/tests/routing/CMakeLists.txt

@ramakrishnap-nv

Copy link
Copy Markdown
Collaborator Author

Closing to rethink approach. The mix of CUOPT_INTERNAL_EXPORT + static-linked tests is not clean. Plan: assess which tests depend on internal APIs, determine if they're valuable, and consolidate into single per-module internal test binaries (one for routing, one for numerical optimization) that link against cuopt_static.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants