Skip to content

fix(build): honor disabled TensorFlow for native interfaces - #6051

Merged
wanghan-iapcm merged 4 commits into
deepmodeling:masterfrom
Zhang-pchao:codex/deepmd-native-backend
Nov 7, 2026
Merged

wanghan-iapcm merged 4 commits into
deepmodeling:masterfrom
Zhang-pchao:codex/deepmd-native-backend

Conversation

@Zhang-pchao

@Zhang-pchao Zhang-pchao commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When an i-PI or LAMMPS interface is requested, derive USE_TF_PYTHON_LIBS from DP_ENABLE_TENSORFLOW instead of unconditionally enabling it. This keeps a PyTorch-only native-interface build from requesting TensorFlow libraries. The existing default behavior, including the default TensorFlow-enabled setting, is preserved.

Tests

A backend-independent regression exercises 9 combinations: Python/i-PI/LAMMPS interfaces and DP_ENABLE_TENSORFLOW set to 0, 1, or unset. It checks that disabled TensorFlow is not discovered and verifies the resulting CMake flags and dp_ipi entry point.

  • PYTHONPATH=. python -m pytest -q source/tests/test_build_backend.py: 9 passed.
  • All applicable repository pre-commit hooks passed after import formatting.

Based on current master (7a9510d). This is an independent build-configuration fix and does not modify model evaluation or introduce new backend dependencies.

Summary by CodeRabbit

  • Bug Fixes
    • Build configuration now applies the TensorFlow setting consistently across supported interfaces. When TensorFlow is disabled, the build no longer attempts to discover or configure it.
  • Chores
    • Updated optional test dependencies to include tomli for Python versions below 3.11, while retaining dependency group support.
  • Tests
    • Added coverage for TensorFlow build settings across the Python, i-PI, and LAMMPS interfaces, including disabled TensorFlow behavior.

CI follow-up (2026-10-07)

The exact-head Read the Docs build fails while installing doc/requirements.txt, before Sphinx runs: the selected PyTorch headers require C++20, while the current target baseline selects C++17. The same build-standard mismatch appears on all five related contribution heads (#6051 through #6055). Existing issue #6048 and PR #6049 already address this compatibility problem; this PR leaves that repair in its independent upstream review. The Read the Docs check remains failed until that dependency is resolved and the build is rerun. The exact-head pre-commit.ci check passes; GitHub Actions still require maintainer approval.

@github-actions github-actions Bot added the Python label Oct 6, 2026
@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The backend reads DP_ENABLE_TENSORFLOW once and uses that value to configure the C++ interface and TensorFlow setup. A parameterized test checks build arguments and script selection across Python, i-PI, and LAMMPS interfaces. The test dependency group adds tomli for Python versions below 3.11 and retains dependency_groups.

Changes: TensorFlow build selection

Layer / File(s) Summary
Apply TensorFlow selection and test interface options
backend/read_env.py, source/tests/test_build_backend.py, pyproject.toml
The backend uses the stored TensorFlow setting for C++ interface configuration and TensorFlow setup. The regression test checks interface and TensorFlow setting combinations. The test dependency group adds tomli for Python versions below 3.11 and retains dependency_groups.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Suggested labels: bug, LAMMPS

Suggested reviewers: njzjz

Merge Risk: Minimal · up to 8868e

No concrete merge-blocking risk was identified. The documentation build remains reported as failing on a separate PyTorch compatibility issue.

Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the main change: honoring disabled TensorFlow for native interfaces.
Docstring Coverage Passed Docstring coverage is inconclusive: files from earlier reviews still need to be rechecked within the check's analysis limits. Not blocking.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 750fe48b-588d-4a00-a48f-e8e26d1175a0
Commits

Reviewing files that changed from the base of the PR and between a866ca2 and 8868e1c.

Files selected for processing: 1

  • source/tests/test_build_backend.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

@njzjz-bot njzjz-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent: dot

Reviewed exact head 063b873. No blocking findings in this scoped build-option fix.

Traced the native-interface flags through source/CMakeLists.txt: USE_TF_PYTHON_LIBS=TRUE re-enables ENABLE_TENSORFLOW there, so deriving that flag from the same boolean as TensorFlow discovery removes the contradiction for PyTorch-only builds. The default/unset and explicit-enabled paths preserve the prior flags; i-PI script selection and LAMMPS version forwarding remain unchanged. The nine-case regression checks the Python/i-PI/LAMMPS matrix, asserts disabled TensorFlow is never discovered, and clears the cached environment result before and after each invocation.

Validation: source review of both changed files plus backend discovery and CMake consumers. I could not rerun pytest in this review environment because pytest is not installed; no full native build was performed. This review does not approve or run fork CI.

@Zhang-pchao Zhang-pchao changed the title Honor disabled TensorFlow when configuring native interfaces fix(build): honor disabled TensorFlow for native interfaces Oct 7, 2026
@Zhang-pchao

Zhang-pchao commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor Author

The Python collection failure was a missing test dependency: test_build_backend.py imports the build backend directly, but dependency_groups was only installed in the isolated build environment. Commit 6ea84f88ecd86c2bdfeb2f01f0898f72f3782b5a adds it to the test extra, together with the existing conditional tomli requirement for Python < 3.11.

The nine build-option regressions pass in clean Python 3.10.22 and 3.13.16 environments, with no skips. The repository's pre-commit hooks and Ruff checks pass too.

The previous C++ sanitizer crash and LAMMPS exit 143 are still under investigation; this dependency fix does not resolve or suppress them. I am checking those paths against the exact base with matched dependencies. The updated upstream workflows currently require approval.

The new-head Read the Docs build 35031571 also fails during installation of doc/requirements.txt, with PyTorch/ATen C++20 header errors. This is the build compatibility issue tracked by #6049, which remains open; I have kept that separate from this patch.

@wanghan-iapcm wanghan-iapcm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix is correct and complete. The unconditional -DUSE_TF_PYTHON_LIBS:BOOL=TRUE for LAMMPS/i-PI builds did override -DENABLE_TENSORFLOW=OFF through source/CMakeLists.txt:334-335, and deriving the flag from DP_ENABLE_TENSORFLOW closes that hole: I traced every consumer of USE_TF_PYTHON_LIBS (the root CMakeLists.txt, Findtensorflow.cmake, the source/lmp/source/ipi env templates and source/install/package_c.sh) and the two flags now agree in every documented 0/1/unset cell. The new test genuinely fails on the pre-PR line, and the extra test dependencies (dependency_groups, tomli for Python < 3.11) match what the backend import chain needs.

The remaining observations are non-blocking and recorded for a later cleanup: the test never asserts the -DTENSORFLOW_VERSION / -DTENSORFLOW_ROOT:PATH arguments emitted by the block whose condition this PR rewrote; it mocks and executes the PyTorch branch in every case but asserts nothing about -DENABLE_PYTORCH / -DCMAKE_PREFIX_PATH / the returned version; the module-scope from backend import read_env breaks collection in non-editable environments such as the documented pytest source/tests tox command, where the sibling source/tests/common/test_pytorch_requirements.py instead uses monkeypatch.syspath_prepend; and the commit subject "Honor disabled TensorFlow when configuring native interfaces" does not follow the repository's conventional-commit format.

@codecov

codecov Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.85%. Comparing base (1ded5d2) to head (8868e1c).
⚠️ Report is 19 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6051      +/-   ##
==========================================
- Coverage   78.36%   77.85%   -0.51%     
==========================================
  Files        1179     1207      +28     
  Lines      142929   146598    +3669     
  Branches     5079     5217     +138     
==========================================
+ Hits       112002   114132    +2130     
- Misses      28990    30502    +1512     
- Partials     1937     1964      +27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

auto-merge was automatically disabled November 7, 2026 15:52

Head branch was pushed to by a user without write access

@wanghan-iapcm
wanghan-iapcm added this pull request to the merge queue Nov 7, 2026
Merged via the queue into deepmodeling:master with commit 4e3264d Nov 7, 2026
58 checks passed
yckbz pushed a commit to yckbz/deepmd-kit that referenced this pull request Nov 8, 2026
…eepmodeling#6052)

## Summary

Do not pass charge_spin=None to scripted model methods that predate that
optional argument. Preserve the keyword when a charge/spin tensor is
supplied. Apply the same rule in inference and the model wrapper.

The regression loads the repository's legacy scripted fixture and
compares native versus external (vesin) neighbor-list evaluation for
energy, forces and virial.

## Validation

Based on current master (7a9510d).

- `pytest -q source/tests/pt/model/test_nlist_backend.py`: 12 passed, 7
conditional GPU/backend skips. The new legacy-model regression executed
and passed with vesin.
- All applicable repository pre-commit hooks passed, including pylint.

Tests used the proposed Python source with the existing compatible
PyTorch 2.11 native operators. This is a Python schema-compatibility
fix; a full native-backend rebuild was not required for that focused
test.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved compatibility with older models that do not accept optional
charge-spin data. Explicitly provided charge-spin values continue to be
passed through during inference and training.
* Added a regression check confirming consistent evaluation results
between native and external neighbor-list backends for a legacy model,
helping ensure users receive matching results across these options.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

## CI follow-up (2026-10-07)

The exact-head Read the Docs build fails while installing
`doc/requirements.txt`, before Sphinx runs: the selected PyTorch headers
require C++20, while the current target baseline selects C++17. The same
build-standard mismatch appears on all five related contribution heads
(deepmodeling#6051 through deepmodeling#6055). Existing issue deepmodeling#6048 and PR deepmodeling#6049 already address
this compatibility problem; this PR leaves that repair in its
independent upstream review. The Read the Docs check remains failed
until that dependency is resolved and the build is rerun. The exact-head
pre-commit.ci check passes; GitHub Actions still require maintainer
approval.

---------

Co-authored-by: Pengchao_Zhang <zpc20@mails.tsinghua.edu.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants