-
Notifications
You must be signed in to change notification settings - Fork 31
Add capability for cmake to fetch libtorch itself if not found #173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: skala
Are you sure you want to change the base?
Conversation
…sion is probably ok (at leat on my mac it works), but gpu still needs to filter versions correctly.
…loads are available. Still needs to be tested.
…nd_package(CUDA REQUIRED) needed to included. The CUDA_VERSION_STRING conflicts, so renamed that to CUDA_VERSION_NO_DOT.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds automatic LibTorch download functionality to the CMake build system, eliminating the requirement for users to manually install PyTorch/LibTorch before building the project. The implementation handles platform-specific downloads (Linux, macOS, Windows) and includes CUDA-aware selection for GPU support.
Changes:
- Added new CMake module
skala-torch.cmakethat detects and downloads LibTorch if not found on the system - Modified
gauxc-onedft.cmaketo use the new module instead of directly callingfind_package(Torch)
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 13 comments.
| File | Description |
|---|---|
| cmake/skala-torch.cmake | New module implementing LibTorch auto-download with platform detection, CUDA version matching, and fallback to CPU version |
| cmake/gauxc-onedft.cmake | Replaced direct Torch package search with include of new skala-torch module |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if(NOT EXISTS ${LIBTORCH_DOWNLOAD_DIR}) | ||
| message(STATUS "Downloading libtorch from ${LIBTORCH_URL}") | ||
| file(DOWNLOAD ${LIBTORCH_URL} ${LIBTORCH_ZIP} | ||
| SHOW_PROGRESS | ||
| STATUS DOWNLOAD_STATUS) | ||
|
|
||
| list(GET DOWNLOAD_STATUS 0 STATUS_CODE) | ||
| if(NOT STATUS_CODE EQUAL 0) | ||
| message(FATAL_ERROR "Failed to download libtorch: ${DOWNLOAD_STATUS}") | ||
| endif() | ||
|
|
||
| # Extract libtorch | ||
| message(STATUS "Extracting libtorch...") | ||
| file(ARCHIVE_EXTRACT INPUT ${LIBTORCH_ZIP} DESTINATION ${CMAKE_BINARY_DIR}) | ||
|
|
||
| # Clean up zip file | ||
| file(REMOVE ${LIBTORCH_ZIP}) | ||
| endif() |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The check for existing LibTorch directory may lead to using an incompatible or corrupted installation. If a previous download was interrupted or if the directory exists but is incomplete/corrupted, the script will skip downloading and proceed to use the potentially broken installation. Consider adding validation (e.g., checking for a specific file like libtorch/share/cmake/Torch/TorchConfig.cmake) or using a stamp file to verify successful extraction.
| @@ -0,0 +1,76 @@ | |||
| # Find or download LibTorch | |||
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filename inconsistency: This file is named "skala-torch.cmake" but other dependency management files in the cmake directory use the "gauxc-" prefix (e.g., gauxc-eigen3.cmake, gauxc-cub.cmake, gauxc-exchcxx.cmake). For consistency with the established naming convention, consider renaming this file to "gauxc-torch.cmake" or similar.
|
|
||
| # Set libtorch version and download URL | ||
| set(LIBTORCH_VERSION "2.9.1") | ||
| set(USE_CUDA_LIBTORCH FALSE) #default is not to use the cuda version but cpu version |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after '#' in comment. For consistency with CMake commenting style used elsewhere in the codebase, there should be a space between the '#' and the comment text.
| set(USE_CUDA_LIBTORCH FALSE) #default is not to use the cuda version but cpu version | |
| set(USE_CUDA_LIBTORCH FALSE) # default is not to use the cuda version but cpu version |
| # Find or download LibTorch | ||
| find_package(Torch QUIET) | ||
|
|
||
| if(NOT Torch_FOUND) | ||
| message(STATUS "Torch not found. Downloading libtorch...") | ||
|
|
||
| # Set libtorch version and download URL | ||
| set(LIBTORCH_VERSION "2.9.1") |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing documentation for LibTorch version and CUDA version compatibility. Consider adding a comment at the top of the file explaining which LibTorch version is being downloaded, which CUDA versions are supported, and where to find information about updating these versions. This would help future maintainers understand the constraints and how to update the version when needed.
| set(LIBTORCH_VERSION "2.9.1") | ||
| set(USE_CUDA_LIBTORCH FALSE) #default is not to use the cuda version but cpu version | ||
| if(GAUXC_HAS_CUDA) | ||
| find_package(CUDAToolkit) |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Inconsistent indentation detected. This file uses tabs for indentation, but the rest of the codebase (e.g., gauxc-eigen3.cmake, gauxc-cutlass.cmake) uses 2 spaces. Please convert all tabs to 2 spaces for consistency with the project's coding conventions.
| set(USE_CUDA_LIBTORCH FALSE) #default is not to use the cuda version but cpu version | ||
| if(GAUXC_HAS_CUDA) | ||
| find_package(CUDAToolkit) | ||
| set(SUPPORTED_CUDA_VERSION_NO_DOTS "126" "128" "130") |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The CUDA version format should support CUDA 12.1, 12.4, and other minor versions within major version 12. The current supported list only includes "126" (12.6), "128" (12.8), and "130" (13.0), but CUDA 12.1 and 12.4 are common versions that would be rejected. Consider either expanding the supported version list or using a more flexible version matching approach (e.g., checking only the major version or using VERSION_GREATER_EQUAL comparisons).
| set(SUPPORTED_CUDA_VERSION_NO_DOTS "126" "128" "130") | |
| set(SUPPORTED_CUDA_VERSION_NO_DOTS "121" "124" "126" "128" "130") |
| # Download libtorch | ||
| if(NOT EXISTS ${LIBTORCH_DOWNLOAD_DIR}) | ||
| message(STATUS "Downloading libtorch from ${LIBTORCH_URL}") | ||
| file(DOWNLOAD ${LIBTORCH_URL} ${LIBTORCH_ZIP} | ||
| SHOW_PROGRESS | ||
| STATUS DOWNLOAD_STATUS) | ||
|
|
||
| list(GET DOWNLOAD_STATUS 0 STATUS_CODE) | ||
| if(NOT STATUS_CODE EQUAL 0) | ||
| message(FATAL_ERROR "Failed to download libtorch: ${DOWNLOAD_STATUS}") | ||
| endif() | ||
|
|
||
| # Extract libtorch | ||
| message(STATUS "Extracting libtorch...") | ||
| file(ARCHIVE_EXTRACT INPUT ${LIBTORCH_ZIP} DESTINATION ${CMAKE_BINARY_DIR}) | ||
|
|
||
| # Clean up zip file | ||
| file(REMOVE ${LIBTORCH_ZIP}) |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider using FetchContent_Declare with URL instead of manual file(DOWNLOAD) and file(ARCHIVE_EXTRACT). The codebase consistently uses FetchContent for dependency management (see gauxc-eigen3.cmake, gauxc-onedft.cmake). FetchContent provides better integration with CMake's build system, automatic caching, and cleaner error handling. This would also eliminate the need for manual cleanup of the zip file and provide better consistency with other dependency handling in the project.
| # Set CMAKE_PREFIX_PATH to find the downloaded libtorch | ||
| set(CMAKE_PREFIX_PATH "${LIBTORCH_DOWNLOAD_DIR};${CMAKE_PREFIX_PATH}") |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The CMAKE_PREFIX_PATH is modified but not scoped properly. This modification affects the entire project, which could have unintended side effects. Consider using set(CMAKE_PREFIX_PATH ... PARENT_SCOPE) if this is intended to propagate to the parent scope, or document why this global modification is necessary. Also, the modification happens outside the if(NOT EXISTS) block, so it will be set even when LibTorch was already extracted, which is correct but could be clearer.
| # Set CMAKE_PREFIX_PATH to find the downloaded libtorch | |
| set(CMAKE_PREFIX_PATH "${LIBTORCH_DOWNLOAD_DIR};${CMAKE_PREFIX_PATH}") | |
| # Extend CMAKE_PREFIX_PATH so that this directory (and any code including it) | |
| # can locate the downloaded LibTorch. This is intentionally done at directory | |
| # scope so that subsequent find_package(Torch) calls in the project can reuse | |
| # the same installation. It is placed outside the download/extract block so | |
| # that an already-extracted LibTorch is still picked up on re-configure. | |
| list(PREPEND CMAKE_PREFIX_PATH "${LIBTORCH_DOWNLOAD_DIR}") |
| find_package(CUDA REQUIRED) # Needed for Caffe cmake, since it use the old CUDA inclusion | ||
| set(USE_CUDA_LIBTORCH TRUE) | ||
| message(STATUS "CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}") | ||
| set(CUDA_INCLUDE_DIRS "${CUDAToolkit_INCLUDE_DIRS}") # and pass over the correct headers to Caffe cmake |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The find_package(CUDA) module is deprecated in modern CMake (deprecated since CMake 3.10) in favor of find_package(CUDAToolkit). The comment mentions "Caffe cmake" which seems out of place in a LibTorch download script. If this is truly needed for LibTorch's internal CMake, please clarify the comment. Otherwise, consider removing this deprecated call and relying solely on CUDAToolkit.
| find_package(CUDA REQUIRED) # Needed for Caffe cmake, since it use the old CUDA inclusion | |
| set(USE_CUDA_LIBTORCH TRUE) | |
| message(STATUS "CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}") | |
| set(CUDA_INCLUDE_DIRS "${CUDAToolkit_INCLUDE_DIRS}") # and pass over the correct headers to Caffe cmake | |
| set(USE_CUDA_LIBTORCH TRUE) | |
| message(STATUS "CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}") | |
| # Provide CUDA_INCLUDE_DIRS for any legacy CMake code that still expects this variable, | |
| # using the paths reported by CUDAToolkit instead of the deprecated CUDA module. | |
| set(CUDA_INCLUDE_DIRS "${CUDAToolkit_INCLUDE_DIRS}") |
| file(DOWNLOAD ${LIBTORCH_URL} ${LIBTORCH_ZIP} | ||
| SHOW_PROGRESS | ||
| STATUS DOWNLOAD_STATUS) | ||
|
|
||
| list(GET DOWNLOAD_STATUS 0 STATUS_CODE) | ||
| if(NOT STATUS_CODE EQUAL 0) | ||
| message(FATAL_ERROR "Failed to download libtorch: ${DOWNLOAD_STATUS}") | ||
| endif() | ||
|
|
||
| # Extract libtorch | ||
| message(STATUS "Extracting libtorch...") | ||
| file(ARCHIVE_EXTRACT INPUT ${LIBTORCH_ZIP} DESTINATION ${CMAKE_BINARY_DIR}) |
Copilot
AI
Jan 27, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This CMake logic downloads and extracts a third-party binary (libtorch) from download.pytorch.org during the build without any integrity verification, creating a supply-chain risk: if the download endpoint or TLS chain is compromised, an attacker could supply a malicious libtorch archive that is then extracted and linked into your build. Because this code runs automatically when Torch is not found, the build may silently consume and execute unverified remote code. Consider pinning the download to a verified artifact by enforcing an expected hash (for example via CMake's hash-checking options) or using a distribution mechanism that provides cryptographic signature verification before extraction.
Currently the user needs to install torch him-/herself to be able to use skala. libtorch can actually be downloaded from the pytorch site, so the PR adds this functionality to cmake.
Tested on: