Skip to content

[NativeEngine] Add updateTextureDirectly texture-loader sink (#218)#1778

Draft
bkaradzic-microsoft wants to merge 4 commits into
BabylonJS:masterfrom
bkaradzic-microsoft:native-update-texture-directly
Draft

[NativeEngine] Add updateTextureDirectly texture-loader sink (#218)#1778
bkaradzic-microsoft wants to merge 4 commits into
BabylonJS:masterfrom
bkaradzic-microsoft:native-update-texture-directly

Conversation

@bkaradzic-microsoft

@bkaradzic-microsoft bkaradzic-microsoft commented Jul 9, 2026

Copy link
Copy Markdown
Member

What

Adds a single new N-API method, updateTextureDirectly, to NativeEngine — the native half of #218 (single-file .dds/.ktx/.ktx2 cubemap loading).

It implements the sink for Babylon's shared JS texture-loader path (_uploadDataToTextureDirectly / _uploadCompressedDataToTextureDirectly), so NativeEngine can reuse the exact same stock loaders that WebGL/WebGPU use (DDS/KTX/KTX2/Basis/IES/HDR/EXR/TGA), rather than needing a bespoke native path per container format.

The loaders upload one (face, mip) at a time, WebGL texImage2D-style. bgfx instead needs the whole texture allocated before any update, so the texture is created lazily on the first upload and subsequent (face, mip) calls update into it.

Why draft / paired change

This is a dormant sink: nothing in the current published babylonjs calls updateTextureDirectly, so this PR is a no-op at runtime until the paired Babylon.js JS change ships. It compiles cleanly and is safe to merge independently.

  • Paired JS half (the loader wiring + diffuse-IBL spherical-harmonics computation): draft at [Native] Single-file DDS/KTX/KTX2 cubemap loading via JS texture loaders bkaradzic-microsoft/Babylon.js#1, to be re-targeted at BabylonJS/Babylon.js when ready.
  • A follow-up BabylonNative PR will re-enable the 7 PBR/IBL validation tests in Apps/Playground/Scripts/config.json that this feature fixes — held until the JS half is published to npm (Apps/package.json pins babylonjs: "^9.3.4", which will auto-resolve), so the test re-enable doesn't turn CI red before the runtime dependency exists.

Implementation notes

All inputs to this binding come from JS, so every scalar is validated before it is narrowed or used to index a table:

  • Build guard: the body is wrapped in BABYLON_NATIVE_PLUGIN_NATIVEENGINE_LOAD_IMAGES (matching LoadRawTexture/LoadCubeTexture) and throws when image loading is disabled, so the plugin still compiles with LOAD_IMAGES=OFF (the bimg headers are gated on that flag).
  • Dimensions: base/mip dimensions are checked against bgfx::getCaps()->limits.maxTextureSize (and non-zero) before narrowing to uint16_t, so an out-of-range value can't wrap into an in-range one and drive an OOB read/upload.
  • Face index & lod: read as uint32 and validated before narrowing to uint8_t — cube face < 6, 2D face == 0, and lod within the mip chain implied by the base dimensions (so hasMips == false restricts lod to 0).
  • Cube shape & mip consistency: cube uploads must be square, and mipWidth/mipHeight must equal max(1, base >> lod), so an inconsistent update region can't reach bgfx::updateTexture* and assert / hit backend UB.
  • Format: read as uint32 and rejected when >= bimg::TextureFormat::Count before the enum cast, so an out-of-range value can't index past bimg/bgfx's format tables in Create*/imageGetSize.
  • Size check + overflow guard: data.ByteLength() is validated against bimg::imageGetSize(...) for the given mip dimensions and format, and the size is rejected if it would exceed bgfx::copy's 32-bit length.
  • Async-safe upload: uses bgfx::copy so bgfx owns the buffer and releases it after the GPU consumes it.
  • Flip conventions match the existing loaders: cube faces flip only on origin-bottom-left (OpenGL), like LoadCubeTextureFromImages; 2D follows the raw/loadTexture convention. Compressed block data is never row-flipped.

Testing

Validated locally against the paired JS build via the Playground headless validation harness — the 7 PBR/IBL tests that previously exceeded the pixel-diff budget (because prefiltered-cube SH wasn't loading) now pass well within budget (all << 240,000-px budget):

Test Pixel diff
NME glTF 461
Anisotropic 311
Clear Coat 473
PBRMetallicRoughnessMaterial 308
PBRSpecularGlossinessMaterial 363
PBR 266
PBR in mirror 676

The config.json re-enable of these tests is held for the follow-up PR (see above).

Review

Addressed multiple rounds of Copilot review: the LOAD_IMAGES build guard, the pre-narrow validation of face index / lod / format, the cube-square and mip-dimension-consistency checks, and the bgfx::copy overflow guard were all added in response. One suggestion (re-deriving and comparing an already-valid texture's dimensions/format on every update) was intentionally declined as redundant — the loader is the sole caller and always drives consistent parameters.

Implement the native sink for Babylon's _uploadDataToTextureDirectly /
_uploadCompressedDataToTextureDirectly so single-file container textures
(.dds/.ktx/.ktx2, plus Basis/IES/HDR/EXR/TGA) load through the same JS
texture loaders WebGL/WebGPU use. The loaders upload one (face, mip) at a
time WebGL texImage2D-style; bgfx needs the whole texture allocated first,
so the underlying texture is created lazily on the first upload.

Validates JS-provided dimensions against maxTextureSize before uint16
narrowing and the payload size against bimg::imageGetSize, uses bgfx::copy
for async-owned upload memory, and matches the existing loader flip
conventions (skipping row-flips for compressed formats).

Addresses BabylonJS#218 (paired with the Babylon.js single-file cubemap loader change).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new N-API entry point on NativeEngine to support Babylon’s shared JS “upload-to-texture directly” sink, enabling reuse of the stock JS texture loaders (DDS/KTX/KTX2/etc.) while handling bgfx’s requirement to allocate the full texture before sub-updates.

Changes:

  • Declares NativeEngine::UpdateTextureDirectly in the engine header.
  • Registers the new JS-exposed instance method updateTextureDirectly.
  • Implements lazy texture creation + per-(face,mip) uploads with dimension and payload-size validation and optional Y-flip behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
Plugins/NativeEngine/Source/NativeEngine.h Adds the UpdateTextureDirectly method declaration.
Plugins/NativeEngine/Source/NativeEngine.cpp Exposes and implements updateTextureDirectly, performing validation, lazy create, and bgfx uploads.

Comment thread Plugins/NativeEngine/Source/NativeEngine.cpp
Comment thread Plugins/NativeEngine/Source/NativeEngine.cpp Outdated
Comment thread Plugins/NativeEngine/Source/NativeEngine.cpp
Comment thread Plugins/NativeEngine/Source/NativeEngine.cpp
- Guard the body with BABYLON_NATIVE_PLUGIN_NATIVEENGINE_LOAD_IMAGES
  (matches LoadRawTexture/LoadCubeTexture); throw when image loading is off
  so the plugin still compiles with LOAD_IMAGES=OFF (bimg headers are gated).
- Read face index and lod as uint32 and validate BEFORE narrowing to uint8_t.
- Reject non-square cube dimensions and out-of-range face indices
  (cube face < 6, 2D face == 0).
- Reject lod values that exceed the mip chain implied by the base dimensions.
- Reject uploads whose size would truncate bgfx::copy's 32-bit length.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bkaradzic-microsoft

Copy link
Copy Markdown
Member Author

Addressed the Copilot review in c823821 (UpdateTextureDirectly):

  • LOAD_IMAGES guard — body wrapped in #ifndef BABYLON_NATIVE_PLUGIN_NATIVEENGINE_LOAD_IMAGES (throws when off) so the plugin compiles with image loading disabled.
  • faceIndex/lod validation — read as uint32 and validated before narrowing (cube face < 6, 2D face == 0, lod within the mip chain).
  • Cube square check — non-square base/mip dimensions now throw.
  • Overflow guard — reject uploads whose size would truncate bgfx::copy's 32-bit length.

I skipped the suggested already-valid texture matches params re-validation as redundant (the loader is the sole driver and always passes consistent params); happy to add it if preferred.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread Plugins/NativeEngine/Source/NativeEngine.cpp
Reject UpdateTextureDirectly calls whose mipWidth/mipHeight don't match
max(1, base >> lod), so an inconsistent update region can't drive a bgfx
assertion or backend-dependent undefined behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread Plugins/NativeEngine/Source/NativeEngine.cpp
Read the JS-provided format as uint32 and reject values >= bimg::TextureFormat::Count
before narrowing to the enum, so an out-of-range value can't index past bimg/bgfx's
format tables in Create*/imageGetSize. Mirrors the existing pre-narrow validation of
faceIndex/lod/dimensions in this method.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants