sea: support embedding assets

[joyeecheung]

src: print string content better in BlobDeserializer

When it's a short string, print it inline, otherwise print it
from a separate line. Also add the missing line breaks finally.

sea: support embedding assets

With this patch:

Users can now include assets by adding a key-path dictionary
to the configuration as the assets field. At build time, Node.js
would read the assets from the specified paths and bundle them into
the preparation blob. In the generated executable, users can retrieve
the assets using the sea.getAsset() and sea.getAssetAsBlob() API.

{
  "main": "/path/to/bundled/script.js",
  "output": "/path/to/write/the/generated/blob.blob",
  "assets": {
    "a.jpg": "/path/to/a.jpg",
    "b.txt": "/path/to/b.txt"
  }
}

The single-executable application can access the assets as follows:

const { getAsset } = require('node:sea');
// Returns a copy of the data in an ArrayBuffer
const image = getAsset('a.jpg');
// Returns a string decoded from the asset as UTF8.
const text = getAsset('b.txt', 'utf8');
// Returns a Blob containing the asset without copying.
const blob = getAssetAsBlob('a.jpg');

Drive-by: update the documentation to include a section dedicated
to the injected main script and refer to it as "injected main
script" instead of "injected module" because it's a script, not
a module.

Refs: nodejs/single-executable#68

[nodejs-github-bot]

Review requested:

• @nodejs/loaders
• @nodejs/single-executable
• @nodejs/startup

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/55991/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/55994/

[targos]

Should we have a getAssetNames() API or something else that allows the user to list the bundled assets?

[joyeecheung]

From #50941 (comment)

What is wrong with saying in docs here is an ArrayBuffer, but if you mess with it you’ll crash? Then as a developer I could just adhere to that and live happily ever after.

One thing we need to decide: should we expose getRawAsset() to allow users reading the asset directly without copying. With a Blob users still have to do some form of copying at the end of the day unless their use case is piping the whole thing. Currently if you get the raw buffer and then wrap it with an ArrayBufferView and then mutate any of the elements, the process would crash due to access protection (probably because the segments postject uses are protected, or I'm not sure if there are unprotected segments we can do this injection with)

[pipobscure]

Just a note: I really appreciate this happening. I have previously been using nexe and pkg and therefore know a lot of the pain points of trying to keep current with node. So I just want to give @joyeecheung huge thanks and kudos for moving this forward! THANKS

[aduh95]

A few nits

[anonrig]

@nodejs/single-executable I recommend creating a JSON schema and distributing it through nodejs.org, to have better intellisense when this json includes "$schema" parameter. This could help with maintenance and usability.

[joyeecheung]

Do you want to open an issue in https://github.com/nodejs/single-executable/issues instead?

[targos]

Just repeating #50960 (comment) in case you missed it:

Should we have a getAssetNames() API or something else that allows the user to list the bundled assets?

[CMCDragonkai]

Can this be used for native so binaries and then pass them somehow into process.dlopen?

[pipobscure]

Yes that's how. Except you'll need to extract onto disk first because dlopen requires a file.

…

On Sat, 18 May 2024, 04:26 Roger Qiu, ***@***.***> wrote: Can this be used for native so binaries and then pass them somehow into process.dlopen? — Reply to this email directly, view it on GitHub <#50960 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADM5LZKZPT3S46CQAQHFOTZC3C7BAVCNFSM6AAAAAA76SV46KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYGYZDAMBXGY> . You are receiving this because you commented.Message ID: ***@***.***>

[CMCDragonkai]

What about the VFS idea?

[pipobscure]

That will make it easy for most files, but native addons will still need to be put on disk, because the dlopen os call simply requires it. VFS would work if it's on yhe OS level (think kernel vfs). But that's outside the scope of what node/sea could do.

…

On Sat, 18 May 2024, 11:27 Roger Qiu, ***@***.***> wrote: What about the VFS idea? — Reply to this email directly, view it on GitHub <#50960 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADM5L6ZM2F7LABRUKRC65DZC4UJ5AVCNFSM6AAAAAA76SV46KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYG43DMMBRG4> . You are receiving this because you commented.Message ID: ***@***.***>

[joyeecheung]

VFS would need to built on top of proper fs hooks which is related to SEA, but also there are many other use cases for such hooks, one previous proposal is nodejs/single-executable#43 (I think there are others, but can't find the links).