Add Linux-guest HotAdd transport

README.md

@@ -13,21 +13,25 @@ Python naming).
 
 VIM login and inventory use [pyVmomi](https://github.com/vmware/pyvmomi).
 The NFC ticket, ESXi authd handshake, and disk I/O were reverse-engineered
-from VDDK 8 NBD traffic; see `docs/`.
+from VDDK 8 NBD traffic; see `docs/`. Linux HotAdd uses the public
+vSphere `ReconfigureVM` API (see `docs/hotadd.md`).
 
 ## Status
 
 Implemented against vCenter 8 / ESXi 8. Default transport is `nbdssl`
-(`nbd` is still available):
+(`nbd` is still available). Linux guests can also use `hotadd`:
 
 - `VixDiskLib_ConnectEx` (UID credentials)
 - `VixDiskLib_Open` (datastore path, read-only or read-write)
 - `VixDiskLib_Read` (optional ``skip_decompression`` packs FastLZ extras)
 - `VixDiskLib_Write`
+- HotAdd on a Linux VMware guest (SCSI, NVMe, or SATA source disks,
+  attached onto a proxy SCSI controller)
 
 Not implemented: compression open flags other than FastLZ, CBT /
 allocated-block queries, disk geometry (`DDB_GET`), encrypted disks,
-and direct ESXi `ha-nfc` without vCenter `vpxa-nfc`.
+direct ESXi `ha-nfc` without vCenter `vpxa-nfc`, SAN / file transports,
+Windows HotAdd, and HotAdd onto a proxy NVMe controller.
 
 Requires Python 3.10 or later.
 
@@ -73,6 +77,7 @@ VDDK-shaped handle.
 | `openvixdisklib/openvixdisklib.py` | Drop-in handle (`connect` / `open` / `read` / `write`) |
 | `openvixdisklib/nfc_auth.py`       | VIM login, NFC ticket, authd on 902                    |
 | `openvixdisklib/nfc_open.py`       | Classic NFC handshake, AIO open, sector read/write     |
+| `openvixdisklib/hotadd.py`         | Linux-guest SCSI HotAdd attach, local block I/O        |
 | `openvixdisklib/fastlz.py`         | FastLZ NFC adapter (pip `pyfastlz`)                    |
 | `tests/integration/`               | Live pytest suite against a lab vCenter                |
 | `tests/perf/`                      | Throughput comparison of OpenVixDiskLib vs VDDK        |
@@ -96,11 +101,16 @@ password: secret
 allow_untrusted: true
 datacenter: Datacenter
 datastore: datastore0
+hotadd_proxy:
+  host: hotadd-proxy.example.com
+  user: root
 ```
 
 A session-scoped pytest fixture creates an empty VM with a 10 GiB thin
 disk on that datastore and tears it down when the session ends. Tests
-write known patterns and read them back.
+write known patterns and read them back. HotAdd tests SSH into
+`hotadd_proxy` (a Linux guest on the same datastore) and skip if SSH
+fails.
 
 ```bash
 tox -e integration
@@ -117,7 +127,10 @@ tox -e integration -- --runslow
 Compare write/read throughput of OpenVixDiskLib and native VDDK
 (`64KiB`, 129-sector, and `32MiB` transfers; `nbdssl` and `nbd`;
 plain, FastLZ, and OpenVixDiskLib FastLZ ``skip_decompression``;
-AIO sessions 64 KiB×1, 1 MiB×1, 2 MiB×1, and 2 MiB×4).
+AIO sessions 64 KiB×1, 1 MiB×1, 2 MiB×1, and 2 MiB×4). The same sizes
+are also timed over Linux-guest ``hotadd`` (plain OpenVixDiskLib I/O
+on `hotadd_proxy`; FastLZ and NFC AIO do not apply) and skipped if
+SSH to the proxy fails.
 
 ```bash
 tox -e perf
@@ -145,5 +158,6 @@ Lint and typecheck: `tox -e pep8`, `tox -e mypy`.
 | `docs/nfc_open.md`                      | Classic NFC and AIO open         |
 | `docs/nfc_read.md`                      | AIO IO / `VixDiskLib_Read`       |
 | `docs/nfc_write.md`                     | AIO IO / `VixDiskLib_Write`      |
+| `docs/hotadd.md`                        | Linux-guest SCSI HotAdd          |
 | `docs/ssl_hook.md`                      | TLS intercept used for capture   |
 | `docs/reverse_engineering_procedure.md` | How the protocol was recovered   |

docs/hotadd.md

@@ -0,0 +1,81 @@
+# HotAdd transport
+
+OpenVixDiskLib can SCSI-HotAdd a VMDK onto the Linux guest that is
+running the library, then read and write it as a local block device.
+This is not an NFC protocol: it uses public VIM `ReconfigureVM` plus
+guest SCSI I/O. There is no VixTransport linked clone and no VMDK
+parser; ESXi presents a single SCSI LUN.
+
+NBD and NBDSSL remain the default. `transport_modes=None` is still
+`nbdssl`. `hotadd` is advertised and selected only when the process is
+a VMware guest (`/sys/class/dmi/id/sys_vendor`).
+
+## Mapping from VDDK
+
+| VDDK behaviour | OpenVixDiskLib |
+| -------------- | -------------- |
+| Run inside a proxy VM | Same. DMI UUID is matched to `config.uuid`. |
+| SCSI HotAdd of the source VMDK | `ReconfigureVM` add of an existing backing onto a **SCSI** controller on the proxy |
+| Linked clone via VixTransport | Not implemented. The snapshot or base VMDK is attached directly. |
+| Open as a whole-disk VMDK | Open `/dev/sdX` with `pread` / `pwrite` |
+| IDE disks | Not supported (same as VDDK) |
+| NVMe / SATA source disks | Supported. The backing file is attached onto proxy SCSI; the guest sees `/dev/sdX`, not `/dev/nvme*`. |
+| HotAdd onto a proxy NVMe controller | Not implemented |
+
+Colon lists such as `file:san:hotadd:nbdssl:nbd` pick the first **usable**
+mode. On a bare-metal host that is `nbdssl`. Inside a guest it is
+`hotadd`. `"hotadd"` alone on bare metal raises `NotImplementedError`.
+
+## Attach and detach
+
+1. Find this guest in vCenter (`SearchIndex.FindByUuid`).
+2. Resolve `disk_path` on the source VM. SCSI, NVMe
+   (`VirtualNVMEController`), and SATA (`VirtualAHCIController`) are
+   accepted. IDE and RDM are rejected. A powered-on source VM requires
+   `snapshot_ref`; a powered-off VM may attach the base disk.
+3. Add the existing VMDK to a free SCSI unit on the proxy (unit 7 is
+   skipped). If every unit is taken, a PVSCSI controller is added.
+   Read-only opens use `independent_nonpersistent` (redo log, source
+   stays clean). Writable opens use `persistent`.
+4. Rescan SCSI hosts and wait for the device. Matching prefers sysfs
+   `bus:0:unit:0`, then `*:0:unit:0` when `unit != 0`.
+5. `close` detaches with `Operation.remove` and **no** `fileOperation`.
+   The source VMDK must not be deleted. Leftover attachments of the
+   same backing are detached before a new open.
+
+Never HotAdd the proxy's own boot disk. Never use “newest `sdX`” as the
+only match when a unique SCSI address exists.
+
+Do not remove the source VM or its snapshot while the disk is still
+attached. Independent-nonpersistent attaches create a redo log on the
+source datastore; detach is what cleans it up.
+
+## API
+
+`VixDiskLibHandle.connect(..., transport_modes="hotadd")` then
+`open` / `read` / `write` / `close` as for NBD. Compression open flags
+and NFC `skip_decompression` do not apply; FastLZ flags on a HotAdd
+open raise `NotImplementedError`. `readinto` returns a `ReadResult`
+with empty `fragments`.
+
+Implementation: `openvixdisklib.hotadd`.
+
+## Lab
+
+Live tests SSH into a Linux proxy that shares the lab datastore and
+run `tests/integration/hotadd_remote.py` there. Configure
+`.test_config.yaml`:
+
+```yaml
+hotadd_proxy:
+  host: hotadd-proxy.example.com
+  user: root
+  # identity_file: /home/user/.ssh/id_ed25519
+```
+
+Tests skip when SSH is unavailable. The session lab VM (PVSCSI) and a
+function-scoped NVMe VM are HotAdded onto the proxy, written, and
+checked again over `nbdssl` from the runner. `tox -e perf` times the
+same transfer sizes over HotAdd (plain I/O; FastLZ and NFC AIO do not
+apply). Dependencies on the proxy are installed into
+`/tmp/openvixdisklib-hotadd/.venv`, not the system Python.

docs/reverse_engineering_procedure.md

@@ -9,9 +9,11 @@ NFC work can follow the same loop instead of rediscovering it.
 
 Scope so far: `VixDiskLib_ConnectEx` + `VixDiskLib_Open` +
 `VixDiskLib_Read` + `VixDiskLib_Write` against lab vCenter 8.0.1 /
-ESXi 8, transports `nbd` and `nbdssl`. Validation method:
-`tests/integration/` (the session-scoped `lab` fixture creates a temporary
-empty VM with a 10 GiB disk and destroys it when the pytest session ends).
+ESXi 8, transports `nbd`, `nbdssl`, and Linux-guest `hotadd`.
+Validation method: `tests/integration/` (the session-scoped `lab`
+fixture creates a temporary empty VM with a 10 GiB disk and destroys it
+when the pytest session ends). HotAdd live tests also SSH into a Linux
+proxy guest; see `docs/hotadd.md`.
 
 Rule from `AGENTS.md`: reuse pyVmomi for every public VIM operation.
 Only reimplement what pyVmomi does not expose.
@@ -409,3 +411,11 @@ Not yet reversed, same loop as above:
 - `VixDiskLib_GetInfo` capacity
 - Host-switch AIO messages
 - Direct ESXi `ha-nfc` without vCenter `vpxa-nfc`
+
+## HotAdd (not NFC)
+
+HotAdd does not use the capture loop above. VDDK SCSI-attaches the
+source VMDK to the proxy VM and opens a local whole disk. OpenVixDiskLib
+reuses pyVmomi `ReconfigureVM` for attach/detach and `pread`/`pwrite` on
+the Linux SCSI device. NVMe and SATA source disks are remapped onto a
+proxy SCSI controller. Details: `docs/hotadd.md`.