fix(sdk): watch handle callback awaiting, request timeouts, and exit semantics

[mishushakov]

Description

Fixes several WatchHandle issues across the SDKs and aligns the watch lifecycle contract between JS and Python:

• JS: async onEvent/onExit callbacks are now awaited (sequential event handling, callback errors routed to onExit).
• JS: stop() reports a clean exit instead of a misleading TimeoutError, and now resolves only after the watching has fully ended and onExit has completed — re-throwing errors raised by onExit. An in-flight onEvent callback is abandoned on stop (the JS equivalent of Python's cancelled handler task), which also prevents a deadlock when stop() is awaited from inside onEvent.
• Python (sync): get_new_events()/stop() now apply the default request timeout and the user auth headers (previously unbounded and sent as the default user) and accept a request_timeout parameter.
• Python (async): on_exit is now invoked whenever the watching ends — with the exception on stream failure, or None on normal stream end and on stop() — and stop() re-raises exceptions from on_exit instead of silently swallowing them.

The resulting contract is identical in both SDKs: the exit callback fires exactly once however the watching ends (None/undefined on normal end or user stop, the error otherwise), and stop() returns only after exit handling has completed.

Includes unit tests for all three handles, a secured-envd watchDir test for JS, and a fix for the async secured-envd test which wasn't actually using a secure sandbox.

Usage

// JS: async callbacks are now awaited; stop() reports a clean exit
const handle = await sandbox.files.watchDir('/home/user', async (event) => {
  await processEvent(event) // completes before the next event is handled
}, {
  onExit: (err) => {
    // err is undefined on normal stream end or stop(), the error otherwise
  },
})
await handle.stop() // resolves after watching fully ended and onExit completed

# Python (async): on_exit now fires on every end of watching
handle = await sandbox.files.watch_dir(
    "/home/user",
    on_event=lambda e: print(e),
    on_exit=lambda err: print("watch ended:", err),  # None on normal end/stop()
)

# Python (sync): polling calls now honor timeouts and user auth
handle = sandbox.files.watch_dir("/home/user", user="root")
events = handle.get_new_events(request_timeout=10)  # runs as "root", times out
handle.stop()

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 0db7048

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
e2b	Patch
@e2b/python-sdk	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cursor]

PR Summary

Medium Risk
Watch lifecycle and callback semantics changed in both SDKs (behavioral contract for onExit/stop and sequential async handlers); sync Python watch RPCs now send user auth headers, which fixes wrong-user polling but changes prior behavior.

Overview
Aligns filesystem directory watch behavior across the JS and Python SDKs and fixes several edge cases in how watches start and shut down.

JS WatchHandle now awaits async onEvent / onExit, processes events sequentially, and routes onEvent failures (sync or async) into onExit. stop() waits until teardown and onExit finish, treats user stop as a clean exit (no spurious timeout), abandons in-flight onEvent on stop, supports re-entrant stop() from callbacks without deadlock, and re-throws onExit errors. Docs for onExit clarify the error vs clean-exit contract.

Python async AsyncWatchHandle matches that contract: on_exit runs on every end (error, normal end, or stop() with None), async on_event is awaited in order, stop() uses asyncio.shield, propagates on_exit errors, and distinguishes caller cancellation from watch cancellation.

Python sync WatchHandle polling RPCs get_new_events / stop now pass auth headers and config request timeouts, with optional request_timeout overrides.

handleProcessStartEvent / handleWatchDirStartEvent throw Expected start event when the stream is empty or wrong-shaped instead of a TypeError on undefined.

Adds unit tests for start-event helpers and watch handles, secured-envd watch coverage (JS + fixed async Python test), and changesets for patch releases.

^{Reviewed by Cursor Bugbot for commit 0db7048. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Package Artifacts

Built from cc0f4fc. Download artifacts from this workflow run.

JS SDK (e2b@2.30.6-mishushakov-fix-watchhandle-callbacks-timeout.0):

npm install ./e2b-2.30.6-mishushakov-fix-watchhandle-callbacks-timeout.0.tgz

CLI (@e2b/cli@2.12.3-mishushakov-fix-watchhandle-callbacks-timeout.0):

npm install ./e2b-cli-2.12.3-mishushakov-fix-watchhandle-callbacks-timeout.0.tgz

Python SDK (e2b==2.29.5+mishushakov-fix-watchhandle-callbacks-timeout):

pip install ./e2b-2.29.5+mishushakov.fix.watchhandle.callbacks.timeout-py3-none-any.whl

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bfb9c5bd8f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4f48d20. Configure here.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0977f9817

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[claude]

🔴 stop() returns early whenever inOnExit is true, even when the call is not re-entrant — so an external await handle.stop() that races with an in-flight async onExit skips await this.handlingEvents, and any error onExit subsequently raises is silently swallowed by the constructor's this.handlingEvents.catch(() => {}). This breaks the documented contract added in this PR (and stated in .changeset/watch-handle-stop-clean-exit.md): "stop() … re-throwing errors raised by onExit". The inOnExit boolean cannot distinguish a true re-entrant call (same async task, where awaiting handlingEvents would deadlock) from an external call from a separate async context (where it would not); a per-call-context token via AsyncLocalStorage would fix this without re-introducing the deadlock.

Extended reasoning...

What is going wrong

The PR documents this contract on stop() (watchHandle.ts:107–110, mirrored verbatim in .changeset/watch-handle-stop-clean-exit.md):

Resolves after the watching has fully ended and the onExit callback (if any) has completed; errors thrown by onExit are re-thrown here.

The inOnExit early-return at watchHandle.ts:124-126 was added to prevent the legitimate re-entrant deadlock when stop() is awaited from inside onExit. But inOnExit is a single instance flag — it is true any time handleEvents is parked on await this.onExit?.(error), regardless of which async task is calling stop(). An external caller racing with an in-flight async onExit therefore takes the same early-return branch as a re-entrant caller, skips await this.handlingEvents, and never observes errors raised by onExit. The constructor's unconditional this.handlingEvents.catch(() => {}) on line 102 then silently absorbs those errors so they don't crash the process either.

Step-by-step proof

Consider this realistic usage — the watch ends on its own and onExit does awaited cleanup that may rethrow the exit error:

const handle = await sandbox.files.watchDir(path, () => {}, {
  timeoutMs: 100,
  onExit: async (err) => {
    await someSlowCleanup()   // takes ~300ms
    if (err) throw err         // expected to be re-thrown by stop()
  },
})

await new Promise(r => setTimeout(r, 200))
await handle.stop()           // expected: waits, re-throws the TimeoutError
                              // actual:   returns immediately, error is lost

Trace through watchHandle.ts:

1. t ≈ 100ms: the watch times out. iterateEvents() rejects with the stream-abort error. In handleEvents, the catch at line 197-203 fires. this.stopped is still false, so error = streamError is captured.
2. t ≈ 100ms: line 205 sets this.inOnExit = true. Line 207 awaits this.onExit?.(error). Control yields — someSlowCleanup() is in flight.
3. t ≈ 200ms: the user code calls await handle.stop(). In stop() (line 116-128): sets this.stopped = true, runs notifyStopped(), runs handleStop(), then evaluates if (this.inOnExit) — TRUE, so it returns.
4. The user's await handle.stop() resolves at t ≈ 200ms, with no error.
5. t ≈ 400ms: someSlowCleanup() completes. onExit re-throws the TimeoutError. The await this.onExit?.(error) at line 207 rejects, propagates out of handleEvents, and rejects handlingEvents.
6. The constructor's this.handlingEvents.catch(() => {}) (line 102) marks the rejection as handled. The error is gone.

The caller never sees the TimeoutError that onExit explicitly rethrew, even though it did await handle.stop(). Worst-class failure mode: no log, no propagation, contract silently violated.

Why existing safeguards don't catch this

• The constructor's handlingEvents.catch(() => {}) is correct for the "no caller is waiting" case (watch ends on its own, nobody calls stop()), but the same chain swallows the rejection when the new inOnExit shortcut detaches the external caller from handlingEvents.
• The new test stop can be awaited from inside onExit without deadlocking (watchHandle.test.ts:165-205) covers the legitimate re-entrant case with a synchronous onExit.
• The test stop rethrows onExit error when called after the watch already ended (watchHandle.test.ts:208-229) deliberately inserts await new Promise(r => setTimeout(r, 0)) after onExit resolves, so by the time stop() runs, the finally block at line 208-210 has already cleared inOnExit back to false. The intermediate window — onExit mid-flight, external stop() called — is not exercised.

Trigger window

Any watch that ends without an explicit stop() (timeout, stream failure, normal stream end) where the user's onExit does awaited work. Trying to clean up in onExit and rethrowing the original exit error is exactly the pattern this PR encourages by documenting that errors propagate from stop(), so the failure mode hits the most realistic usage.

How to fix it

Track re-entrance with AsyncLocalStorage (or a per-invocation token set just around the await this.onExit?.(error) call) instead of a shared boolean. Only skip await this.handlingEvents when the current async context is the one running onExit. External callers from a separate async task then still await handlingEvents and receive the documented error.