feat(data-inspector): inline sources, lazy deep-node expansion, auto-rerun, playground toolbar (#111)

Co-authored-by: Anthony Fu <github@antfu.me>

Author: antfubot

docs/plugins/data-inspector.md

@@ -10,8 +10,10 @@ Package: `@devframes/plugin-data-inspector` · framework: **Vue + Vite**
 
 ## What it does
 
-- **Query workbench** — a CodeMirror jora editor with syntax highlighting and server-computed autocomplete; queries auto-run as you type, with a client-side syntax gate so malformed input never hits the wire. Source, query, and filters persist in the URL, so any workbench state is shareable.
+- **Query workbench** — a CodeMirror jora editor with syntax highlighting and server-computed autocomplete; queries auto-run as you type, with a client-side syntax gate so malformed input never hits the wire. A toolbar copies the query, and the editor pairs with expand-all / collapse-all and copy-as-JSON controls over the results. Source, query, filters, and the auto-rerun setting persist in the URL, so any workbench state is shareable.
+- **Auto rerun** — an optional poller under the filters (`auto rerun every N seconds`) re-runs the current query against the live object on a fixed period, so a value that changes over time updates on its own. Ticks are skipped while a run is in flight or the query is syntactically broken.
 - **Result viewer** — results normalize to strict JSON (circulars become `$ref` markers; Maps, Sets, class instances, functions, and Dates get type badges) with per-query stats: jora / normalize / rpc timings, payload size, node count. The value-actions popup copies paths and turns any key into a query.
+- **Lazy expansion** — deep graphs return one level at a time: a node past the depth cap renders a `load deeper` link that fetches just that subtree with a fresh budget and splices it in place, so a huge object stays responsive and loads on demand.
 - **Data shape panel** — a one-level type skeleton of the active source, independent of the query; click a property to query it.
 - **Filters** — exclude functions, `_`-prefixed, or `$`-prefixed properties from results and skeleton alike.
 - **Saved queries** — recipes (`query` + optional title/description + the filters they were authored with), id-keyed, in two scopes: **workspace** (committable, shared with the team) and **project** (per-checkout).
@@ -114,15 +116,29 @@ The target process opts in by starting the agent:
 ```ts
 import { exposeDataInspector } from '@devframes/plugin-data-inspector/inject'
 
-await exposeDataInspector()
+await exposeDataInspector({
+  sources: [{ id: 'app:store', title: 'App store', data: () => store }],
+})
 ```
 
-or with zero code changes:
+`sources` registers the given entries before the endpoint opens — a convenience over calling `registerDataSource` yourself; both paths share the one process-global registry. Call `exposeDataInspector()` with no sources to expose whatever is already registered.
+
+Or with zero code changes:
 
 ```sh
 DEVFRAME_DATA_INSPECTOR=1 node --import @devframes/plugin-data-inspector/inject server.js
 ```
 
+On this zero-code path there is nowhere to call `registerDataSource`, so the agent auto-registers a **`globalThis`** source. Assign anything you want to inspect onto the global object and query it live:
+
+```ts
+// somewhere in the running process
+globalThis.store = store
+globalThis.cache = cache
+```
+
+Then query `store`, `cache`, or `keys($)` to see what's there. The source reads `globalThis` at query time, so assignments made after the agent started show up on the next run. Opt out with `DEVFRAME_DATA_INSPECTOR_GLOBAL=0`.
+
 The agent binds `127.0.0.1`, requires devframe's trust handshake with a per-run pre-shared token, and advertises its endpoint in `node_modules/.data-inspector/agent.json` — `devframe-data-inspector attach` consumes it automatically (or pass `ws://…` and `--token` explicitly). Queries execute inside the target process, where the live objects are. Treat the endpoint like a debugger port.
 
 ## RPC surface
@@ -133,6 +149,7 @@ All functions are namespaced `devframes:plugin:data-inspector:*`:
 |----------|------|---------|
 | `sources` | `query` | Every registered source (meta and suggested queries). |
 | `query` | `query` | Runs a jora query against a source; normalized result with stats. |
+| `queryPath` | `query` | Re-runs a query and returns a fresh, depth-limited slice of the subtree at a node path (lazy expansion). |
 | `skeleton` | `query` | The type skeleton of a source, honoring the filter options. |
 | `suggest` | `query` | Autocomplete candidates from jora's stat mode at a cursor position. |
 | `saved:list` / `saved:save` / `saved:delete` | `query` / `action` | Saved-query recipes in the `workspace` and `project` scopes. |

package.json

@@ -3,7 +3,7 @@
   "type": "module",
   "version": "0.7.4",
   "private": true,
-  "packageManager": "pnpm@11.13.0",
+  "packageManager": "pnpm@11.13.1",
   "author": "Anthony Fu <anthonyfu117@hotmail.com>",
   "license": "MIT",
   "funding": "https://github.com/sponsors/antfu",

plugins/data-inspector/README.md

@@ -1,6 +1,6 @@
 # @devframes/plugin-data-inspector
 
-Inspect live server-side objects interactively. Other plugins and hosts register **data sources**; the workbench composes [jora](https://github.com/discoveryjs/jora) queries against them — executed in the process that owns the objects — and renders normalized results in a [discovery.js](https://github.com/discoveryjs/discovery) struct view with type badges, a shape panel, saved queries, and shareable URL state.
+Inspect live server-side objects interactively. Other plugins and hosts register **data sources**; the workbench composes [jora](https://github.com/discoveryjs/jora) queries against them — executed in the process that owns the objects — and renders normalized results in a [discovery.js](https://github.com/discoveryjs/discovery) struct view with type badges, a shape panel, saved queries, and shareable URL state. Deep graphs expand a level at a time (`load deeper` fetches each subtree on demand), an optional poller re-runs the query every N seconds, and a toolbar offers expand/collapse-all and copy.
 
 ## Register a data source
 
@@ -50,7 +50,10 @@ Static exports embed the dataset and run the same query engine client-side, so s
 ```ts
 import { exposeDataInspector } from '@devframes/plugin-data-inspector/inject'
 
-await exposeDataInspector()
+// pass sources inline, or register them separately beforehand
+await exposeDataInspector({
+  sources: [{ id: 'app:store', title: 'App store', data: () => store }],
+})
 ```
 
 or with zero code changes:
@@ -59,6 +62,8 @@ or with zero code changes:
 DEVFRAME_DATA_INSPECTOR=1 node --import @devframes/plugin-data-inspector/inject server.js
 ```
 
+On the zero-code path there's nowhere to call `registerDataSource`, so the agent auto-registers a **`globalThis`** source: assign what you want to inspect onto the global object (`globalThis.store = store`) and query it live. Opt out with `DEVFRAME_DATA_INSPECTOR_GLOBAL=0`.
+
 The agent binds `127.0.0.1`, requires devframe's trust handshake with a per-run token by default, and advertises its endpoint in `node_modules/.data-inspector/agent.json`, which `devframe-data-inspector attach` picks up automatically.
 
 > [!WARNING]

plugins/data-inspector/src/engine/contract.ts

@@ -10,6 +10,27 @@ export interface FilterOptions {
   excludeDollarProps?: boolean
 }
 
+/**
+ * One structural step from a query result toward a nested node, recorded by
+ * the normalizer on each depth-truncated marker so the client can lazily
+ * re-fetch that subtree with a fresh depth budget (see `queryPath`). Each
+ * segment mirrors exactly one descent the normalizer makes:
+ *
+ *   - `['k', key]`   own object property, or a string-keyed Map value
+ *   - `['i', index]` array item (index into the filtered array)
+ *   - `['s', index]` Set value
+ *   - `['mk', index]` / `['mv', index]` non-string Map entry key / value
+ */
+export type PathSegment
+  = | ['k', string]
+    | ['i', number]
+    | ['s', number]
+    | ['mk', number]
+    | ['mv', number]
+
+/** A path from the query root to a nested node: a list of structural steps. */
+export type NodePath = PathSegment[]
+
 /** A query recipe: the text plus the filter options it was authored with. */
 export interface Query extends FilterOptions {
   query: string

plugins/data-inspector/src/engine/normalize.ts

@@ -12,8 +12,14 @@
  *   - BigInt / Symbol      -> tagged string forms
  *   - Error                -> { $type: 'Error', name, message }
  *   - Promise / WeakMap/.. -> opaque tags
- *   - depth / entry caps   -> { $truncated: ... } markers + stats
+ *   - depth cap            -> { $truncated: 'depth', $preview, $path } markers
+ *   - entry cap            -> { $truncated: 'entries', ... } markers + stats
+ *
+ * Depth-truncated markers carry a `$path` (a `NodePath` of structural steps
+ * from the query root) so the client can lazily re-fetch that subtree with a
+ * fresh depth budget through `navigate` + `runQueryAtPath`.
  */
+import type { NodePath, PathSegment } from './contract'
 
 export interface NormalizeOptions {
   /** Max object/array nesting depth before truncation. */
@@ -79,12 +85,47 @@ export function normalize(value: unknown, options: NormalizeOptions = {}): { dat
       excludeDollarProps: options.excludeDollarProps ?? false,
     },
   }
-  const data = walk(value, walker, 0, '#')
+  const data = walk(value, walker, 0, '#', [])
   walker.stats.ms = Math.round((performance.now() - start) * 100) / 100
   return { data, stats: walker.stats }
 }
 
-function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
+/**
+ * Re-descend a live graph along a `NodePath`, mirroring the walker's own
+ * traversal (and re-applying `excludeFunctions` to array indices so a lazily
+ * fetched index lines up with what the client saw). Returns the node the
+ * path addresses, or `undefined` if any step falls off the graph.
+ */
+export function navigate(value: unknown, path: NodePath, options: Pick<NormalizeOptions, 'excludeFunctions'> = {}): unknown {
+  let cur: unknown = value
+  for (const [kind, at] of path) {
+    if (cur === null || typeof cur !== 'object')
+      return undefined
+    switch (kind) {
+      case 'k':
+        cur = cur instanceof Map ? cur.get(at) : (cur as Record<string, unknown>)[at]
+        break
+      case 'i': {
+        const arr = cur as unknown[]
+        const source = options.excludeFunctions ? arr.filter(item => typeof item !== 'function') : arr
+        cur = source[at]
+        break
+      }
+      case 's':
+        cur = [...(cur as Set<unknown>)][at]
+        break
+      case 'mk':
+        cur = [...(cur as Map<unknown, unknown>).entries()][at]?.[0]
+        break
+      case 'mv':
+        cur = [...(cur as Map<unknown, unknown>).entries()][at]?.[1]
+        break
+    }
+  }
+  return cur
+}
+
+function walk(value: unknown, w: Walker, depth: number, path: string, segs: NodePath): unknown {
   w.stats.nodes++
 
   // ── primitives ──────────────────────────────────────────────────────
@@ -138,7 +179,7 @@ function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
 
   if (depth >= w.opts.maxDepth) {
     w.stats.truncatedDepth++
-    return { $truncated: 'depth', $preview: preview(obj) }
+    return { $truncated: 'depth', $preview: preview(obj), $path: segs }
   }
 
   w.seen.set(obj, path)
@@ -148,7 +189,7 @@ function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
     const cap = Math.min(source.length, w.opts.maxEntries)
     const out: unknown[] = Array.from({ length: cap })
     for (let i = 0; i < cap; i++)
-      out[i] = walk(source[i], w, depth + 1, `${path}[${i}]`)
+      out[i] = walk(source[i], w, depth + 1, `${path}[${i}]`, seg(segs, ['i', i]))
     if (source.length > cap) {
       w.stats.truncatedEntries++
       out.push({ $truncated: 'entries', $total: source.length, $shown: cap })
@@ -169,15 +210,15 @@ function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
     if (allStringKeys) {
       const value: Record<string, unknown> = {}
       for (const [k, v] of entries)
-        value[k as string] = walk(v, w, depth + 1, `${path}.${String(k)}`)
+        value[k as string] = walk(v, w, depth + 1, `${path}.${String(k)}`, seg(segs, ['k', k as string]))
       return { $type: 'Map', size: obj.size, value }
     }
     return {
       $type: 'Map',
       size: obj.size,
       entries: entries.map(([k, v], i) => ({
-        key: walk(k, w, depth + 1, `${path}~keys[${i}]`),
-        value: walk(v, w, depth + 1, `${path}~values[${i}]`),
+        key: walk(k, w, depth + 1, `${path}~keys[${i}]`, seg(segs, ['mk', i])),
+        value: walk(v, w, depth + 1, `${path}~values[${i}]`, seg(segs, ['mv', i])),
       })),
     }
   }
@@ -186,7 +227,7 @@ function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
     const values = [...obj].slice(0, w.opts.maxEntries)
     if (obj.size > values.length)
       w.stats.truncatedEntries++
-    return { $type: 'Set', size: obj.size, values: values.map((v, i) => walk(v, w, depth + 1, `${path}~set[${i}]`)) }
+    return { $type: 'Set', size: obj.size, values: values.map((v, i) => walk(v, w, depth + 1, `${path}~set[${i}]`, seg(segs, ['s', i]))) }
   }
 
   // Plain object or class instance: own enumerable string-keyed props.
@@ -213,7 +254,7 @@ function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
     }
     if (w.opts.excludeFunctions && typeof v === 'function')
       continue
-    out[key] = walk(v, w, depth + 1, `${path}.${key}`)
+    out[key] = walk(v, w, depth + 1, `${path}.${key}`, seg(segs, ['k', key]))
   }
   if (keys.length > cap) {
     w.stats.truncatedProps++
@@ -222,6 +263,11 @@ function walk(value: unknown, w: Walker, depth: number, path: string): unknown {
   return out
 }
 
+/** Append one structural step to a path, returning a fresh array. */
+function seg(path: NodePath, step: PathSegment): NodePath {
+  return [...path, step]
+}
+
 function preview(obj: object): string {
   if (Array.isArray(obj))
     return `Array(${obj.length})`

plugins/data-inspector/src/engine/query-engine.ts

@@ -10,10 +10,10 @@
  *   - suggestions come from jora's stat mode, flattened into plain
  *     RPC-safe completion items.
  */
-import type { QueryOutcome, SuggestItem, SuggestOutcome } from './contract'
+import type { NodePath, QueryOutcome, SuggestItem, SuggestOutcome } from './contract'
 import type { NormalizeOptions } from './normalize'
 import jora from 'jora'
-import { normalize } from './normalize'
+import { navigate, normalize } from './normalize'
 
 export type { SuggestItem, SuggestOutcome } from './contract'
 
@@ -103,6 +103,29 @@ export function runQuery(target: unknown, query: string, options?: NormalizeOpti
   }
 }
 
+/**
+ * Lazy-expand a depth-truncated node: re-run the base query against the live
+ * object, re-descend to the node the `NodePath` addresses, and normalize just
+ * that subtree with a fresh depth budget. The path comes from a `$truncated:
+ * 'depth'` marker the client is expanding, so the same filter options must be
+ * threaded through (they shift array indices and drop keys).
+ */
+export function runQueryAtPath(target: unknown, query: string, path: NodePath, options?: NormalizeOptions): QueryOutcome {
+  try {
+    const started = performance.now()
+    const raw = createQuery(query)(target)
+    const node = navigate(raw, path, options)
+    const queryMs = Math.round((performance.now() - started) * 100) / 100
+    const { data, stats } = normalize(node, options)
+    const payloadBytes = new TextEncoder().encode(JSON.stringify(data) ?? '').length
+    return { ok: true, result: data, stats: { queryMs, normalize: stats, payloadBytes } }
+  }
+  catch (error) {
+    const e = error instanceof Error ? error : new Error(String(error))
+    return { ok: false, error: { message: e.message, name: e.name } }
+  }
+}
+
 interface JoraStatEntry {
   type: string
   from: number

plugins/data-inspector/src/inject/index.ts

@@ -5,10 +5,15 @@
  * Two ways in:
  *
  * ```ts
- * // 1. explicit, from the target's code
+ * // 1. explicit, from the target's code — pass sources inline …
  * import { exposeDataInspector } from '@devframes/plugin-data-inspector/inject'
- * import { registerDataSource } from '@devframes/plugin-data-inspector/registry'
  *
+ * await exposeDataInspector({
+ *   sources: [{ id: 'app:store', title: 'App store', data: () => store }],
+ * })
+ *
+ * // … or register them separately through the global registry.
+ * import { registerDataSource } from '@devframes/plugin-data-inspector/registry'
  * registerDataSource({ id: 'app:store', title: 'App store', data: () => store })
  * await exposeDataInspector()
  * ```
@@ -18,6 +23,11 @@
  * DEVFRAME_DATA_INSPECTOR=1 node --import @devframes/plugin-data-inspector/inject server.js
  * ```
  *
+ * On that zero-code path there's nowhere to call `registerDataSource`, so the
+ * agent auto-registers a **`globalThis`** source: assign anything you want to
+ * inspect onto the global object (`globalThis.store = store`) and query it
+ * live. Opt out with `DEVFRAME_DATA_INSPECTOR_GLOBAL=0`.
+ *
  * It binds `127.0.0.1` and requires devframe's trust handshake by
  * default: a random pre-shared token is minted per run, printed to stderr,
  * and written (with the endpoint) to the discovery file
@@ -27,6 +37,7 @@
  * treat the endpoint like a debugger port.
  */
 import type { DevframeHost, DevframeNodeContext } from 'devframe/types'
+import type { DataSourceEntry } from '../registry/index'
 import { mkdirSync, rmSync, writeFileSync } from 'node:fs'
 import { homedir } from 'node:os'
 import { dirname, join } from 'node:path'
@@ -49,6 +60,14 @@ export interface AgentDiscovery {
 }
 
 export interface ExposeDataInspectorOptions {
+  /**
+   * Data sources to expose, registered before the endpoint opens. A
+   * convenience over calling `registerDataSource` yourself; the two paths
+   * share one process-global registry, so inline sources and separately
+   * registered ones coexist (a later registration replaces an earlier one
+   * with the same id).
+   */
+  sources?: DataSourceEntry[]
   /** Preferred port (falls back to a free one nearby). Default 9878. */
   port?: number
   /**
@@ -77,12 +96,36 @@ export interface DataInspectorAgent {
   close: () => Promise<void>
 }
 
+/**
+ * A data source exposing the inspected process's global object. On the
+ * zero-code `--import` path there is nowhere to call `registerDataSource`, so
+ * assigning to `globalThis` is how you surface things to inspect
+ * (`globalThis.store = store`); this source makes them queryable live. The
+ * factory reads `globalThis` at query time, so late assignments show up on the
+ * next run.
+ */
+export function createGlobalThisDataSource(): DataSourceEntry {
+  return {
+    id: 'globalThis',
+    title: 'globalThis',
+    description: 'The global object of the inspected process. Assign values to inspect them, e.g. globalThis.store = store',
+    icon: 'i-ph:globe-duotone',
+    data: () => globalThis,
+  }
+}
+
 /** Start the agent endpoint in the current process. */
 export async function exposeDataInspector(options: ExposeDataInspectorOptions = {}): Promise<DataInspectorAgent> {
   // Deferred so `--import`ing the agent never pulls the whole node surface
   // into processes that don't enable it.
   const { setupDataInspector } = await import('../node/index')
 
+  if (options.sources?.length) {
+    const { registerDataSource } = await import('../registry/index')
+    for (const source of options.sources)
+      registerDataSource(source)
+  }
+
   const cwd = process.cwd()
   const port = await getPort({ port: options.port ?? 9878, portRange: [9878, 9978] })
   const websocket = `ws://127.0.0.1:${port}`
@@ -154,6 +197,9 @@ if (process.env.DEVFRAME_DATA_INSPECTOR === '1' || process.env.DEVFRAME_DATA_INS
     auth: process.env.DEVFRAME_DATA_INSPECTOR_AUTH !== '0',
     token: process.env.DEVFRAME_DATA_INSPECTOR_TOKEN,
     exampleSource: process.env.DEVFRAME_DATA_INSPECTOR_EXAMPLE !== '0',
+    // Zero-code path: with no chance to call `registerDataSource`, expose
+    // `globalThis` so assigning to it is enough to inspect anything.
+    sources: process.env.DEVFRAME_DATA_INSPECTOR_GLOBAL !== '0' ? [createGlobalThisDataSource()] : undefined,
   }).catch((error) => {
     console.error('[data-inspector] agent failed to start:', error)
   })