Skip to content

Conversation

@alexanderbianchi
Copy link

@alexanderbianchi alexanderbianchi commented Dec 19, 2025

What Was Implemented

1. Core API (crates/iceberg/src/io/file_io.rs)

Added RuntimeHandle type that wraps a tokio::runtime::Handle:

#[derive(Clone, Debug)]
pub struct RuntimeHandle(pub tokio::runtime::Handle);

impl RuntimeHandle {
    pub fn new(handle: tokio::runtime::Handle) -> Self;
    pub fn current() -> Self;
}

How It Works

  1. User configures runtime handle:

    let file_io = FileIOBuilder::new("s3")
        .with_extension(RuntimeHandle::new(io_handle))
        .build()?;
  2. iceberg-rust extracts handle during build:

    • Storage::build() gets RuntimeHandle from extensions
    • Creates CustomTokioExecutor wrapping the handle
    • Wraps in opendal::Executor
  3. Executor applied to operators:

    • When create_operator() is called, executor is applied via update_executor()
    • All opendal async operations spawn tasks on the configured runtime

@alexanderbianchi alexanderbianchi force-pushed the draft/runtime-handle-contribution branch 2 times, most recently from bd1b83d to df15d6e Compare December 19, 2025 19:36
@alexanderbianchi alexanderbianchi force-pushed the draft/runtime-handle-contribution branch from df15d6e to 166d2a8 Compare December 19, 2025 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant