(pkg-R) Encountering latency with querychat server when using complex database

[jperone]

Hello, thank you for this package and the update, it's been integral to my work.

I've run into a loading issue with my app after converting to use version 0.2.0.9000. In the initial pkg version, I supplied the full system prompt, data description, and extra instructions so nothing was generated by the LLM, which produced a very quick app startup time. Now, I supply it the same way but I'm getting 50K+ms of load time on app startup. I profiled and it looks like the get_schema_impl() method is where the bottleneck is:

My data source is a duckdb table with 1.5M rows with about 52 columns, of which I intend to supply the full schema in the prompt, as it won't change much in my case. It seems as though calling $server() also creates a call to my duckdb conn to imply the schema, which I don't need as I supply it to the LLM. Maybe I'm misinterpreting some of the profiling, but I was wondering if there is a way to circumvent the schema imply, if data_description is supplied? Thank you

[cpsievert]

Thanks for reporting!

In the initial pkg version, I supplied the full system prompt, data description, and extra instructions so nothing was generated by the LLM

Could you provide a little code snippet of how this worked with the initial package?

[gadenbuie]

@cpsievert we can do this by putting the schema info gathering behind delayedAssign() and having the user provide a custom template

library(whisker)

# Template data
data <- new.env()
data$name <- "Alice"
data$date <- Sys.Date()

# Simulate expensive calculation
expensive_info <- function() {
  message("Computing expensive info...")
  Sys.sleep(2)
  "This took a long time to calculate"
}

# Lazily evaluate the expensive info
delayedAssign("expensive_info", expensive_info(), assign.env = data)

# Template that doesn't use expensive_info - no computation happens
template1 <- "Hello {{name}}! Today is {{date}}."
whisker.render(template1, data = data)
#> [1] "Hello Alice! Today is 2026-02-09."

# Template that uses expensive_info - computation happens here
template2 <- "Hello {{name}}! Info: {{expensive_info}}"
whisker.render(template2, data = data)
#> Computing expensive info...
#> [1] "Hello Alice! Info: This took a long time to calculate"

[jperone]

Thanks for reporting!

In the initial pkg version, I supplied the full system prompt, data description, and extra instructions so nothing was generated by the LLM

Could you provide a little code snippet of how this worked with the initial package?

Here is a brief snippet of what i was doing before if that helps:

querychat_config <- reactive({
  querychat_init(
    data_source = querychat_conn(),
    system_prompt = aris_system_prompt(),
    greeting = aris_greeting(),
    client = ellmer::chat_azure_openai(
      endpoint = azure_openai_endpoint, 
      api_key = azure_openai_api_key, 
      model = azure_openai_model_version
    )
  )
})

So just supplying the system prompt and greeting. I misspoke, I do include the data description/schema info in the system prompt I pass in.