Lakekeeper warehouse endpoint refresh fails for warehouses created before the key prefix was added

[kz930]

What happened?

lakekeeper-init refreshes the existing warehouse's stored S3 endpoint on every up, so that the endpoint follows the host LAN IP across networks. That refresh now fails for any warehouse created before #7740, and the endpoint is never updated.

#7740 added key-prefix to both the create and the update payload in bin/single-node/docker-compose.yml. A warehouse created before it has key_prefix: null, and Lakekeeper refuses to change that field at all, so it rejects the whole request with HTTP 400 and Field 'key_prefix' cannot be updated to prevent loss of data. The endpoint is part of the same payload, so it stays stale forever.

The consequence is not obvious from the symptom. Once the host's LAN IP changes, every execution fails before any operator runs: IcebergUtil.createTable posts to the catalog, waits about thirty seconds on an S3 connect timeout to the old address, and gets an empty body back, which surfaces as RESTException: Unable to process: with nothing pointing at the endpoint or the warehouse. A second failure hides behind it, because that first failed execution leaves the workflow's service cached in a state where every later run throws a NullPointerException on executionRuntimeService before starting, which the UI shows as being stuck on "Submitting" indefinitely. Reloading the page does not clear it.

A fresh checkout is unaffected: the warehouse is created with the same prefix the refresh later sends, so the values match. What is affected is any environment whose warehouse predates 2026-08-19, which is every contributor who has been running the local dev stack since before that date.

Expected: the refresh updates the endpoint and leaves key_prefix as it is, rather than failing because of a field it did not intend to change.

How to reproduce?

1. Bring the stack up on a checkout from before feat(storage): give the default Iceberg warehouse its own key prefix #7740, so the warehouse is created without a key prefix.
2. Bring it up again on a checkout that includes feat(storage): give the default Iceberg warehouse its own key prefix #7740, on a machine whose LAN IP has changed since step 1.
3. lakekeeper-init exits 1 with the 400 above. bin/local-dev.sh up does not stop for it, so the run continues and looks successful.
4. Run any workflow. It fails before any operator runs. Every run after that hangs on "Submitting" until the service is bounced.

Confirm the cause by reading the stored profile with curl -s http://localhost:8181/management/v1/warehouse/<id> and comparing its endpoint against the machine's current address.

Two things worth noting for whoever picks this up. The init exits non-zero but the stack still reports as up, so the failure is only visible if you read that container's log. And the same request carries both the field being changed and the field that cannot be changed, so sending the stored key_prefix back, or omitting it on the update path, is enough to fix it.

Version/Branch

1.3.0-incubating-SNAPSHOT (main)

Commit Hash (Optional)

51f38e1

Relevant log output

Lakekeeper Warehouse 'texera' already exists. Refreshing its storage endpoint to 'http://texera-minio:9000'...
Failed to refresh Lakekeeper Warehouse storage. HTTP Code: 400
ERROR RESPONSE:
{"error":{"message":"Field key_prefix cannot be updated to prevent loss of data.","type":"UpdateError","code":400}}

[WARN] [org.apache.iceberg.rest.ErrorHandlers] - Unable to parse error response
java.io.UncheckedIOException: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
[ERROR] [org.apache.texera.web.service.WorkflowService] - error during execution
org.apache.iceberg.exceptions.RESTException: Unable to process:
at org.apache.iceberg.rest.RESTCatalog.createTable(RESTCatalog.java:132)
at org.apache.texera.amber.util.IcebergUtil$.createTable(IcebergUtil.scala:225)
at org.apache.texera.web.service.WorkflowExecutionService.executeWorkflow(WorkflowExecutionService.scala:132)

[ERROR] [org.apache.texera.web.resource.WorkflowWebsocketResource] - error occurred in websocket
java.lang.NullPointerException: Cannot invoke "org.apache.texera.web.service.ExecutionRuntimeService.unsubscribeAll()" because the return value of "org.apache.texera.web.service.WorkflowExecutionService.executionRuntimeService()" is null
at org.apache.texera.web.service.WorkflowExecutionService.unsubscribeAll(WorkflowExecutionService.scala:182)