Antalya 26:6 Fix alter operations for iceberg by subkanthi · Pull Request #2157 · Altinity/ClickHouse

subkanthi · 2026-08-04T01:00:24Z

continuation of work from #1841

Changelog category (leave one):

Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

...

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

Regression jobs to run:

github-actions · 2026-08-04T01:01:58Z

Workflow [PR], commit [a73db85]

subkanthi · 2026-08-06T21:11:05Z

ADD COLUMN

Ubuntu-2404-noble-amd64-base :) ALTER table ice.`default.dest5` add column new_column Nullable(UInt64);

ALTER TABLE ice.`default.dest5`
    (ADD COLUMN `new_column` Nullable(UInt64))

Query id: 825e8155-ec00-45f0-a54e-63291f46ef10

Ok.

0 rows in set. Elapsed: 0.314 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`default.dest5`;

SHOW CREATE TABLE ice.`default.dest5`

Query id: 0c79176a-55dd-4aa4-a4c4-2125f5cde5fe

   ┌─statement────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`default.dest5`                        ↴│
   │↳(                                                       ↴│
   │↳    `event_month` Int32,                                ↴│
   │↳    `id` Int64,                                         ↴│
   │↳    `event_time` DateTime64(6),                         ↴│
   │↳    `user_id` Int32,                                    ↴│
   │↳    `category_name` String,                             ↴│
   │↳    `value` Float64,                                    ↴│
   │↳    `payload` String,                                   ↴│
   │↳    `col2` Nullable(Int64),                             ↴│
   │↳    `new_column` Nullable(Int64)                        ↴│
   │↳)                                                       ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/dest6/') │
   └──────────────────────────────────────────────────────────┘

DROP COLUMN

 alter table ice.`default.dest5` drop column col2;

ALTER TABLE ice.`default.dest5`
    (DROP COLUMN col2)

Query id: a7905249-670b-4ac2-a626-f95535c02730

Connecting to database ice at localhost:9007 as user default.
Connected to ClickHouse server version 26.6.2.

Ok.

0 rows in set. Elapsed: 0.122 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`default.dest5`;

SHOW CREATE TABLE ice.`default.dest5`

Query id: 9d3b156a-ca6d-4997-917a-8bf36d4d7058

   ┌─statement────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`default.dest5`                        ↴│
   │↳(                                                       ↴│
   │↳    `event_month` Int32,                                ↴│
   │↳    `id` Int64,                                         ↴│
   │↳    `event_time` DateTime64(6),                         ↴│
   │↳    `user_id` Int32,                                    ↴│
   │↳    `category_name` String,                             ↴│
   │↳    `value` Float64,                                    ↴│
   │↳    `payload` String,                                   ↴│
   │↳    `new_column` Nullable(Int64)                        ↴│
   │↳)                                                       ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/dest6/') │
   └──────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.016 sec.

RENAME COLUMN

 alter table ice.`default.dest5` rename column new_column to new_column_2;

ALTER TABLE ice.`default.dest5`
    (RENAME COLUMN new_column TO new_column_2)

Query id: e464f77e-0e3d-4ffc-883e-70db1260327f

Ok.

0 rows in set. Elapsed: 0.169 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`default.dest5`;

SHOW CREATE TABLE ice.`default.dest5`

Query id: 0b4b4e06-7d58-4583-821a-95e27f856ba9

   ┌─statement────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`default.dest5`                        ↴│
   │↳(                                                       ↴│
   │↳    `event_month` Int32,                                ↴│
   │↳    `id` Int64,                                         ↴│
   │↳    `event_time` DateTime64(6),                         ↴│
   │↳    `user_id` Int32,                                    ↴│
   │↳    `category_name` String,                             ↴│
   │↳    `value` Float64,                                    ↴│
   │↳    `payload` String,                                   ↴│
   │↳    `new_column_2` Nullable(Int64)                      ↴│
   │↳)                                                       ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/dest6/') │
   └──────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.017 sec.

subkanthi · 2026-08-06T23:41:17Z

 {
    switch (type->getTypeId())
    {
+        case TypeIndex::UInt8:


Added support for bool

subkanthi · 2026-08-06T23:41:31Z

            return {"string", true};
        case TypeIndex::UUID:
            return {"uuid", true};
+        case TypeIndex::Decimal32:


Added support for iceberg decimal types.

….com/Altinity/ClickHouse into antalya_26_6_fix_alter_table_iceberg

subkanthi · 2026-08-07T20:19:46Z

Decimal support


ALTER TABLE ice.`flowers.sample`
ADD COLUMN dec_col Nullable(Decimal(10,2));

ALTER TABLE ice.`flowers.sample`
    (ADD COLUMN `dec_col` Nullable(Decimal(10, 2)))

Query id: 83a0ae12-2b00-40de-85ba-654daa0d7dc0

Ok.

0 rows in set. Elapsed: 0.203 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`flowers.sample`;

SHOW CREATE TABLE ice.`flowers.sample`

Query id: bb3b8676-9b7e-4000-9b08-2fcb30b65079

   ┌─statement─────────────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`flowers.sample`                                ↴│
   │↳(                                                                ↴│
   │↳    `value` Nullable(Int64),                                     ↴│
   │↳    `boolean_col` Nullable(Bool),                                ↴│
   │↳    `dec_col` Nullable(Decimal(10, 2))                           ↴│
   │↳)                                                                ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/flowers/sample/') │
   └───────────────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.009 sec.

DimensionWieldr · 2026-08-10T19:55:13Z

Context: #2090

Retested with this PR's build.

Code: 290. DB::Exception: Too many unsuccessed retries to alter iceberg table. (LIMIT_EXCEEDED)
(query: ALTER TABLE … DROP COLUMN new_column_…)

Previously this class of failure surfaced as Code 736 (Iceberg alter: catalog commit failed … after metadata file was written successfully). With the retry loop, permanent catalog rejections are retried until the budget is exhausted, so the user-visible code is now 290.

Every retry hits the same HTTP 400:

{"error":{"message":"Invalid last column ID: 41 < 42 (previous last column ID)",
          "type":"IllegalArgumentException","code":400}}

So ClickHouse is proposing a schema update whose last-column-id decreases. Iceberg requires last-column-id to be non-decreasing. On DROP COLUMN (and any schema evolution), is the last-column-id being written incorrectly somewhere?

mkmkme · 2026-08-11T10:26:11Z

I asked Claude to analyze this PR and it came with some review. Please have a look: https://gist.github.com/mkmkme/b28a41f136a62d5a6d41a73dee0132bd

….com/Altinity/ClickHouse into antalya_26_6_fix_alter_table_iceberg

… fix medium defects

subkanthi · 2026-08-11T23:06:30Z


Ubuntu-2404-noble-amd64-base :) alter table ice.`flowers.sample` add column dec_col2 Nullable(Decimal(10,2));

ALTER TABLE ice.`flowers.sample`
    (ADD COLUMN `dec_col2` Nullable(Decimal(10, 2)))

Query id: 37668c59-de09-4b90-84ec-a542e702b05a

Ok.

0 rows in set. Elapsed: 0.249 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`flowers.sample`;

SHOW CREATE TABLE ice.`flowers.sample`

Query id: 54a5faf6-7016-41ab-865d-93f6102f6808

   ┌─statement─────────────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`flowers.sample`                                ↴│
   │↳(                                                                ↴│
   │↳    `value` Nullable(Int64),                                     ↴│
   │↳    `boolean_col` Nullable(Bool),                                ↴│
   │↳    `dec_col` Nullable(Decimal(10, 2)),                          ↴│
   │↳    `dec_col2` Nullable(Decimal(10, 2))                          ↴│
   │↳)                                                                ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/flowers/sample/') │
   └───────────────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.012 sec. 

Ubuntu-2404-noble-amd64-base :) alter table ice.`flowers.sample` drop column dec_col2;

ALTER TABLE ice.`flowers.sample`
    (DROP COLUMN dec_col2)

Query id: 49447f08-f164-47a1-aff3-d4852e4736cb

Ok.

0 rows in set. Elapsed: 0.147 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`flowers.sample`;

SHOW CREATE TABLE ice.`flowers.sample`

Query id: 8cd4a422-13d5-472a-9cf2-5208c79e0a99

   ┌─statement─────────────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`flowers.sample`                                ↴│
   │↳(                                                                ↴│
   │↳    `value` Nullable(Int64),                                     ↴│
   │↳    `boolean_col` Nullable(Bool),                                ↴│
   │↳    `dec_col` Nullable(Decimal(10, 2))                           ↴│
   │↳)                                                                ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/flowers/sample/') │
   └───────────────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.006 sec. 

Ubuntu-2404-noble-amd64-base :) alter table ice.`flowers.sample` rename column dec_col to dec_col_2;

ALTER TABLE ice.`flowers.sample`
    (RENAME COLUMN dec_col TO dec_col_2)

Query id: 816db79c-7370-4ab2-8a6c-d8535337c2aa

Ok.

0 rows in set. Elapsed: 0.119 sec. 

Ubuntu-2404-noble-amd64-base :) show create table ice.`flowers.sample`;

SHOW CREATE TABLE ice.`flowers.sample`

Query id: b752811b-2a78-4097-ba40-5411e1f2a140

   ┌─statement─────────────────────────────────────────────────────────┐
1. │ CREATE TABLE ice.`flowers.sample`                                ↴│
   │↳(                                                                ↴│
   │↳    `value` Nullable(Int64),                                     ↴│
   │↳    `boolean_col` Nullable(Bool),                                ↴│
   │↳    `dec_col_2` Nullable(Decimal(10, 2))                         ↴│
   │↳)                                                                ↴│
   │↳ENGINE = Iceberg('http://localhost:9000/bucket1/flowers/sample/') │
   └─────────────────────────────────────────────────────────────────

subkanthi · 2026-08-13T03:50:30Z

I asked Claude to analyze this PR and it came with some review. Please have a look: https://gist.github.com/mkmkme/b28a41f136a62d5a6d41a73dee0132bd

@mkmkme can u check again please

Fix alter operations for iceberg

e93ce1c

subkanthi added 2 commits August 4, 2026 10:18

Merge branch 'antalya-26.6' into antalya_26_6_fix_alter_table_iceberg

fa311a9

Merge branch 'antalya-26.6' into antalya_26_6_fix_alter_table_iceberg

818f446

subkanthi changed the title ~~Fix alter operations for iceberg~~ Antalya 26:6 Fix alter operations for iceberg Aug 6, 2026

subkanthi added 3 commits August 5, 2026 22:16

Merge branch 'antalya-26.6' into antalya_26_6_fix_alter_table_iceberg

d62b9ac

fix medium defects with retries

0eb31fc

Added support for boolean and decimal

f76fd1d

subkanthi marked this pull request as ready for review August 6, 2026 04:26

subkanthi added 2 commits August 6, 2026 16:46

Add missing include to gtest_iceberg_type_mapping

bdb0d15

Merge branch 'antalya-26.6' into antalya_26_6_fix_alter_table_iceberg

d3b65e3

subkanthi closed this Aug 6, 2026

subkanthi reopened this Aug 6, 2026

subkanthi added the antalya-26.6 label Aug 6, 2026

subkanthi closed this Aug 6, 2026

subkanthi reopened this Aug 6, 2026

subkanthi closed this Aug 6, 2026

subkanthi reopened this Aug 6, 2026

subkanthi closed this Aug 6, 2026

subkanthi reopened this Aug 6, 2026

subkanthi commented Aug 6, 2026

View reviewed changes

subkanthi added 2 commits August 7, 2026 03:13

Restored removed tests

04c009e

Merge branch 'antalya_26_6_fix_alter_table_iceberg' of https://github…

a4e6f39

….com/Altinity/ClickHouse into antalya_26_6_fix_alter_table_iceberg

This was referenced Aug 7, 2026

Iceberg DataLakeCatalog: ALTER DELETE fails with Metadata is not initialized (LOGICAL_ERROR) #2136

Open

Iceberg ALTER ADD COLUMN does not support Bool/Decimal #2089

Open

subkanthi added 2 commits August 7, 2026 22:34

Fix to support decimal datatypes in iceberg

6e6f1f7

Merge branch 'antalya-26.6' into antalya_26_6_fix_alter_table_iceberg

ad9e5c7

subkanthi added 9 commits August 11, 2026 18:29

Fix nullptr exception when there are no snapshots

8e9c6f7

Merge branch 'antalya_26_6_fix_alter_table_iceberg' of https://github…

3da03ba

….com/Altinity/ClickHouse into antalya_26_6_fix_alter_table_iceberg

Fix compiler error

bedcad9

Merge branch 'antalya-26.6' into antalya_26_6_fix_alter_table_iceberg

304fc24

Fix compiler error

12df291

Merge branch 'antalya_26_6_fix_alter_table_iceberg' of https://github…

7f35e6c

….com/Altinity/ClickHouse into antalya_26_6_fix_alter_table_iceberg

Revert back change to settings regarding iceberg manifest compaction,…

d4c5df2

… fix medium defects

Reverted back settings changes(removal)

422f409

Reverted back settings changes(removal)

8bbea38

subkanthi added 2 commits August 12, 2026 01:07

Address medium defects

c72983e

Fix high defect with nullptr in failsafe

a73db85

Navigation Menu

Uh oh!

Antalya 26:6 Fix alter operations for iceberg - #2157

Antalya 26:6 Fix alter operations for iceberg#2157
subkanthi wants to merge 23 commits into
antalya-26.6from
antalya_26_6_fix_alter_table_iceberg

subkanthi commented Aug 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

subkanthi commented Aug 6, 2026 •

edited

Loading

Uh oh!

subkanthi Aug 6, 2026

Uh oh!

subkanthi Aug 6, 2026

Uh oh!

subkanthi commented Aug 7, 2026 •

edited

Loading

Uh oh!

DimensionWieldr commented Aug 10, 2026

Uh oh!

mkmkme commented Aug 11, 2026

Uh oh!

subkanthi commented Aug 11, 2026

Uh oh!

subkanthi commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

subkanthi commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

CI/CD Options

Exclude tests:

Regression jobs to run:

Uh oh!

github-actions Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

subkanthi commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

subkanthi Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

subkanthi Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

subkanthi commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DimensionWieldr commented Aug 10, 2026

Uh oh!

mkmkme commented Aug 11, 2026

Uh oh!

subkanthi commented Aug 11, 2026

Uh oh!

subkanthi commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

subkanthi commented Aug 4, 2026 •

edited

Loading

github-actions Bot commented Aug 4, 2026 •

edited

Loading

subkanthi commented Aug 6, 2026 •

edited

Loading

subkanthi commented Aug 7, 2026 •

edited

Loading