Skip to content

[Python] concat_tables silently ignores unrecognised keyword arguments, including unify_schemas #50957

Description

@slachiewicz

Describe the bug, including details regarding any error messages, version, and platform.

pa.concat_tables accepts **kwargs but only ever inspects promote. Every other keyword is dropped without a warning or an error:

>>> import pyarrow as pa
>>> t1 = pa.Table.from_pydict({'a': [1.0]})
>>> t2 = pa.Table.from_pydict({'a': [2.0]})
>>> pa.concat_tables([t1, t2], totally_bogus_kwarg=123).num_rows
2

https://github.com/apache/arrow/blob/515410b2a1/python/pyarrow/table.pxi#L6324-L6338

Two ways this bites:

  1. A typo in a keyword is silently ignored rather than raising TypeError, which is what a reader would expect from a normal Python signature.
  2. unify_schemas in particular looks like it should work. It is a real field on the underlying ConcatenateTablesOptions, R's concat_tables() takes a unify_schemas argument, and the C++ option is what promote_options sets on the caller's behalf. So this reads as if it silently does the opposite of what was asked:
>>> pa.concat_tables(
...     [pa.Table.from_pydict({'a': [1]}), pa.Table.from_pydict({'a': [1.0]})],
...     unify_schemas=False, promote_options="permissive").schema.field('a').type
DataType(double)

Schemas were unified despite unify_schemas=False.

This surfaced in GH-38809, where the reporter passed unify_schemas=True and reasonably assumed it had an effect. The type-promotion half of that issue is fixed; this half is not.

A fix could be either to raise TypeError on unrecognised keywords, keeping **kwargs only for the deprecated promote, or to retire promote altogether (it has warned FutureWarning since 14.0.0) and give concat_tables an explicit signature with no **kwargs. The second is a breaking change and would need its own deprecation window; the first is not.

Reproduced on pyarrow 25.0.1, macOS/arm64, and the code path is unchanged on main.

Component(s)

Python

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions