|
| 1 | +<!-- |
| 2 | +Licensed to the Apache Software Foundation (ASF) under one |
| 3 | +or more contributor license agreements. See the NOTICE file |
| 4 | +distributed with this work for additional information |
| 5 | +regarding copyright ownership. The ASF licenses this file |
| 6 | +to you under the Apache License, Version 2.0 (the |
| 7 | +"License"); you may not use this file except in compliance |
| 8 | +with the License. You may obtain a copy of the License at |
| 9 | +
|
| 10 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 11 | +
|
| 12 | +Unless required by applicable law or agreed to in writing, |
| 13 | +software distributed under the License is distributed on an |
| 14 | +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| 15 | +KIND, either express or implied. See the License for the |
| 16 | +specific language governing permissions and limitations |
| 17 | +under the License. |
| 18 | +--> |
| 19 | + |
| 20 | +# Apache DataFusion Python 54.0.0 Changelog |
| 21 | + |
| 22 | +This release consists of 51 commits from 8 contributors. See credits at the end of this changelog for more information. |
| 23 | + |
| 24 | +**Breaking changes:** |
| 25 | + |
| 26 | +- Update datafusion dependency to latest in preparation for DF54 [#1532](https://github.com/apache/datafusion-python/pull/1532) (timsaucer) |
| 27 | +- feat: enable pickling for Python aggregate and window UDFs [#1545](https://github.com/apache/datafusion-python/pull/1545) (timsaucer) |
| 28 | +- feat: pass calling SessionContext to Python UDTF callbacks [#1555](https://github.com/apache/datafusion-python/pull/1555) (timsaucer) |
| 29 | +- feat: accept distinct kwarg on sum and avg [#1556](https://github.com/apache/datafusion-python/pull/1556) (timsaucer) |
| 30 | + |
| 31 | +**Implemented enhancements:** |
| 32 | + |
| 33 | +- feat: add AI skill to find and improve the Pythonic interface to functions [#1484](https://github.com/apache/datafusion-python/pull/1484) (timsaucer) |
| 34 | +- feat: enable pickling of most Expr except udaf and udwf [#1544](https://github.com/apache/datafusion-python/pull/1544) (timsaucer) |
| 35 | +- feat: expose variety of features from DF54 update [#1554](https://github.com/apache/datafusion-python/pull/1554) (timsaucer) |
| 36 | +- feat: Python UDFs: per-session inlining toggle and strict refusal setting [#1546](https://github.com/apache/datafusion-python/pull/1546) (timsaucer) |
| 37 | +- feat: create free-threaded python wheels [#1553](https://github.com/apache/datafusion-python/pull/1553) (timsaucer) |
| 38 | +- feat: expose lambda and higher-order array functions [#1561](https://github.com/apache/datafusion-python/pull/1561) (timsaucer) |
| 39 | +- feat: import user-defined physical optimizer rules over FFI [#1557](https://github.com/apache/datafusion-python/pull/1557) (timsaucer) |
| 40 | +- feat: expose SessionContext.copied_config and parse_capacity_limit [#1570](https://github.com/apache/datafusion-python/pull/1570) (timsaucer) |
| 41 | +- feat: expose array_compact, array_normalize, cosine_distance, inner_product [#1567](https://github.com/apache/datafusion-python/pull/1567) (timsaucer) |
| 42 | +- feat: expose arrow_field, arrow_try_cast, cast_to_type, with_metadata [#1568](https://github.com/apache/datafusion-python/pull/1568) (timsaucer) |
| 43 | +- feat: expose spark-compatible functions [#1564](https://github.com/apache/datafusion-python/pull/1564) (timsaucer) |
| 44 | +- feat: improve pythonic interface on date/time functions [#1563](https://github.com/apache/datafusion-python/pull/1563) (timsaucer) |
| 45 | + |
| 46 | +**Fixed bugs:** |
| 47 | + |
| 48 | +- fix: type scalar UDF returns as Arrow arrays [#1528](https://github.com/apache/datafusion-python/pull/1528) (BharatDeva) |
| 49 | +- fix: Skip `fork` and `forkserver` on `win32` [#1566](https://github.com/apache/datafusion-python/pull/1566) (nuno-faria) |
| 50 | + |
| 51 | +**Documentation updates:** |
| 52 | + |
| 53 | +- docs: enrich module docstrings and add doctest examples [#1498](https://github.com/apache/datafusion-python/pull/1498) (timsaucer) |
| 54 | +- docs: add README section for AI coding assistants [#1503](https://github.com/apache/datafusion-python/pull/1503) (timsaucer) |
| 55 | +- docs: add upstream sync process documentation [#1524](https://github.com/apache/datafusion-python/pull/1524) (timsaucer) |
| 56 | +- docs: document null-handling function arguments [#1527](https://github.com/apache/datafusion-python/pull/1527) (BharatDeva) |
| 57 | +- docs: user guide + runnable examples for distributing expressions [#1547](https://github.com/apache/datafusion-python/pull/1547) (timsaucer) |
| 58 | +- docs: convert reStructuredText sources to MyST markdown [#1579](https://github.com/apache/datafusion-python/pull/1579) (timsaucer) |
| 59 | + |
| 60 | +**Other:** |
| 61 | + |
| 62 | +- Release 53.0.0 [#1491](https://github.com/apache/datafusion-python/pull/1491) (timsaucer) |
| 63 | +- ci: disable symbol export on Windows verification [#1486](https://github.com/apache/datafusion-python/pull/1486) (timsaucer) |
| 64 | +- Add Python bindings for accessing ExecutionMetrics [#1381](https://github.com/apache/datafusion-python/pull/1381) (ShreyeshArangath) |
| 65 | +- Support None comparisons for null expressions [#1489](https://github.com/apache/datafusion-python/pull/1489) (zeel2104) |
| 66 | +- chore: update release documentation [#1494](https://github.com/apache/datafusion-python/pull/1494) (timsaucer) |
| 67 | +- Fix error on show() with an explain plan [#1492](https://github.com/apache/datafusion-python/pull/1492) (timsaucer) |
| 68 | +- Add SKILL.md and enrich package docstring [#1497](https://github.com/apache/datafusion-python/pull/1497) (timsaucer) |
| 69 | +- minor: fix header on agent SKILL.md file [#1501](https://github.com/apache/datafusion-python/pull/1501) (timsaucer) |
| 70 | +- tpch examples: rewrite queries idiomatically and embed reference SQL [#1504](https://github.com/apache/datafusion-python/pull/1504) (timsaucer) |
| 71 | +- Move public skills to a directory to avoid downloading the whole repo [#1519](https://github.com/apache/datafusion-python/pull/1519) (ntjohnson1) |
| 72 | +- Update user documentation for AI agent skill usage [#1505](https://github.com/apache/datafusion-python/pull/1505) (timsaucer) |
| 73 | +- build(deps): combined dependabot bumps (Cargo + workflows) [#1534](https://github.com/apache/datafusion-python/pull/1534) (timsaucer) |
| 74 | +- Add support for logical and physical codecs [#1541](https://github.com/apache/datafusion-python/pull/1541) (timsaucer) |
| 75 | +- Add details on caching to skill [#1521](https://github.com/apache/datafusion-python/pull/1521) (ntjohnson1) |
| 76 | +- Bump DataFusion to prepare for DF54 release candidate [#1562](https://github.com/apache/datafusion-python/pull/1562) (timsaucer) |
| 77 | +- Export `to_datafusion_err` from the util crate root [#1487](https://github.com/apache/datafusion-python/pull/1487) (kosiew) |
| 78 | +- chore: remove unused PyConfig [#1485](https://github.com/apache/datafusion-python/pull/1485) (timsaucer) |
| 79 | +- refactor(context): deduplicate register/read option-building logic [#1479](https://github.com/apache/datafusion-python/pull/1479) (mesejo) |
| 80 | +- Improve documentation site layout [#1578](https://github.com/apache/datafusion-python/pull/1578) (timsaucer) |
| 81 | +- Allow `DataFrame.aggregate` to accept `None` for no grouping [#1581](https://github.com/apache/datafusion-python/pull/1581) (kosiew) |
| 82 | +- Update to released DF 54.0.0 [#1588](https://github.com/apache/datafusion-python/pull/1588) (timsaucer) |
| 83 | +- build(deps): batch dependabot dependency updates [#1589](https://github.com/apache/datafusion-python/pull/1589) (timsaucer) |
| 84 | +- Deprecate `Expr` temporal part arguments in date extraction and truncation functions [#1587](https://github.com/apache/datafusion-python/pull/1587) (kosiew) |
| 85 | +- chore: update rust dependencies [#1604](https://github.com/apache/datafusion-python/pull/1604) (timsaucer) |
| 86 | +- Add deprecation warnings for Expr passed to confirmed literal-only function arguments [#1605](https://github.com/apache/datafusion-python/pull/1605) (kosiew) |
| 87 | +- chore: resolve audits after DF54.0.0 update to skill [#1608](https://github.com/apache/datafusion-python/pull/1608) (timsaucer) |
| 88 | +- chore: remove 3.13 freethreaded builds [#1609](https://github.com/apache/datafusion-python/pull/1609) (timsaucer) |
| 89 | + |
| 90 | +## Credits |
| 91 | + |
| 92 | +Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor. |
| 93 | + |
| 94 | +``` |
| 95 | + 39 Tim Saucer |
| 96 | + 4 kosiew |
| 97 | + 2 BharatDeva |
| 98 | + 2 Nick |
| 99 | + 1 Daniel Mesejo |
| 100 | + 1 Nuno Faria |
| 101 | + 1 Shreyesh |
| 102 | + 1 Zeel Desai |
| 103 | +``` |
| 104 | + |
| 105 | +Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release. |
| 106 | + |
0 commit comments