nextchanges: Enforce changelog fragment format by janniklasrose · Pull Request #6395 · databricks/cli

janniklasrose · 2026-08-27T12:17:54Z

Changes

Update tools/validate_nextchanges.py to be more opinionated:
- Leading bullet * required
- All #123 numbers are expanded github links (issues and PRs)
- Ends in a bullet and has trailing PR in parentheses.
  - Trailing PR link is skipped if no PR is associated to the branch/commit (e.g. while locally developing)
Remove tools/update_github_links.py and have all link validation be done in tools/validate_nextchanges.py
.github/workflows/changelog-preview.yml workflow (runs on PRs and pushes to main) does strict validation on the PR being added to new fragments

Why

We've been skipping PR numbers/links for a few releases. This enforces it.

Tests

Added 4 test.md files (see https://github.com/databricks/cli/tree/99d5458b3248482f21e2b5f12568bbd8d74d6c44/.nextchanges/cli)

Workflow validation of test files added to this PR:

/home/runner/work/cli/cli/.nextchanges/cli/test1.md: must start with a "* " bullet marker
/home/runner/work/cli/cli/.nextchanges/cli/test2.md: unexpanded reference #123: write it as a markdown link, e.g. [#123](https://github.com/databricks/cli/pull/123)
/home/runner/work/cli/cli/.nextchanges/cli/test3.md: trailing PR link #6394 must include the PR that added this fragment (#6395)

When fixing the PR attribution in test3.md to 6395 locally and running task changelog-preview it passes (because locally the test calls gh and associates the branch to an open PR). If running locally before opening a PR (or quickly renaming branch) the test passes since we don't enforce PR link logic there.

Valid entries look like this (task changelog-preview):

## Release v1.15.0 (2026-08-27)

### CLI

 * This is an entry. ([#6395](https://github.com/databricks/cli/pull/6395))
 * This is another entry. This feature was developed in a stack of PRs. ([#6394](https://github.com/databricks/cli/pull/6394), [#6395](https://github.com/databricks/cli/pull/6395))

…extchanges

…nges-enforce-format

…iling period

eng-dev-ecosystem-bot · 2026-08-27T13:07:35Z

Integration test report

Commit: 6b463c6

Run: 33077006761

	Env	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux	1	4	274	1191	4:09
💚	aws windows	1	4	276	1189	3:29
💚	azure linux	1	4	273	1191	4:01
💚	azure windows	1	4	275	1189	3:30
💚	gcp linux	1	4	274	1191	4:14
💚	gcp windows	1	4	276	1189	3:30

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
3:26	azure windows	TestAccept
3:25	aws windows	TestAccept
3:15	gcp windows	TestAccept

pietern · 2026-09-01T09:29:44Z

 - Create `.nextchanges/<section>/<name>.md`, picking the section folder that fits: `cli`, `bundles`, `dependency-updates`, `notable-changes`, or `api-changes`. `<name>` is arbitrary (a feature name or your PR number) — just keep it unique.
- Write one or two sentences in user-facing language, no Jira links. The leading `* ` is optional. Match the voice and tense of existing changelog entries.
- A PR link is optional: write `(#NNNN)` (with NNNN being the PR number) in the text and it's expanded to a full link automatically.
+- Write a single line in user-facing language, no Jira links: start it with a `* ` bullet marker and end it with a period. Match the voice and tense of existing changelog entries.


Re: the voice and tense, this should be prescriptive because existing entries are ambiguous.

True, but will do this as follow-up (unrelated to this change). Agents are too verbose in the entries currently...

pietern · 2026-09-01T09:31:22Z

+      # is inferred from the commit that added it.
      - name: Validate .nextchanges placement
-        run: uv run tools/validate_nextchanges.py
+        run: uv run tools/validate_nextchanges.py --strict ${{ github.event_name == 'pull_request' && format('--pr-number {0}', github.event.pull_request.number) || '' }}


This always fails on the first push. Could we e.g. not run this on drafts or something?

This is a very cheap test (only takes a couple of seconds or so) so failing on draft actually gives you/the agent the signal that it will need an update before making this a proper PR. On the flip side, you can create a draft and have this skipped/pass, and once you do changes locally and run your local check it will pick up the PR and fail.

All that being sad, there's pros and cons and I'm ok to skip the workflow on draft but don't see the value right now (since there's no clear upside of using drafts, even integration tests run on draft; maybe something we can actually change too?)

pietern · 2026-09-01T09:33:02Z

-    problems = find_problems(changelog_dir, sections)
+    # A trailing PR link is required whenever the change can be associated with a
+    # PR, and must name that PR. CI passes --strict (pull requests and pushes to
+    # main) so enforcement never fails open there, plus --pr-number for pull


What happens if we have to squash merge something without a changelog entry?

If it doesn't have a changelog entry at all, this won't be enforced. We only enforce the PR link for new fragments.

denik · 2026-09-01T09:38:46Z

+    parser.add_argument(
+        "--strict",
+        action="store_true",
+        help="fail closed: require every fragment's trailing PR link even when the branch's PR can't be auto-detected (set in CI)",


do we ever need a non-strict mode?

Yes, for failing open. But as you stated below, this can be inferred via the CI env variable.

denik · 2026-09-01T09:39:21Z

+    parser.add_argument(
+        "--pr-number",
+        default=None,
+        help="the PR under review; used as the expected link for not-yet-merged fragments (CI passes it for pull requests)",


Unclear what this does from the description. Especially give that we have code above to get current PR?

basically there are three modes:

Local run, no PR yet: fail open on gh command
Local run, with PR: fail open on gh command, but if we can get PR, update in place
CI run: require link to be present, don't run PR number check.

CI can be detected via env var, no script argument necessary.

but if we can get PR, update in place

not implemented, but this would be a nice addition!

CI run: don't run PR number check

do run the PR number check, because you might add the wrong link. That's what --pr-number {{ github.event.pull_request.number }} is for, to be authoritative and not infer (avoid having to handle multiple PRs from the same branch etc)

That's what --pr-number {{ github.event.pull_request.number }} is for, to be authoritative and not infer (avoid having to handle multiple PRs from the same branch etc)

That makes sense!

not implemented, but this would be a nice addition!

can we have this? without it it's almost not worth it.

can we have this? without it it's almost not worth it.

will do the auto-fix as a follow-up. I think it's still worth it to enforce the format in CI, even if not auto-fixable.

The best-effort `gh pr view` lookup swallowed OSError/SubprocessError silently, hiding why local PR detection did nothing. Print the error to stderr so it is visible. Co-authored-by: Isaac <no-reply@databricks.com>

…nges-enforce-format

janniklasrose added 8 commits August 26, 2026 15:30

Remove update_github_links.py

c5c6190

Update validate_nextchanges.py

6a9f7d3

Make changelog-preview pass PR number and --strict flag to validate_n…

2eeb831

…extchanges

Documentation

8e9768b

TEST ONLY: add test fragments

2e51904

Fix test4 which should be happy path

7f4fbc8

Merge remote-tracking branch 'origin/main' into janniklasrose/nextcha…

7909e2a

…nges-enforce-format

Correctly attribute trailing parentheses group with wrong link vs tra…

99d5458

…iling period

janniklasrose marked this pull request as ready for review August 27, 2026 12:54

janniklasrose requested a review from denik August 27, 2026 12:54

Remove test files

6b463c6

janniklasrose mentioned this pull request Aug 27, 2026

nextchanges: Automatically add the PR link to a newly added nextchanges fragments #6177

Closed

pietern reviewed Sep 1, 2026

View reviewed changes

denik reviewed Sep 1, 2026

View reviewed changes

janniklasrose and others added 7 commits September 1, 2026 12:04

Infer cli args from CI env var and PR_NUMBER (added to workflow)

bd6956d

Surface gh error when branch PR detection fails

a4c33b3

The best-effort `gh pr view` lookup swallowed OSError/SubprocessError silently, hiding why local PR detection did nothing. Print the error to stderr so it is visible. Co-authored-by: Isaac <no-reply@databricks.com>

Merge remote-tracking branch 'origin/main' into janniklasrose/nextcha…

8f8903a

…nges-enforce-format

Fix existing violations

6a8f368

Merge remote-tracking branch 'origin/main' into janniklasrose/nextcha…

2983fbb

…nges-enforce-format

Fix more files

9b06a48

More fixes

078c1af

janniklasrose requested review from denik and pietern September 1, 2026 11:55

janniklasrose and others added 3 commits September 2, 2026 17:08

Merge branch 'main' into janniklasrose/nextchanges-enforce-format

39ea9dc

Fix violations

e66ee02

Fix Bump Go toolchain changelog entry format

b93db2a

denik approved these changes Sep 2, 2026

View reviewed changes

janniklasrose enabled auto-merge September 2, 2026 15:35

janniklasrose added this pull request to the merge queue Sep 2, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 2, 2026

Skip requiring PR links on shallow clones

c11a60e

janniklasrose enabled auto-merge September 2, 2026 16:08

janniklasrose added this pull request to the merge queue Sep 2, 2026

Merged via the queue into main with commit 1a7f8df Sep 2, 2026
41 checks passed

janniklasrose deleted the janniklasrose/nextchanges-enforce-format branch September 2, 2026 16:41

Sankalp-Mittal mentioned this pull request Sep 2, 2026

cluster_policies: Stop allowing Sequence types for value #6492

Open

Uh oh!

Conversation

janniklasrose commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

eng-dev-ecosystem-bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

janniklasrose commented Aug 27, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Aug 27, 2026 •

edited

Loading