DOC-2254: Add Confluent Schema Registry schema migration via Shadowing#1776
Merged
Conversation
Documents the new v26.2 Schema Registry API-mode replication (shadow_schema_registry_api) for shadow links: - New page: Migrate Schemas from Confluent Schema Registry (intro, use cases, prerequisites, limitations, config steps, filtering, context mapping, validation policy, monitoring, failover) - setup.adoc: present topic mode vs API mode in the Schema Registry synchronization section - overview.adoc: add Schema Registry Sync task tab and update the replication lists - monitor.adoc: add Schema Registry sync status bullet - nav.adoc: add nav entry under Shadowing Feature is unreleased (ENG-1171, targets 26.2). DRAFT-VERIFY comments mark items to re-check after redpanda#30999 and redpanda#30984 merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Feediver1
commented
Jul 8, 2026
Intro now follows what-it-does / how-it-works / benefit / when-to-use instead of opening with an imperative. Duplicate glossterms downgraded now that first mentions moved into the intro. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lication Full-file example anchors the per-setting fragments that follow: name + client_options + topic sync (from the canonical setup.adoc sample) plus the complete shadow_schema_registry_api section (field names verified against rpk types_test.go). Links to Configure Shadowing for the full connection/auth reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds lines=20-38 highlighting to the sample shadow link configuration so the shadow_schema_registry_api block stands out from the framing sections, and points to it from the lead-in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8 tasks
Feediver1
requested review from
bartoszpiekny-redpanda,
mattschumpert and
pgellert
July 8, 2026 23:56
Addresses review feedback asking for explicit guidance on choosing between the two Schema Registry sync modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pgellert
reviewed
Jul 13, 2026
pgellert
reviewed
Jul 13, 2026
…as-confluent.adoc Co-authored-by: Gellért Peresztegi-Nagy <gellert.nagy@redpanda.com>
Corrections and additions from pgellert's review: - Two cluster tunables do exist (schema_registry_sync_memory_bytes, schema_registry_sync_parallelism); replace the "no cluster configuration properties" claim - mTLS is supported alongside HTTP basic auth for the source registry - Prerequisites: destination contexts owned by the link must be empty (rest of the registry need not be); qualified subjects config required for non-default contexts - Limitations: topic data and schema replication are not coordinated (SerDes records can reference not-yet-replicated schema IDs) - Remove topic_metadata_sync_options from the sample config and fix the highlight range - Filters accept literal names only (no wildcards/prefixes) - exact context mappings must be exhaustive: unmapped source contexts fail the task; recommend scoping source_filter - List the unsupported Confluent features (rule sets, metadata tags, override/default metadata and rule sets, compatibility groups) - Monitoring: task runs on the _schemas partition leader; counters are ephemeral and reset on restart or leadership movement - Failover: pausing the SR sync task (paused: true) is a required step; failover alone does not stop schema replication - overview.adoc: only API mode runs as a separate task in link status Resolves all DRAFT-VERIFY items except the sync-defaults re-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
default_tail_interval=10s, default_full_sync_interval=5m, default_max_source_requests_per_second=30 confirmed in src/v/cluster_link/model/types.h at the v26.2.1-rc3 tag, matching the documented values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
trevpanda
requested changes
Jul 16, 2026
trevpanda
left a comment
trevpanda
left a comment
Contributor
There was a problem hiding this comment.
This looks great, I left a few suggestions.
Co-authored-by: Trevor Blackford <trevor.blackford@redpanda.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
pgellert
reviewed
Jul 17, 2026
…as-confluent.adoc Co-authored-by: Gellért Peresztegi-Nagy <gellert.nagy@redpanda.com>
Per Gellert's answers to Trevor's review questions: - Rework the Select contexts and subjects example so the two lists no longer overlap (the old example's subject was already inside a listed context), and add the union semantics from shadow_link.proto: contexts replicate in full, subjects use qualified syntax, both lists union, overlaps replicate once - Extend the SerDes coordination limitation with the user impact: no replication errors (schema IDs not validated during topic data replication), but consumers fail on unreplicated schema IDs, so wait for schema replication to catch up before consuming Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cloud-docs has no consuming stub for the new page (the cloud twin is deferred to DOC-2307), so xrefs to it from the shared regions of the single-sourced shadowing pages would break every cloud build with target-not-found errors. Gate the cross-link sentences ifndef::env-cloud, gate monitor.adoc's Schema Registry sync status bullet entirely (it describes API-mode-only output), and give the overview's replication paragraph a cloud variant that mentions topic-mode replication only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Feediver1
commented
Jul 20, 2026
Feediver1
left a comment
Feediver1
left a comment
Contributor
Author
There was a problem hiding this comment.
Final-pass review: DOC-2254 Confluent Schema Registry migration (#1776)
Files reviewed: 5 (new migration guide, setup, overview, monitor, nav)
Overall assessment: Content-wise the most thoroughly SME-vetted page in the shadowing set — 23/23 review threads addressed across three rounds with commit-cited replies. The final-pass single-sourcing audit found one pre-merge blocker, now fixed in 6cedb4a.
Critical issue — found and fixed (6cedb4a)
Four ungated xrefs to migrate-schemas-confluent.adoc, which has no cloud-docs consuming stub (the cloud twin is deferred to DOC-2307). All four sat in shared regions of the single-sourced pages — verified by conditional-depth analysis — so every cloud build would have failed with target of xref not found once this content flowed downstream. Fixed:
monitor.adoc: the "Schema Registry sync status" bullet is nowifndef::env-cloud[]-gated in full (it describes API-mode-only output; consistent with the adjacent role-sync bullet's treatment).overview.adoc(task tab + replication paragraph): cross-link sentences gated; the replication paragraph gets a cloud variant that mentions topic-mode replication only.setup.adoc(API-mode bullet): cross-link sentence gated.
All four sites re-audited as gated after the fix.
Review-state summary
- All 23 human threads content-addressed; the last two (filter-union semantics per shadow_link.proto, SerDes consumer impact) landed in 84f515f. Six threads await only resolve-clicks.
- Trevor's re-review is requested; his CHANGES_REQUESTED decision is the remaining gate.
- Mechanical lint of all changed files: clean.
- Glossterm dependencies:
shadow link/shadow clusterresolve against #1777 — merge order stands: #1777 first, then this PR tobetawith branch-delete so #1780 auto-retargets.
Suggestion (non-blocking)
overview.adoc lines ~197–220 carry pre-existing relative xrefs (xref:./setup.adoc[…], from PR #1491 — not introduced here). They happen to resolve in both builds because all four targets have cloud stubs, but they violate the full-resource-ID convention. Cheap cleanup while this file is open, or a follow-up.
Impact notes
- cloud-docs: no action needed now that the gating is in; DOC-2307 owns the eventual cloud stub and un-gating.
- Release notes: confirm the 26.2 release-notes entry for Confluent SR replication is covered by the GA regen rather than assumed.
Jira
DOC-2254 (ENG-1171) fully addressed for the Self-Managed scope, with the Cloud deferral now leak-free.
🤖 Generated with Claude Code
trevpanda
approved these changes
Jul 20, 2026
trevpanda
left a comment
trevpanda
left a comment
Contributor
There was a problem hiding this comment.
Looks great! Thank you!
Five xref:./ links (from PR #1491) resolved only by luck of the consuming repos' identical layout. Use full Antora resource IDs per convention; all four targets have cloud-docs stubs, so both renderings resolve. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Feediver1
added a commit
that referenced
this pull request
Jul 21, 2026
…erged # Conflicts: # modules/manage/pages/disaster-recovery/shadowing/monitor.adoc # modules/manage/pages/disaster-recovery/shadowing/overview.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the new v26.2 Schema Registry API-mode replication (
shadow_schema_registry_api) for shadow links:Review deadline: July14
Description
Resolves https://redpandadata.atlassian.net/browse/DOC-2254
Page previews
Sources of truth
All YAML field names verified against
src/go/rpk/pkg/cli/shadow/types.go(dev) and thetypes_test.goexample config. Defaults (10s tail / 5m full sync / 30 rps,FAILpolicy) fromsrc/v/cluster_link/model/types.h. Merged engineering PRs: redpanda#30695, #30723, #30764, #30835, #30860, #30905, #30933.Two engineering PRs were still open at draft time.
// DRAFT-VERIFY (DOC-2254)comments in the new page mark each item:paused(redpanda#30984)source_filter(epic mentions wildcards; rpk source shows plain string lists)rpk shadow createargument order against a 26.2 buildRelated
single-sourcetags withenv-cloudconditionals, ready for the cloud-docs stub):learning-objective-N:attributes are defined outside thesingle-sourcetag, but the{learning-objective-N}references are inside it. The cloud-docs stub header must define the same three attributes or the cloud build will render the references literally.Checks
🤖 Generated with Claude Code
Preview pages