Skip to content

Add Spectrum troubleshooting docs for top support case drivers - #30683

Merged
steve-cloudflare merged 10 commits into
cloudflare:productionfrom
stechedo:spectrum-troubleshooting-docs
Jul 20, 2026
Merged

steve-cloudflare merged 10 commits into
cloudflare:productionfrom
stechedo:spectrum-troubleshooting-docs

Conversation

@stechedo

@stechedo stechedo commented May 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a new Spectrum Troubleshooting page and improves existing Spectrum documentation to address the top 3 case-generating issues identified in the April 2026 Network Support-by-Request (SBR) case review (~190 cases analyzed).

Estimated case deflection: 9-13 cases/month across these 3 issues.

Changes

New page

  • /spectrum/reference/troubleshooting/ — New troubleshooting page with structured symptom/cause/solution sections covering:
    • Health checks failing when IP Access Rules are enabled
    • DNS record conflict when creating Spectrum apps (same hostname as proxied record)
    • Protocol mismatch (origin receives HTTP instead of HTTPS)
    • Error 525 (SSL handshake failed) in Spectrum context
    • Common Spectrum event log status code patterns (444, 445, 497-499, 521, 522)

Modified pages (5)

  • /spectrum/reference/configuration-options/ — Added warnings about IP Access Rules blocking health checks, Custom Rules not working with Spectrum, protocol upgrade not being supported, and double-proxy chains causing 525 errors. Expanded intro paragraph to clarify Spectrum's L4 pass-through behavior.
  • /spectrum/reference/limitations/ — Added new "IP access control" section and "Using the same hostname for HTTP proxy and Spectrum" section documenting the platform limitation and recommending split-hostname architecture (with Citrix Gateway use case).
  • /spectrum/reference/logs/ — Added note clarifying that Spectrum status codes are distinct from HTTP status codes, with cross-link to troubleshooting page.
  • /load-balancing/additional-options/spectrum/ — Added caution callout in the Monitors setup step warning about IP Access Rules blocking health checks.
  • /dns/manage-dns-records/troubleshooting/records-with-same-name/ — Added note explaining that DNS record conflicts also apply to Spectrum applications, with cross-link to Spectrum limitations.

Motivation

These documentation gaps were identified by reviewing all April 2026 Network SBR cases. The top case-generating patterns were:

  1. IP Access Rules + Healthchecks (~3-5 cases/month): Customers enable IP Access Rules, health checks fail silently. No docs explain the interaction.
  2. Same Hostname WAF + Spectrum (~3-4 cases/month): Customers hit "DNS record already exists" error with no explanation that it's a hard platform limitation. Common in Citrix/VDI deployments.
  3. Protocol Mismatch / TLS errors (~3-4 cases/month): Customers expect Spectrum to upgrade HTTP→HTTPS or chain through another Cloudflare proxy. Neither is supported but neither was documented.

Content area

Spectrum, Load Balancing, DNS

@stechedo
stechedo requested a review from dcpena as a code owner May 7, 2026 16:58
@stechedo
stechedo requested a review from a team May 7, 2026 16:58
@stechedo
stechedo requested a review from RebeccaTamachiro as a code owner May 7, 2026 16:58
@github-actions github-actions Bot added product:load-balancing Related to Load Balancing product product:spectrum Related to Spectrum product product:dns Issues or PRs related to DNS product:network size/m labels May 7, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hey there, we've marked this pull request as stale because there's no recent activity on it. This label helps us identify PRs that might need updates (or to be closed out by our team if no longer relevant).

@cloudflare-docs-bot

cloudflare-docs-bot Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Review

⚠️ 1 warning, 💡 4 suggestions found in commit 88a2323.

Fix in your agent
Fix the following review findings in PR #30683 (https://github.com/cloudflare/cloudflare-docs/pull/30683).

Before making changes, review each finding and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g. the fix requires disproportionate effort for minimal benefit,
  or the finding is factually incorrect), explain why
- If you need clarification before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order

After triaging, fix all legitimate findings. For any you decide to skip,
post a comment on this PR with the finding ID and your reasoning.

---

## Code Review

### Suggestions (1)

#### CR-66981dfca718 · Reference link does not match content
- **File:** `src/content/docs/spectrum/reference/troubleshooting.mdx` line 63
- **Issue:** Line 63 links to [Spectrum Limitations](/spectrum/reference/limitations/) for more details on the DNS-record conflict limitation, but the limitations page does not mention this conflict.
- **Fix:** Add the DNS-record conflict limitation to /spectrum/reference/limitations/ or change the link so it points to a page that actually covers the topic.

---

## Style Guide Review

### Warnings (1)

#### SG-7a0c3d2323f0 · Use turn on / turn off for toggles
- **File:** `src/content/docs/spectrum/reference/configuration-options.mdx` line 123
- **Issue:** Added line instructs users to "enable Edge TLS Termination" for a UI toggle.
- **Fix:** Replace "enable" with "turn on" (e.g., "turn on Edge TLS Termination").

### Suggestions (3)

#### SG-904e62b2f26d · Bullet list has fewer than three items
- **File:** `src/content/docs/spectrum/reference/troubleshooting.mdx` line 92
- **Issue:** The Symptoms section begins a bulleted list with only two items
- **Fix:** Combine the two symptom sentences into prose or expand with more context

#### SG-35673bbbbcb6 · Bullet list has fewer than three items
- **File:** `src/content/docs/spectrum/reference/troubleshooting.mdx` line 81
- **Issue:** The Solution explains Full and Full (Strict) as a two-item bulleted list
- **Fix:** Write the two Edge TLS Termination options as prose instead of bullets

#### SG-1988155d8519 · Avoid multiple same-type admonitions in the same section
- **File:** `src/content/docs/spectrum/reference/configuration-options.mdx` line 122
- **Issue:** Patch adds `:::caution[Spectrum does not perform protocol upgrade]` and a second `:::caution` marker in the Edge TLS Termination section.
- **Fix:** Consolidate the two caution blocks or integrate one warning into the surrounding prose.

Code Review

This code review is in beta and may not always be helpful — use your judgment.

Suggestions (1)
File Issue
spectrum/reference/troubleshooting.mdx line 63 Reference link does not match content — Line 63 links to Spectrum Limitations for more details on the DNS-record conflict limitation, but the limitations page does not mention this conflict. Fix: Add the DNS-record conflict limitation to /spectrum/reference/limitations/ or change the link so it points to a page that actually covers the topic.

Conventions

No convention issues found.

Style Guide Review

Warnings (1)
File Issue
spectrum/reference/configuration-options.mdx line 123 Use turn on / turn off for toggles — Added line instructs users to "enable Edge TLS Termination" for a UI toggle. Fix: Replace "enable" with "turn on" (e.g., "turn on Edge TLS Termination").
Suggestions (3)
File Issue
spectrum/reference/troubleshooting.mdx line 92 Bullet list has fewer than three items — The Symptoms section begins a bulleted list with only two items Fix: Combine the two symptom sentences into prose or expand with more context
spectrum/reference/troubleshooting.mdx line 81 Bullet list has fewer than three items — The Solution explains Full and Full (Strict) as a two-item bulleted list Fix: Write the two Edge TLS Termination options as prose instead of bullets
spectrum/reference/configuration-options.mdx line 122 Avoid multiple same-type admonitions in the same section — Patch adds :::caution[Spectrum does not perform protocol upgrade] and a second :::caution marker in the Edge TLS Termination section. Fix: Consolidate the two caution blocks or integrate one warning into the surrounding prose.
Commands

Only codeowners can run commands. Post a comment with the command to trigger it.

Command Description
/review Runs a review now. Incremental if a prior review exists, full if not.
/full-review Re-reviews the entire PR diff from scratch, ignoring incremental history. Useful after a rebase, when you want a fresh review, or if the bot gets out of sync and reports issues that no longer exist.
/ignore-review-limit Permanently lifts the 2-review automatic limit for this PR. Future pushes will trigger reviews as normal.
/disable-auto-review Stops automatic reviews from triggering on future pushes to this PR. Codeowners can still run /review or /full-review manually.
/rebase Rebases the PR branch against production. Stops if there are conflicts and reports which files conflict.
/rebaseWithConflicts Rebases against production and attempts to resolve conflicts automatically using AI. Stops with an explanation if confidence is not high enough.

@stechedo
stechedo force-pushed the spectrum-troubleshooting-docs branch from f311329 to 4e5ee85 Compare June 26, 2026 19:10
@stechedo
stechedo force-pushed the spectrum-troubleshooting-docs branch from ea9679c to ad01610 Compare July 14, 2026 20:11
@msimoncf

Copy link
Copy Markdown

Thanks Stephen, have reviewed.

Could we just hold the IP Access Rules + health checks section? I need a bit more time to verify the mechanism, but I don't want to block everything else.

For the rest:

1. Same-hostname clarification
Files: spectrum/reference/troubleshooting.mdx, dns/manage-dns-records/troubleshooting/records-with-same-name.mdx
The limitation is specifically about manually-created proxied DNS records colliding with Spectrum-managed records. Multiple Spectrum apps can share the same hostname, including HTTP/HTTPS + TCP/UDP apps, because Spectrum manages the DNS records.
Could we update the wording to something like:
Cloudflare does not support having both a manually-created proxied DNS record (A, AAAA, or CNAME) and a Spectrum application on the same hostname. This is because Spectrum provisions and manages its own DNS record for the application. Multiple Spectrum applications can share the same hostname.
And note that the split-hostname workaround is only needed when a manually-created proxied record must be kept.

(opencode picked up the below)
2. error 525 inconsistency
File: spectrum/reference/configuration-options.mdx
The troubleshooting page says TCP apps log 521/522 for origin TLS failures, while 525 is for HTTP/HTTPS apps. The caution in configuration-options.mdx currently says TCP double-proxy may result in 525, so the two pages contradict each other.
Could we align it to something like:
For TCP applications, a failed TLS handshake to the origin is reported as an origin connection failure (521 or 522). For HTTP/HTTPS applications, clients may receive error 525 (SSL handshake failed).

Thanks!

stechedo added a commit to stechedo/cloudflare-docs that referenced this pull request Jul 17, 2026
…split health checks

Per review feedback on cloudflare#30683:

- Move the 'Health checks fail when IP Access Rules are enabled'
  section and related callouts to a separate PR (cloudflare#32137) so this PR
  isn't blocked while the mechanism is verified.
- Clarify that the DNS record conflict is specifically between a
  manually-created proxied record and a Spectrum application, not
  between Spectrum applications themselves. Multiple Spectrum
  applications (including a mix of HTTP/HTTPS and TCP/UDP) can share
  a hostname because Spectrum manages the DNS record for each.
- Align the double-proxy caution in configuration-options.mdx with
  the troubleshooting page: TCP applications surface a failed origin
  TLS handshake as 521/522, not error 525, which is HTTP/HTTPS-app
  specific.
@stechedo

Copy link
Copy Markdown
Collaborator Author

Thanks for the review — pushed changes addressing both items, plus split out the held section.

1. Same-hostname clarification

Updated spectrum/reference/troubleshooting.mdx (Cannot create Spectrum application section) and dns/manage-dns-records/troubleshooting/records-with-same-name.mdx to make clear the conflict is specifically between a manually-created proxied record and a Spectrum application (Spectrum provisions/manages its own record), and that multiple Spectrum applications — HTTP/HTTPS and TCP/UDP alike — can share a hostname. The split-hostname workaround is now framed as only needed when you must keep a manually-created proxied record on the hostname.

2. Error 525 inconsistency

Aligned the double-proxy caution in spectrum/reference/configuration-options.mdx with the troubleshooting page: TCP applications surface a failed origin TLS handshake as 521/522, not 525 — 525 is HTTP/HTTPS-application-specific.

Held section moved to a new PR

Split the "Health checks fail when IP Access Rules are enabled" content (troubleshooting.mdx section, the two callouts in configuration-options.mdx, and the Monitors-step callout in load-balancing/spectrum.mdx) out to #32137, so it doesn't block the rest of this PR while the mechanism is verified.

Re: the automated code review findings posted above

Those findings (CR-9746e1fd141a, CR-b72d87deb669, CR-a65044b3f463, CR-a49b1d2042d9, CR-205ecfa2f49f, CR-790138ed0478, CR-86f40bf1274b, CR-45351e1f8985, CR-ea45836bc4bb, CR-8c6d3f362c23, CR-5da00f3a901d, CR-52aaf6ceb860, CR-c0ce4fd12e7d) all reference files that are not part of this PR's diff (src/components/AiSearchIndexingDiagram.astro, .flue/workflows/finalize-review.ts, .flue/lib/*, src/content/catalog-models/anthropic-claude-*.json, etc. — this PR only touches 5 Spectrum/DNS docs pages). Skipping all of them as out of scope for this PR; flagging in case the review got attached to the wrong PR.

@stechedo
stechedo force-pushed the spectrum-troubleshooting-docs branch from 426e94b to effc1d7 Compare July 17, 2026 14:03
stechedo added a commit to stechedo/cloudflare-docs that referenced this pull request Jul 17, 2026
…split health checks

Per review feedback on cloudflare#30683:

- Move the 'Health checks fail when IP Access Rules are enabled'
  section and related callouts to a separate PR (cloudflare#32137) so this PR
  isn't blocked while the mechanism is verified.
- Clarify that the DNS record conflict is specifically between a
  manually-created proxied record and a Spectrum application, not
  between Spectrum applications themselves. Multiple Spectrum
  applications (including a mix of HTTP/HTTPS and TCP/UDP) can share
  a hostname because Spectrum manages the DNS record for each.
- Align the double-proxy caution in configuration-options.mdx with
  the troubleshooting page: TCP applications surface a failed origin
  TLS handshake as 521/522, not error 525, which is HTTP/HTTPS-app
  specific.
@steve-cloudflare
steve-cloudflare enabled auto-merge (squash) July 17, 2026 14:40
@github-actions github-actions Bot removed the stale label Jul 18, 2026

@hannes-cf hannes-cf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small style/structure suggestions on records-with-same-name.mdx:

1. Move the Spectrum case into the existing ## Causes bullet list — the page already has one :::note under ## Solution, so a second note reads redundant. Listing Spectrum alongside the other two record-conflict scenarios also puts it exactly where readers scan first.

GitHub doesn't let me attach a one-click suggestion to line 25 (it's outside this PR's diff hunk), so please paste this as a third bullet in the Causes list, right after the existing A/AAAA bullet:

- Create a [Spectrum](/spectrum/) application for a name that already has a manually-created `A`, `AAAA`, or `CNAME` record. Spectrum provisions and manages its own DNS record for the application, so Cloudflare does not support having both on the same name. Multiple Spectrum applications can, however, share the same name. Refer to [Spectrum Troubleshooting](/spectrum/reference/troubleshooting/#cannot-create-spectrum-application--dns-record-already-exists) for recommended workarounds.

2. Remove the trailing :::note[Spectrum applications] block — see the inline suggestion below. The empty ```suggestion block deletes those 6 lines in one click. This also drops the titled-note syntax (Cloudflare docs style is plain :::note, no [Title]).

stechedo and others added 10 commits July 20, 2026 08:33
…se drivers

Based on analysis of April 2026 Network SBR cases, add documentation
to address the top 3 case-generating Spectrum issues:

1. IP Access Rules + Health Checks interaction: Customers enable IP
   Access Rules on Spectrum apps, then Cloudflare health checks fail
   because probe IPs are blocked. Added warnings to configuration-options
   and load-balancing/spectrum pages.

2. Same hostname for WAF + Spectrum: Customers attempt to use the same
   hostname for HTTP (WAF) and TCP (Spectrum) on different ports. This
   is a platform limitation. Added new section to limitations page and
   cross-reference on DNS troubleshooting page.

3. Edge TLS / Protocol mismatch: Customers configure HTTP edge ports
   expecting HTTPS to origin. Spectrum does not upgrade protocols.
   Added warnings to Edge TLS Termination section.

Also:
- Created new /spectrum/reference/troubleshooting/ page with structured
  symptom/cause/solution sections for all three issues plus error 525
  and common status code patterns
- Added note to /spectrum/reference/logs/ clarifying that Spectrum
  status codes are distinct from HTTP status codes
Fix style guide violations flagged by cloudflare-docs-bot:
- Use 'turn on' instead of 'enable' for toggles
- Use bullet points for unordered causes, numbered list for sequential steps
- Remove -ing verb from heading ('Understanding common...' -> 'Common...')
- Replace 'the table below' with direct reference
- Replace 'etc.' with 'and similar'
- Monospace DNS record types (A, AAAA, CNAME)
- Add introductory sentence before status code table
- Convert two-item symptom list to prose
- Fix TLS mode guidance to specify Full/Full (Strict) instead of generic 'enabled'
- Fix broken cross-links to removed limitations.mdx anchors
- Reframe error 525 section around TLS handshake failures with
  Spectrum event log code 540, clarify HTTP 525 applies only to
  HTTP/HTTPS-type applications
- Update DNS records-with-same-name cross-link to point to the
  troubleshooting page (which documents the workaround) instead
  of the limitations page (which does not)
- Monospace status code examples (444, 499) in logs.mdx
- Update anchor in configuration-options.mdx to match renamed heading
…ection

Replace 540 (client/edge TLS failure) with origin-side error codes
520/521/522 which correctly describe edge-to-origin connection failures.
540 refers to the client-to-edge handshake per the Spectrum logs
reference, not edge-to-origin.
Separate TCP and HTTP/HTTPS symptoms: Edge TLS Termination applies
to TCP apps (origin errors 521/522), while error 525 applies to
HTTP/HTTPS apps. Avoids conflating transport-level connection codes
with TLS handshake failures.
Spectrum has no dedicated origin-side TLS handshake status code.
A failed TLS handshake at the origin is reported as an origin
connection failure (521/522). Added explanation and cross-link
to the Event logs reference.
…split health checks

Per review feedback on cloudflare#30683:

- Move the 'Health checks fail when IP Access Rules are enabled'
  section and related callouts to a separate PR (cloudflare#32137) so this PR
  isn't blocked while the mechanism is verified.
- Clarify that the DNS record conflict is specifically between a
  manually-created proxied record and a Spectrum application, not
  between Spectrum applications themselves. Multiple Spectrum
  applications (including a mix of HTTP/HTTPS and TCP/UDP) can share
  a hostname because Spectrum manages the DNS record for each.
- Align the double-proxy caution in configuration-options.mdx with
  the troubleshooting page: TCP applications surface a failed origin
  TLS handshake as 521/522, not error 525, which is HTTP/HTTPS-app
  specific.
…s-with-same-name.mdx

Co-authored-by: Hannes <105781579+hannes-cf@users.noreply.github.com>
…s-with-same-name.mdx

Co-authored-by: Hannes <105781579+hannes-cf@users.noreply.github.com>
@stechedo
stechedo force-pushed the spectrum-troubleshooting-docs branch from 27816ff to 88a2323 Compare July 20, 2026 12:33
@hannes-cf
hannes-cf self-requested a review July 20, 2026 14:10
@steve-cloudflare
steve-cloudflare merged commit c92789c into cloudflare:production Jul 20, 2026
10 checks passed
@github-actions
github-actions Bot deleted the spectrum-troubleshooting-docs branch September 19, 2026 08:41
stechedo added a commit to stechedo/cloudflare-docs that referenced this pull request Oct 15, 2026
Resolve conflict in spectrum/reference/troubleshooting.mdx by keeping
both sides: the 'Health checks fail when IP Access Rules are enabled'
section from this PR (in its original position after the virtual
network origins content, matching the pre-split ordering) followed by
the case-driven sections added to production via cloudflare#30683 (DNS record
conflict, protocol mismatch, error 525, status codes).
jeffh-cloudflare pushed a commit to jeffh-cloudflare/cloudflare-docs that referenced this pull request Oct 15, 2026
…e#32137)

* [Spectrum] Document IP Access Rules blocking health checks

Split out from cloudflare#30683 per reviewer request to allow more time to
verify the allowlist/health-check mechanism.

- Add troubleshooting section for health checks failing when IP
  Access Rules use an allowlist model
- Add caution + note callouts in configuration-options.mdx
- Add caution callout in the Load Balancing Spectrum setup guide

* [Spectrum] Apply review feedback on IP Access Rules + health checks

Per review feedback on this PR:

- Troubleshooting: add an 'Applies to' section scoping the issue to
  Spectrum apps targeted by a standalone Health Check or LB monitor
  (not apps whose origin is an LB hostname), clarify that switching
  HTTP to TCP health checks does not bypass IP Access Rules, and
  remove the misleading TCP-check-type alternative.
- Load Balancing setup guide: scope the caution callout to Spectrum
  apps used as LB endpoints and exclude origin-is-LB-hostname setups.
- Configuration options: reword the caution so the limitation is not
  overstated; Custom Rules note unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

product:dns Issues or PRs related to DNS product:load-balancing Related to Load Balancing product product:network product:spectrum Related to Spectrum product size/m

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants