Skip to content

Fix connection reset logic to ensure the old subscribe goroutine is cancelled before a new connection is made#1700

Draft
dhurley wants to merge 1 commit into
mainfrom
fix/connection-reset-operations-order
Draft

Fix connection reset logic to ensure the old subscribe goroutine is cancelled before a new connection is made#1700
dhurley wants to merge 1 commit into
mainfrom
fix/connection-reset-operations-order

Conversation

@dhurley
Copy link
Copy Markdown
Collaborator

@dhurley dhurley commented May 28, 2026

Proposed changes

  • A gRPC server that is tracking connections could remove a connection when the Subscribe stream is closed by the client and then track a connection again if it receives a CreateConnection RPC call. So we need to ensure that the old subscribe goroutine is cancelled before a new connection is made when a connection reset happens to ensure that the server is able to track connections.
  • Also I spotted a potential race condition in file service operator so I provided a fix for that as well.

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING document
  • I have run make install-tools and have attached any dependency changes to this pull request
  • If applicable, I have added tests that prove my fix is effective or that my feature works
  • If applicable, I have checked that any relevant tests pass after adding my changes
  • If applicable, I have updated any relevant documentation (README.md)
  • If applicable, I have tested my cross-platform changes on Ubuntu 22, Redhat 8, SUSE 15 and FreeBSD 13

… reset logic to ensure the old subscribe goroutine is cancelled before a new connection is made.
@github-actions github-actions Bot added bug Something isn't working chore Pull requests for routine tasks labels May 28, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 28, 2026

Codecov Report

❌ Patch coverage is 70.58824% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.01%. Comparing base (8dd9122) to head (4299490).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
internal/file/file_service_operator.go 64.70% 2 Missing and 4 partials ⚠️
internal/command/command_plugin.go 76.47% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1700   +/-   ##
=======================================
  Coverage   85.00%   85.01%           
=======================================
  Files         104      104           
  Lines       13579    13596   +17     
=======================================
+ Hits        11543    11558   +15     
- Misses       1522     1524    +2     
  Partials      514      514           
Files with missing lines Coverage Δ
internal/command/command_plugin.go 71.60% <76.47%> (+0.79%) ⬆️
internal/file/file_service_operator.go 77.47% <64.70%> (-0.07%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8dd9122...4299490. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

slog.InfoContext(ctxWithMetadata, "Starting new subscribe stream after connection reset")
subscribeCtx, cp.subscribeCancel = context.WithCancel(ctxWithMetadata)
go cp.commandService.Subscribe(subscribeCtx)
cp.subscribeWg.Add(1)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Are we missing a cp.subscribeWg.Wait() statement?


cp.conn = newConnection

// Wait for the old Subscribe goroutine to fully exit before creating a new one with the new connection. .
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: Trailing dot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working chore Pull requests for routine tasks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants