Skip to content

Update Store with Version Info#398

Closed
marcw-replicated wants to merge 3 commits intomainfrom
mw-version-update
Closed

Update Store with Version Info#398
marcw-replicated wants to merge 3 commits intomainfrom
mw-version-update

Conversation

@marcw-replicated
Copy link
Copy Markdown

What does this PR do?

Problem: In HA deployments (replicaCount > 1), SDK pods initialize version-related fields (channel sequence, version label, etc.) from the config file once at startup and never update them. If a pod survives a rollback or restarts at a different time than its peers, it can end up with a stale channel sequence in memory. Both pods then pass the canReport check and send heartbeats every 4 hours at slightly different times, causing the vendor backend to see the instance oscillating between two versions and firing repeated instance.upgraded notifications.

Fix: Added a Kubernetes informer (pkg/secretwatcher) that watches the replicated Secret for updates. When the Secret changes (e.g., after a Helm upgrade), all version-related fields (channelID, channelName, channelSequence, versionLabel, releaseSequence, releaseCreatedAt, releaseNotes) are updated in the in-memory store immediately. This ensures all running replicas converge on the correct version data without requiring a pod restart.

Changes:

  • Added Set* methods to the Store interface and InMemoryStore for all version-related fields
  • Added pkg/secretwatcher package implementing a Secret informer using ListWithContextFunc/WatchFuncWithContext
  • Wired the secret watcher into bootstrap.go, starting it before the heartbeat

Does this PR introduce a user-facing change?

NONE

@laverya
Copy link
Copy Markdown
Member

laverya commented Mar 20, 2026

This isn't needed due to the existing "include config digest in annotations, and report only if latest pod generation" pattern

@laverya laverya closed this Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants