Skip to content

Implement external integration dispatch requests on EF Core - #5765

Draft
abparticular wants to merge 1 commit into
masterfrom
ef-external-integration
Draft

Implement external integration dispatch requests on EF Core#5765
abparticular wants to merge 1 commit into
masterfrom
ef-external-integration

Conversation

@abparticular

Copy link
Copy Markdown
Contributor

Implements IExternalIntegrationRequestsDataStore for the SQL Server and PostgreSQL persisters, replacing the no-op stub. This is ServiceControl's outbox for publishing NServiceBus events to external subscribers on domain events (message failed, custom check failed, etc.): StoreDispatchRequest persists a batch, a background loop wakes up, hands dispatched contexts to the subscribed callback, and deletes the rows once the callback succeeds.

Storage. New ExternalIntegrationDispatchRequests table (auto-increment Id for FIFO ordering). DispatchContext's concrete type is only known at runtime and lives in ServiceControl.csproj, which the EF Core project doesn't reference, so there's no way to type this at compile time - a new DispatchContextSerializer stores it as a versionless assembly-qualified type name alongside a System.Text.Json blob, and resolves it back via Type.GetType(...) on read. New migrations for both SQL Server and PostgreSQL.

Notification. SQL has no equivalent of RavenDB's server-push Changes API, so the dispatch loop is woken by an in-process signal (SemaphoreSlim) on every StoreDispatchRequest, with a 30s PeriodicTimer as a safety net in case a signal is ever missed. This assumes single-process-per-role deployment, same as every other EF Core data store.

Batch dispatch. RavenDB does select → callback → delete inside one document session. There's no relational equivalent, so this is three separate phases, each its own DbContext scope (same shape as RetentionSweeper.Sweep), so no DB transaction is held open across the network call to publish on the bus. Delete only happens if the callback returns without throwing — matching RavenDB's actual behavior (verified by reading TryDispatchEventBatch: it has no try/catch around the callback, so a throw skips the deletes too). This is at-least-once with redelivery on callback failure, not at-most-once.

Settings. Added ExternalIntegrationsDispatchingBatchSize (default 100) to EFPersisterSettings, read from the same config key the RavenDB backend already exposes, so switching backends doesn't silently change behavior.

Testing. New shared cross-backend tests (ExternalIntegrationRequestsDataStoreTests, run against RavenDB/SqlServer/PostgreSql) covering store→dispatch→delete, callback-throws-then-retries, batching over ExternalIntegrationsDispatchingBatchSize, and concurrent stores. EF-only tests (ExternalIntegrationRequestsDataStoreEFTests, DispatchContextSerializerTests) exercise the new DispatchNow test hook directly rather than polling on timers, plus a serializer round-trip test.

@abparticular
abparticular force-pushed the ef-external-integration branch from 5e4a7ae to 0bc18fd Compare August 13, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant