diff --git a/java/outbox.md b/java/outbox.md index 0cba54d407..508083c561 100644 --- a/java/outbox.md +++ b/java/outbox.md @@ -34,56 +34,33 @@ Once the transaction succeeds, the messages are read from the database table and - If an emit was successful, the respective message is deleted from the database table. - If an emit wasn't successful, there will be a retry after some (exponentially growing) waiting time. After a maximum number of attempts, the message is ignored for processing and remains in the database table. Even if the app crashes the messages can be redelivered after successful application startup. -To enable the persistence for the outbox, you need to add the service `outbox` of kind `persistent-outbox` to the `cds.requires` section in the _package.json_ or _cdsrc.json_, which will automatically enhance your CDS model in order to support the persistent outbox. - -```jsonc -{ - // ... - "cds": { - "requires": { - "outbox": { - "kind": "persistent-outbox" - } - } - } -} -``` - -::: warning -Be aware that you need to migrate the database schemas of all tenants after you've enhanced your model with an outbox version from `@sap/cds` version 6.0.0 or later. -::: -For a multitenancy scenario, make sure that the required configuration is also done in the MTX sidecar service. Make sure that the base model in all tenants is updated to activate the outbox. +CAP Java provides the persistent outbox service `DefaultOutboxUnordered` by default. It's used by the [AuditLog service](../java/auditlog) and registered as the primary Spring bean for `OutboxService`. You can inject it directly without a qualifier: -::: info Option: Add outbox to your base model -Alternatively, you can add `using from '@sap/cds/srv/outbox';` to your base model. In this case, you need to update the tenant models after deployment but you don't need to update MTX Sidecar. -::: - -If enabled, CAP Java provides two persistent outbox services by default: - -- `DefaultOutboxOrdered` - is used by default by [messaging services](../java/messaging) -- `DefaultOutboxUnordered` - is used by default by the [AuditLog service](../java/auditlog) +```java +@Autowired +private OutboxService outboxService; +``` -The default configuration for both outboxes can be overridden using the `cds.outbox.services` section, for example in the _application.yaml_: +The default configuration can be overridden using the `cds.outbox.services` section, for example in the _application.yaml_: ::: code-group ```yaml [srv/src/main/resources/application.yaml] cds: outbox: services: - DefaultOutboxOrdered: - maxAttempts: 10 - # ordered: true DefaultOutboxUnordered: maxAttempts: 10 - # ordered: false ``` ::: You have the following configuration options: - `maxAttempts` (default `10`): The number of unsuccessful emits until the message is ignored. It still remains in the database table. -- `ordered` (default `true`): If this flag is enabled, the outbox instance processes the entries in the order they have been submitted to it. Otherwise, the outbox may process entries randomly and in parallel, by leveraging outbox processors running in multiple application instances. This option can't be changed for the default persistent outboxes. The persistent outbox stores the last error that occurred, when trying to emit the message of an entry. The error is stored in the element `lastError` of the entity `cds.outbox.Messages`. +::: info +Additionally, CAP Java creates a `DefaultOutboxOrdered` outbox, which is used by [messaging services](../java/messaging). It can be configured similarly via `cds.outbox.services.DefaultOutboxOrdered`. +::: + ### Configuring Custom Outboxes { #custom-outboxes} Custom persistent outboxes can be configured using the `cds.outbox.services` section, for example in the _application.yaml_: @@ -415,6 +392,417 @@ void handleAuditLogProcessingErrors(OutboxMessageEventContext context) { [Learn more about `EventContext.proceed()`.](./event-handlers/#proceed-on){.learn-more} + +## Outbox Task Scheduling + +CAP Java provides an outbox-based task scheduling mechanism that allows services to emit events on a defined schedule. This mechanism enables recurring jobs, delayed execution, and cron-based task automation — all built on top of the existing outbox infrastructure. + +### Schedule API + +**Package:** `com.sap.cds.services.outbox` +**Class:** `Schedule` + +The `Schedule` class defines the timing configuration for an outbox task. It uses a fluent builder pattern. + +#### Creating a Schedule + +```java + +// Immediate execution (default) +Schedule now = Schedule.NOW; + +// Delayed execution — run once after 30 seconds +Schedule delayed = Schedule.create() + .after(Duration.ofSeconds(30)); + +// Recurring with fixed delay — every 5 minutes, starting immediately +Schedule recurring = Schedule.create() + .every(Duration.ofMinutes(5)); + +// Recurring with initial delay — first run after 10s, then every 5 minutes +Schedule delayedRecurring = Schedule.create() + .after(Duration.ofSeconds(10)) + .every(Duration.ofMinutes(5)); + +// Cron-based — every weekday at 8:00 AM +Schedule cronBased = Schedule.create() + .cron("0 0 8 * * MON-FRI"); +``` + +#### Properties + +| Method | Description | Default | +|--------|-------------|---------| +| `as(String)` | Explicitly names the task, making it a **singleton** with **upsert** semantics (see [Named Tasks](#named-singleton-tasks)) | Event name (for scheduled tasks) | +| `after(Duration)` | Initial delay before first execution | `Duration.ZERO` | +| `every(Duration)` | Delay between recurring executions (after each successful run) | None (single execution) | +| `cron(String)` | Spring Cron Expression for recurring execution | None | +| `cancel()` | Marks the named task for cancellation | `false` | + +#### Constraints + +- **`cron`** is mutually exclusive with `after` and `every`. Combining them throws `IllegalArgumentException`. +- **`every`** determines the delay *after* a successful execution, not a fixed-rate interval. + + +#### Named (Singleton) Tasks + +All scheduled tasks (using any `Schedule` other than `Schedule.NOW`) are **singletons**. The task name determines the outbox message ID, ensuring only one active instance per name exists at any time. + +- If an explicit name is set via `.as(...)`, it's used as the task name. +- If no explicit name is set, the **event name** is used as the task name. + +Re-submitting a task with the same name **replaces** the existing entry entirely — the schedule, message content, and execution timestamp are all updated to reflect the latest submission. Replacement follows **last-write-wins** semantics. + +```java +// Only one "daily-cleanup" task will exist, regardless of how often this code runs +Schedule cleanup = Schedule.create() + .as("daily-cleanup") + .cron("0 0 2 * * *"); // daily at 2 AM +``` + +```java +// Initial submission: run every hour +Schedule hourly = Schedule.create() + .as("sync-job") + .every(Duration.ofHours(1)); +outboxService.submit("sync/trigger", message0, hourly); + +// Later: change to every 30 minutes — replaces the existing "sync-job" +Schedule every30Min = Schedule.create() + .as("sync-job") + .every(Duration.ofMinutes(30)); +outboxService.submit("sync/trigger", message1, every30Min); + +// Result: only ONE task exists with the 30-minute schedule and message2 content +``` + +> **Important:** Both the schedule *and* the message payload are replaced. If you only want to update the timing, you must still provide the full message content. + +The upsert mechanism is safe for concurrent submissions. If a named task is re-submitted while it's currently being processed, the new submission is preserved and will be executed according to the updated schedule after the current execution completes. + +::: warning +If you need multiple independent tasks for the same event (for example, per-user reminders), you **must** set an explicit name via `.as(...)` to distinguish them. Without it, all submissions for the same event share one task name and re-submissions replace the existing task. +::: + + + +#### Canceling a Scheduled Task + +Named tasks can be canceled: + +```java +Schedule cancelCleanup = Schedule.create() + .as("daily-cleanup") + .cancel(); + +// Submit the cancellation +outboxService.submit("maintenance/cleanup", null, cancelCleanup); +``` + +If no explicit name is set via `.as(...)`, the event name is used to identify the task to cancel: + +```java +Schedule cancel = Schedule.create() + .cancel(); + +// Cancels the task identified by the event name "sync/trigger" +outboxService.submit("sync/trigger", null, cancel); +``` + +#### Cancellation Behavior + +When a cancellation is submitted, the named task is **deleted** from the outbox so that no future executions occur. However, a **currently running execution will complete** — cancellation doesn't interrupt in-flight processing. + +**Key details:** + +| Aspect | Behavior | +|--------|----------| +| Future executions | Prevented — task is removed from the schedule | +| Currently running execution | **Completes** — not interrupted | +| At most one additional execution | Possible if the task was already picked up for processing | +| Canceling a non-existent task | Silent no-op (no error thrown) | +| Cancellation without explicit name | Cancels the task identified by the event name | + + +#### Cron Expression Syntax + +The cron expression follows the [Spring Cron Expression](https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronExpression.html) format with **6 fields**: + +``` +┌───────── second (0-59) +│ ┌───────── minute (0-59) +│ │ ┌───────── hour (0-23) +│ │ │ ┌───────── day of month (1-31) +│ │ │ │ ┌───────── month (1-12 or JAN-DEC) +│ │ │ │ │ ┌───────── day of week (0-7 or MON-SUN, 0 and 7 = Sunday) +│ │ │ │ │ │ +* * * * * * +``` + +**Examples:** + +| Expression | Description | +|-----------|-------------| +| `0 0 * * * *` | Every hour | +| `0 */15 * * * *` | Every 15 minutes | +| `0 0 8 * * MON-FRI` | Weekdays at 8:00 AM | +| `0 0 2 * * *` | Daily at 2:00 AM | +| `0 0 0 1 * *` | First day of every month at midnight | + + +**Restrictions:** + +- **`cron` cannot be used** with `after` and `every`. Setting `cron` when `after`/`every` is already defined throws `IllegalArgumentException`. +- **`every` without `after`** starts the first execution immediately (with zero delay), then applies `every` between subsequent executions. +- **Cron expressions that never match** (for example, February 30th) are silently deleted — the task is marked as completed without ever executing. +- **All times are evaluated in UTC.** + + +### Schedulable API + +**Package:** `com.sap.cds.services.outbox` +**Interface:** `Schedulable` + +The `Schedulable` interface provides the **logical CDS service layer** for scheduling. It wraps any CDS `Service` so that all events emitted to it are automatically scheduled via the outbox. + +### Core Concept + +When you call `OutboxService.outboxed(service)`, the returned proxy **always** implements `Schedulable`. This means you can: + +1. Use the outboxed service for immediate async execution (default outbox behavior) +2. Cast to `Schedulable` and call `.scheduled(schedule)` to get a service proxy whose events are scheduled + +### Creating a Schedulable Service + +```java + +// Option 1: Using the static factory method +Schedulable schedulable = Schedulable.of(messagingService, outboxService); +MessagingService scheduled = schedulable.scheduled( + Schedule.create().every(Duration.ofMinutes(5)) +); + +// Option 2: Direct cast from outboxed service +MessagingService outboxed = outboxService.outboxed(messagingService); +Schedulable schedulable = (Schedulable) outboxed; +MessagingService scheduled = schedulable.scheduled( + Schedule.create().cron("0 0 */2 * * *") // every 2 hours +); +``` + +### Using a Scheduled Service + +Once you have a scheduled service instance, use it exactly like the original service. All emitted events are stored in the outbox with the configured schedule: + +```java +// All events emitted to 'scheduled' will follow the defined schedule +scheduled.emit("myTopic", messageData); +``` + +--- + +### End-to-End Examples + +#### Example 1: Recurring Data Sync Every 10 Minutes + +```java +@Autowired +private OutboxService outboxService; + +@Autowired +private MessagingService messagingService; + +public void setupRecurringSync() { + Schedule every10Min = Schedule.create() + .as("data-sync") + .every(Duration.ofMinutes(10)); + + MessagingService scheduled = Schedulable.of(messagingService, outboxService) + .scheduled(every10Min); + + // This event will be emitted every 10 minutes + scheduled.emit("sync/trigger", Map.of("source", "system-a")); +} +``` + +#### Example 2: Delayed One-Time Notification + +```java +public void scheduleReminder(String userId) { + Schedule in24Hours = Schedule.create() + .as("reminder-" + userId) // explicit name ensures one task per user + .after(Duration.ofHours(24)); + + MessagingService scheduled = Schedulable.of(messagingService, outboxService) + .scheduled(in24Hours); + + // Will be emitted once, 24 hours from now + scheduled.emit("notifications/reminder", Map.of("userId", userId)); +} +``` + +#### Example 3: Cron-Based Daily Report + +```java +public void setupDailyReport() { + Schedule dailyAt6AM = Schedule.create() + .as("daily-report") + .cron("0 0 6 * * *"); + + MessagingService scheduled = Schedulable.of(messagingService, outboxService) + .scheduled(dailyAt6AM); + + scheduled.emit("reports/daily", Map.of("type", "summary")); +} +``` + +#### Example 4: Canceling a Recurring Task + +```java +public void stopDailyReport() { + Schedule cancel = Schedule.create() + .as("daily-report") + .cancel(); + + // Submit the cancellation through the outbox + outboxService.submit("reports/daily", outboxMessage, cancel); +} +``` + +#### Example 5: Using the OutboxService Directly + +For lower-level control, you can submit messages with a schedule directly: + +```java +public void submitScheduledMessage() { + OutboxMessage message = createOutboxMessage(); + + Schedule schedule = Schedule.create() + .as("cleanup-job") + .after(Duration.ofMinutes(5)) + .every(Duration.ofHours(1)); + + // Submit directly to the outbox with scheduling + outboxService.submit("maintenance/cleanup", message, schedule); +} +``` + +--- + +### Execution Semantics + +#### Recurring Task Timing + +For `every`-based schedules, the next execution is calculated **after a successful execution**: + +``` +Time ──────────────────────────────────────────────────────► + +submit execute execute execute + │──after──►│──── every ───►│──── every ───►│ + t₀ t₁ t₂ t₃ +``` + +For `cron`-based schedules, the next execution time is determined by evaluating the cron expression after the last successful execution. + +#### Singleton Behavior + +All scheduled tasks are singletons — each task has a name (explicit or derived from the event) and only one active instance per name can exist at a time. Re-submitting a task with the same name **replaces** the existing entry. This Behavior is ideal for recurring background jobs that should not overlap. + +#### Outbox Guarantees + +Scheduled tasks inherit the standard outbox guarantees: +- **At-least-once delivery** — tasks are retried on failure +- **Transactional** — task submission is part of the current transaction (persistent outbox) +- **Tenant-aware** — tasks execute in the context of the tenant that created them + +--- + + +## Outbox Collector Strategies { #outbox-collector-strategies} + +In a multitenant environment, outbox entries reside in tenant-specific persistences. The outbox collector is triggered when events are submitted to the outbox. However, if an application instance crashes, unprocessed outbox entries for a tenant are only retried when that tenant next produces a new outbox event. If after a crash, a tenant becomes inactive or has a long period until its next outbox submission, the remaining entries stay unprocessed until the tenant triggers a new event. + +To address this, CAP Java provides two scheduler-based strategies that periodically check tenant outboxes for unprocessed entries. Both strategies are disabled by default and must be explicitly enabled. + +### Hot-Tenant Task { #hot-tenant-task} + +Instead of iterating over all tenants, the hot-tenant task tracks which tenants have been recently active and only triggers the outbox collector for those tenants. Lookups are well distributed over time to avoid activity jams. + +::: code-group +```yaml [srv/src/main/resources/application.yaml] +cds: + outbox: + persistent: + scheduler: + hotTenantTask: + enabled: true + maxTaskDelay: 2h +``` +::: + +The configuration options are: + +- `maxTaskDelay` (default `2h`): The maximum time to wait after a tenant event before checking that tenant's outbox. Lookups are distributed within this window to spread the load evenly. + +#### Persistence for Hot-Tenant Tracking + +The hot-tenant task manages tenant activity records centrally in the provider persistence. By default, the MTXs persistence (T0 tenant) is used. + +If the application uses a custom provider persistence bound, for instance, via an HDI binding, the property `cds.multiTenancy.provider.persistenceService` can reference the persistence service to use for the hot-tenant task: + +::: code-group +```yaml [srv/src/main/resources/application.yaml] +cds: + multiTenancy: + provider: + persistenceService: "my-custom-ps" +``` +::: + +::: warning +If you previously ran with the default MTXs/T0 persistence and switch to a custom provider persistence, the currently tracked hot tenants will be lost — there's no automatic migration. Plan accordingly before changing this configuration. +::: + + + +### All-Tenants Task { #all-tenants-task} + +The all-tenants task periodically iterates over **all** tenant outboxes and triggers the collector for each tenant. It acts as a safety net to ensure that no outbox entries are missed, regardless of tenant activity. + +::: code-group +```yaml [srv/src/main/resources/application.yaml] +cds: + outbox: + persistent: + scheduler: + enabled: true + allTenantsTask: + enabled: true + startDelay: 30s + interval: 2h + spreadTime: 15m +``` +::: + +The configuration options are: + +- `startDelay` (default `30s`): Delay after application startup before the first execution. +- `interval` (default `2h`): Interval between successive executions. +- `spreadTime` (default `15m`): The time span over which individual tenant checks are randomly distributed. This avoids a thundering-herd effect where all tenant outboxes are checked simultaneously. + +::: warning Performance consideration +For applications with a large number of tenants, traversing all tenants can cause significant overhead due to tenant context switches. This may impact application performance. Consider the [Hot-Tenant Task](#hot-tenant-task) as a lighter alternative that only checks recently active tenants. +::: + + +::: tip Prerequisite +Both strategies require the outbox scheduler to be enabled. By default, `cds.outbox.persistent.scheduler.enabled` is set to `true`. Set this property to false if you want to disable outbox scheduling. +::: + + + ## Outbox Dead Letter Queue The transactional outbox tries to process each entry a specific number of times. The number of attempts is configurable per outbox by setting the configuration `cds.outbox.services..maxAttempts`. @@ -425,7 +813,7 @@ Once the maximum number of attempts is exceeded, the corresponding entry is not ::: warning Changing configuration between deployments -It's possible to increase the value of the configuration `cds.outbox.services..maxAttempts` in between of deployments. Older entries which have reached their max attempts in the past would be retried automatically after deployment of the new microservice version. If the dead letter queue has a large size, this leads to unintended load on the system. +Both strategies require the outbox scheduler to be enabled. Scheduling is enabled by default `cds.outbox.persistent.scheduler.enabled=true`. Ensure that this property is not set to `false`, as disabling the scheduler prevents outbox messages scheduling. :::