Proposal: Temporal.io-backed `MessageQueue` adapter

[robin-maki]

Background and motivation

I'd like to contribute a Fedify MessageQueue adapter backed by Temporal.io, the durable execution platform.

In our project, byulmaru/kosmo, we already use Temporal, while Fedify uses a separate PgMessageQueue for message processing. Reusing our existing Temporal infrastructure could reduce the overhead of managing these separate queue mechanisms.

Proposed approach

I propose a separate adapter package implementing Fedify's MessageQueue interface, with message handlers executed as Temporal Activities.

The initial design would be:

• For messages without an ordering key, start a Workflow per message. The Workflow waits for any requested delay and then invokes the message-processing Activity.
• For ordered messages, use a Workflow per ordering key, addressed through Signal-With-Start. Signal handlers buffer messages, while the main Workflow loop processes eligible messages one at a time.
• Delayed messages become eligible after their delay and do not block later messages that are already eligible. Once an Activity starts, its retries hold up subsequent messages for the same key until it succeeds or reaches a terminal failure.
• Use Temporal's Activity retries and declare nativeRetrial so Fedify does not independently schedule retries for the same failure. Define a bounded retry policy and terminal-failure handling so one failed message cannot block a key indefinitely.
• Have listen() start a Temporal Worker with the supplied handler registered as an Activity, and initiate graceful worker shutdown when its abort signal fires.
• Use Continue-As-New to bound Workflow history, carrying pending messages forward, and allow idle ordering Workflows to complete.

This is an initial design, not yet validated by an implementation. I would like to confirm the delay and retry semantics against the intended MessageQueue contract.

Deduplication also needs verification: using a Workflow ID alone should not be assumed to match Fedify's deduplication window. I would verify that behavior before advertising native deduplication support. Optional queue depth reporting and atomic batch enqueue would be outside the initial scope.

The contribution would include documentation of supported runtimes and worker setup, along with integration tests covering delayed processing, retries, ordering, worker restart and shutdown, and pending-message preservation across Continue-As-New.

Questions for maintainers

Would you be open to including this adapter in the Fedify monorepo, or would you prefer it to start as a third-party package?

Does this approach fit the intended MessageQueue contract? I'd appreciate feedback on the proposed worker lifecycle and ordering model before implementing it.

AI assistance

ChatGPT (GPT-5.6 Sol) assisted with exploring the technical approach. Codex (GPT-6, with GPT-5.6 Luna assisting repository investigation) reviewed repository materials and drafted this issue.

I provided the use case from byulmaru/kosmo and reviewed the proposal's motivation, terminology, scope, and AI disclosure. The technical analysis and English draft were AI-assisted.

[dahlia]

Reusing Temporal in Kosmo makes sense as a use case. I haven't used Temporal myself, though, and I'm not yet convinced that MessageQueue is the right boundary for this integration.

As I understand the proposal, Temporal would take over much of the execution management around Fedify's message handler, including scheduling and retries. Having listen() create a Temporal Worker would also make the queue adapter responsible for that Worker's lifecycle and Workflow registration. I'd like to understand how this fits into an application that already manages its own Temporal Workers.

Fedify's MessageQueue does already include listen(handler) and support backend-managed retries, so this isn't necessarily outside its contract. My concern is whether those hooks are enough to make the integration work naturally. There are also places where execution responsibility would remain split: even with nativeRetrial: true, Fedify can re-enqueue outbox messages for Retry-After or circuit-breaker holds. Those paths would need to fit the proposed ordering model, since they don't remain within the same Activity retry cycle.

Could you share a small sketch of how you would configure and run this in Kosmo? In particular, it would help to see who creates and shuts down the Worker, how the Fedify handler is registered, and whether this uses a dedicated Task Queue or joins an existing Worker setup. An application-level example would help us judge the boundary before settling the adapter design.

I'm open to exploring a Temporal integration, but I'd like to resolve these questions before accepting it as a MessageQueue backend or committing to a package in the monorepo. A third-party prototype could be a useful way to test the fit in Kosmo first.

[robin-maki]

Thanks, the point about Worker ownership is especially helpful. Applications choosing Temporal for Fedify message processing are likely to already use it for their own business logic, as we do in Kosmo. Having listen() create and manage another Worker may therefore be unnecessary. I’ll validate how Fedify processing can fit into Kosmo’s existing application-managed Worker setup and report back.

I also overlooked the distinction between Activity retries and Fedify’s re-enqueue paths for Retry-After and circuit-breaker holds. I’ll cover both paths when validating the ordering behavior.

Rather than building a standalone example, I’ll work on integrating this into Kosmo’s actual message-processing flow, letting that use case guide the integration design. I’ll share the resulting setup—including Worker ownership, handler registration, and Task Queue configuration—along with what worked and any limitations we encounter. That should give us a concrete basis for discussing whether MessageQueue is the right boundary and where the integration should live.