Skip to content

Harden miner fulfillment safety and recovery logging#20

Closed
bitloi wants to merge 2 commits intoentrius:testfrom
bitloi:feature/miner-fulfillment-safety
Closed

Harden miner fulfillment safety and recovery logging#20
bitloi wants to merge 2 commits intoentrius:testfrom
bitloi:feature/miner-fulfillment-safety

Conversation

@bitloi
Copy link
Copy Markdown

@bitloi bitloi commented Apr 8, 2026

Closes #19

Summary

  • Add timeout margin gate (FULFILLMENT_TIMEOUT_MARGIN_BLOCKS = 3) to prevent new sends when too few blocks remain before swap timeout. Cached sends bypass the margin to allow mark_fulfilled retries.
  • Classify terminal mark_fulfilled contract errors (SwapNotFound, InvalidStatus, NotAssignedMiner, MinerNotActive) to stop futile retries.
  • Write append-only JSONL recovery log for stuck-funds scenarios (terminal failures and timed-out swaps with sent funds).

Design

The timeout margin gates new fund sends only. Once funds are already sent (cached), all downstream retry paths remain open regardless of proximity to timeout so the miner can still finalize on-chain.

Test plan

  • Timeout margin blocks new sends within 3 blocks of deadline
  • Cached sends bypass margin and retry mark_fulfilled
  • Terminal contract errors halt retries and write recovery entry
  • Transient errors (RPC_FAILURE, INSUFFICIENT_BALANCE) allow continued retry
  • Stale cache cleanup detects timed-out swaps and logs recovery entry
  • Completed swaps cleaned up without false-positive recovery alerts
  • Focused simulation tests pass for all edge cases

@LandynDev
Copy link
Copy Markdown
Collaborator

hey, double-send is already prevented at fulfillment.py:206 via the _sent short-circuit, with _save_sent_cache() persisting atomically and _load_sent_cache() restoring on startup. Miner stability is present and sufficient for MVP. Further and further engineering requires further and further maintenance, must choose our work carefully too

on the three concerns in #19:

  • near-timeout sends — today it's just the bare current_block >= swap.timeout_block check at fulfillment.py:85. a cushion is a one-line add at the miners choice (could be added with a default, that's fair)
  • futile retries — retries already happen implicitly and safely. process_swap returning False skips mark_processed, so the swap re-enters new_pending next poll tick; the _sent cache at fulfillment.py:206 short-circuits the re-send and we retry mark_fulfilled with the same payload until it lands or the swap exits Active. that covers transient RPC, CALL_FAILED, and balance blips.
  • no recovery visibilitybt.logging.error already hits disk and _sent persists atomically across restart

@bitloi
Copy link
Copy Markdown
Author

bitloi commented Apr 13, 2026

On retries — the implicit loop handles transient errors fine, but SwapNotFound/InvalidStatus/NotAssignedMiner from mark_fulfilled will never land. The miner retries that same doomed call every cycle with no escalation. That's the specific gap: funds left the wallet, contract permanently rejects the update, and the only trace is buried in rotating logs alongside every other error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Miner fulfillment safety: timeout margin, terminal error classification, recovery logging

2 participants