Skip to content

Don't consider inflight orders invalid#4241

Merged
fafk merged 11 commits intomainfrom
fix-invalid-event-2
Mar 10, 2026
Merged

Don't consider inflight orders invalid#4241
fafk merged 11 commits intomainfrom
fix-invalid-event-2

Conversation

@fafk
Copy link
Copy Markdown
Contributor

@fafk fafk commented Mar 6, 2026

Description

When I was working on another feature (the debug endpoint) I noticed that there is a race condition where we sometimes insert "invalid" order event to the DB.

From logs

09:19:13.717  stored order events label=Executing count=1
09:19:13.733  filtered orders reason=insufficient_balance count=1 orders=[0x060a...]
09:19:13.737  stored order events label=Invalid count=1
09:19:13.756  stored order events label=Traded count=1

Order is in executing state, then settles onchain. If the indexing runs now and we set the order to traded all is well. However if the in cache update runs first it's possible the wallet no longer has the necessary balance as it was traded away in the just executed swap and we consider it invalid and we insert the invalid event for an order that went through perfectly well.

Changes

  • Don't consider in-flight orders invalid
  • Move in-flight filtering logic from run loop to solvable_orders (move upstream)

How to test

I had an e2e test to reproduce this which failed 6/10 times. After the fix it fails 0/10.

@fafk fafk changed the title Fix invalid event 2 Don't consider inflight orders invalid Mar 9, 2026
@fafk fafk force-pushed the fix-invalid-event-2 branch from 25faaca to 39c2c27 Compare March 9, 2026 13:25
@fafk fafk marked this pull request as ready for review March 9, 2026 13:25
@fafk fafk requested a review from a team as a code owner March 9, 2026 13:25
@fafk fafk mentioned this pull request Mar 9, 2026
1 task
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively resolves a race condition where in-flight orders could be incorrectly marked as invalid. The change moves the filtering of in-flight orders to an earlier stage in solvable_orders.rs, ensuring they are removed from consideration before balance and validity checks occur. This correctly prevents orders that are being settled from being marked as invalid due to temporary balance changes. The implementation is sound and no critical issues were found.

Note: Security Review did not run due to the size of the PR.

let surplus_capturing_jit_order_owners: Vec<_> = cow_amms
.iter()
.filter(|cow_amm| {
if in_flight_owners.contains(cow_amm.address()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth adding a short comment on why it is intentional to filter AMM orders by owners instead of UID.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea why this is the case, I just kept the original logic from the deleted code. 😅

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orders rebalancing cow amms revert when the cow amm does not have exactly the state the order was crafted for so having multiple orders in-flight for the same cow amm is an issue.
Additionally an amm can be rebalanced in many different ways which would all result in different order UIDs so filtering based on that is not sufficient. That's way we check if there is any order in-flight for that amm based on the owner of the order (i.e. the cow amm) and then discard that amm altogether for that auction.

Copy link
Copy Markdown
Contributor Author

@fafk fafk Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the above as a comment verbatim. 🌚

}

#[instrument(skip_all)]
async fn fetch_in_flight_orders(&self, block: u64) -> HashSet<OrderUid> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to keep this annotation?

Suggested change
async fn fetch_in_flight_orders(&self, block: u64) -> HashSet<OrderUid> {
#[instrument(skip_all)]
async fn fetch_in_flight_orders(&self, block: u64) -> HashSet<OrderUid> {

Copy link
Copy Markdown
Contributor Author

@fafk fafk Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't, the fn that runs the actual query is annotated. This fn that just does the remapping would just add noise.

Copy link
Copy Markdown
Contributor

@MartinquaXD MartinquaXD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments could be cleaned up a bit more but overall LGTM.

let surplus_capturing_jit_order_owners: Vec<_> = cow_amms
.iter()
.filter(|cow_amm| {
if in_flight_owners.contains(cow_amm.address()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orders rebalancing cow amms revert when the cow amm does not have exactly the state the order was crafted for so having multiple orders in-flight for the same cow amm is an issue.
Additionally an amm can be rebalanced in many different ways which would all result in different order UIDs so filtering based on that is not sufficient. That's way we check if there is any order in-flight for that amm based on the owner of the order (i.e. the cow amm) and then discard that amm altogether for that auction.

@fafk fafk force-pushed the fix-invalid-event-2 branch from 1ed78f0 to 033c046 Compare March 10, 2026 11:05
@fafk fafk enabled auto-merge March 10, 2026 11:35
@fafk fafk added this pull request to the merge queue Mar 10, 2026
Merged via the queue into main with commit cd9c49b Mar 10, 2026
19 checks passed
@fafk fafk deleted the fix-invalid-event-2 branch March 10, 2026 11:51
@github-actions github-actions bot locked and limited conversation to collaborators Mar 10, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants