Add position health updated event

[illia-malachyn]

Motivation

The FCM observer currently fetches positions’ health by running scripts (get_position_by_id.cdc). This approach doesn’t scale well.

For example, with ~10,000 positions, we need to execute 10,000 script calls. Access nodes are rate-limited (typically ~1–5 requests/sec), so processing all positions takes a significant amount of time.

Change

• Introduce an event-based approach for tracking position health.
• Instead of polling via scripts, we subscribe to events over a WebSocket (TCP) stream. This allows us to process a high volume of updates in real time, without being constrained by request rate limits.

Result

• Eliminates the need for per-position script calls
• Avoids access node rate limits
• Enables near real-time updates
• Scales much better with the number of positions

[holyfuchs]

The event is only emitted inside _queuePositionForUpdateIfNecessary, which is triggered on deposit, withdrawal, or async update loop processing. It won't fire when health drifts passively due to interest accrual or price oracle changes, which are arguably the most critical cases for the FCM observer to catch.
As-is, this doesn't fully replace polling for positions that aren't being actively touched.

Could get_position_by_id.cdc be extended to accept a list of position IDs and return health for all of them in a single script call? That would collapse 10k requests into far fewer batched calls and might be a simpler fix than the event approach.

Another option would be to calculate position health off-chain entirely.
To do this, the observer would need:

• the scaledBalance per token per position (only changes on withdraw, deposit)
• the pool's current interest indices (creditIndex / debitIndex)
• and the current oracle prices.
  With those three inputs, health can be computed as effectiveCollateral / effectiveDebt for any position without any script calls. The interest indices and oracle prices could be fetched periodically in a single read rather than per-position, making this approach scale to any number of positions.

[illia-malachyn]

@holyfuchs Thanks for starting this discussion!

"It won't fire when health drifts passively due to interest accrual or price oracle changes"**

• Price oracle changes: From what I can see in code, when the oracle is updated via setPriceOracle(), we explicitly queue all positions for update: self.state.setPositionsNeedingUpdates(self.positions.keys). These queued positions are then processed through asyncUpdate() -> asyncUpdatePosition() -> _queuePositionForUpdateIfNecessary().

• Interest accrual: If you're referring to a scenario where interest accrues but no transaction touches the position - how would the script-based approach catch this? Scripts read on-chain state, and if the health factor isn't recalculated on-chain, the script will return stale data as well.

"Could get_position_by_id.cdc be extended to accept a list of position IDs?"**

Yes, we could write a batch script, but it only mitigates the problem rather than solving it:

• Scale ceiling is unknown: I don't know the upper bound of how much data a single script execution can return. With 100k positions, if the script can only handle 500-1000 at a time, we're still making hundreds of calls and falling behind. For our liquidation monitoring (currently manual with Grafana alerts), being slow to detect unhealthy positions could be critical.
• Polling vs streaming: Batch scripts are still polling - we'd be trading many small requests for fewer large requests, but we're still bound by polling intervals and access node rate limits. Events over WebSocket give us a real-time push-based stream with no rate limit concerns.

off-chain health calculation

I think it adds significant complexity - we'd need to replicate the health calculation logic off-chain, keep it in sync with any contract changes, and independently fetch and combine interest indices + oracle prices + per-position balances. I think events are a much simpler and more maintainable approach, and I'd prefer to go with it and only fall back to scripts if events turn out not to be doable for some reason.

[holyfuchs]

Price oracle changes: From what I can see in code, when the oracle is updated via setPriceOracle(), we explicitly queue all positions for update

Yes, but this only queues positions when the PriceOracle contract is swapped out, not when prices change. That will extremely rarely happen. The much more common case — prices moving continuously — is not covered.

Interest accrual: If you're referring to a scenario where interest accrues but no transaction touches the position - how would the script-based approach catch this?

Scripts can't persist state changes, but they do reflect them during execution. getPositionDetails calls _borrowUpdatedTokenState which calls state.updateForTimeChange() — so the interest accrual is applied transiently within the script call and the returned health value is accurate, even though nothing is written on-chain. So a batch script would actually catch interest-driven health drift correctly.

Events

Position health changes constantly — every second due to interest accrual, and on every price tick. Events are only useful if you emit one regularly for every position. You could add a function that iterates all positions and emits health events on a schedule, but unlike a script this isn't free — each event costs gas, so at any meaningful scale this gets expensive fast. Scripts are cheaper here precisely because they don't write to chain.

Off-chain health calculation

I agree it definitely adds a significant amount of complexity but I don't think there is a nice solution here.