Skip to content

Return an empty array when an observation request returns no data#60

Open
philip-khaisman wants to merge 1 commit into
masterfrom
start-replay-when-datasource-is-empty
Open

Return an empty array when an observation request returns no data#60
philip-khaisman wants to merge 1 commit into
masterfrom
start-replay-when-datasource-is-empty

Conversation

@philip-khaisman

@philip-khaisman philip-khaisman commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

I found an issue where if a data source has no data and i try to replay it with other non-empty data sources (with a data synchronizer) no data streams. This is because in the DataSynchronizerAlgoReplay class we gate the streaming of data with this check:

if ((nbFetch + nbSkip) === totalDataSources) { start processing data }

nbFetch increments for every data source that has a status of FETCH_STARTED. The empty data source does not have this status. The status is set in the DelegateReplayHandler class in the startLoop() method:

let data = await this.context.nextBatch();
this.context.onChangeStatus(Status.FETCH_STARTED);

Previously nextBatch() (defined in the SweApiReplayContext class) was implemented like this:

    async nextBatch(properties, masterTimestamp, status = {cancel:false}) {
        let version = this.properties.version;
        return new Promise(async (resolve, reject) => {
            try {
                let data;
                let results = [];

                const moveTimeCursor = async () => {...}

                const fetchNext = async () => {
                    data = await this.collection.nextPage();
                    if (status.cancel) {
                        reject('Status has been cancelled');
                    }
                    if (data.length > 0) {
                        results = data;
                        for(let d of results) {
                            d.version = version;
                        }
                        if (status.cancel) {
                            reject('Status has been cancelled');
                        } else {
                            // start startTime cursor
                            this.relativeStartTimestamp = results[results.length-1].timestamp;
                            resolve(data);
                        }
                    }
                }

                await moveTimeCursor();
                await fetchNext();
            } catch (ex) {
                reject(ex);
            }
        });
    }

Note that the condition where data.length === 0) leaves this promise hanging resulting in the FETCH_STARTED status never being set. By resolving the promise to an empty array this promise allows the logic in the startLoop() method to continue, setting the FETCH_STARTED status allowing the replay algorithm to process data. I also added an early return in the startLoop() method since subsequent logic is unnecessary if there is no data returned.

Testing:

  • Load data with data sources that have data and at least one that does not
  • I used the webtak plugin to test
    • Toggle to replay mode
    • Start playback
    • Verify data is visualized

@philip-khaisman philip-khaisman linked an issue Jul 9, 2026 that may be closed by this pull request
@philip-khaisman
philip-khaisman marked this pull request as ready for review July 9, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replay doesn't work if a datastream in a driver is empty

1 participant