Skip to content

Comments

Fix FileNotFoundException handling in segment download API#17523

Open
rsrkpatwari1234 wants to merge 10 commits intoapache:masterfrom
rsrkpatwari1234:rsrkpatwari1234-16231
Open

Fix FileNotFoundException handling in segment download API#17523
rsrkpatwari1234 wants to merge 10 commits intoapache:masterfrom
rsrkpatwari1234:rsrkpatwari1234-16231

Conversation

@rsrkpatwari1234
Copy link
Contributor

@rsrkpatwari1234 rsrkpatwari1234 commented Jan 18, 2026

Fixes #16231

Problem

When the /segments controller API hit a FileNotFoundException (or HTTP 404), it showed these issues:

  1. Unnecessary retries: The segment fetcher retried up to 3 times even when the file did not exist, wasting time and resources.
  2. Wrong HTTP status: The API returned HTTP 500 (Internal Server Error) instead of HTTP 404 (Not Found).
  3. Obscured root cause: The original FileNotFoundException was wrapped in AttemptsExceededException, making debugging harder.

Solution

In this PR, I have implemented two improvements -

  1. Skip retries for FileNotFoundException: In BaseSegmentFetcher and HttpSegmentFetcher, we detect FileNotFoundException (and, for HTTP, HttpErrorStatusException with status 404) in the exception cause chain and rethrow immediately so no retries are performed.
  2. Return HTTP 404: In PinotSegmentUploadDownloadRestletResource, we catch these cases (via ExceptionUtils.isCauseInstanceOf and getFirstCauseOfType) and return Response.Status.NOT_FOUND (404) instead of 500.

Unit Tests
✅ 4 new tests added to verify FileNotFoundException behaviour
✅ All existing tests pass - no breaking changes

bugfix

@rsrkpatwari1234
Copy link
Contributor Author

@Jackie-Jiang @ankitsultana Requesting your help in running the above workflow and reviewing the changes

Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix the test failure

@codecov-commenter
Copy link

codecov-commenter commented Feb 8, 2026

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
20700 1 20699 59
View the full list of 1 ❄️ flaky test(s)
org.apache.pinot.core.data.manager.BaseTableDataManagerTest::testAddNewSegmentUseLocalCopy

Flake rate in main: 89.80% (Passed 10 times, Failed 88 times)

Stack Traces | 1.35s run time
expected [Operation failed after 3 attempts] but found [org.apache.pinot.spi.filesystem.LocalPinotFS$LocalPinotFSException: java.io.FileNotFoundException: Source '' does not exist]

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@rsrkpatwari1234
Copy link
Contributor Author

@Jackie-Jiang Requesting your review. The test failure seems unrelated to my change

@Jackie-Jiang
Copy link
Contributor

I think it is related:

[ERROR]   BaseTableDataManagerTest.testAddNewSegmentUseLocalCopy:585 expected [Operation failed after 3 attempts] but found [org.apache.pinot.spi.filesystem.LocalPinotFS$LocalPinotFSException: java.io.FileNotFoundException: Source '' does not exist]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid retries for FileNotFound exception in /segments controller API

3 participants