Skip to content

feat: Improve Table Previews - #2593

Open
camielvs wants to merge 1 commit into
07-21-feat_improve_parquet_viewerfrom
07-29-feat_parquet_improvements_2
Open

feat: Improve Table Previews#2593
camielvs wants to merge 1 commit into
07-21-feat_improve_parquet_viewerfrom
07-29-feat_parquet_improvements_2

Conversation

@camielvs

@camielvs camielvs commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Description

Follow-up to #2557. This PR adds extra viewer affordances for both Parquet and CSV/TSV.

1. Load more / Load max (Parquet). The base PR previews the top 100 rows and stops. Here the viewer can pull additional pages on demand via range reads — Load more fetches the next batch, Load max fills up to the preview cap. Only the newly requested range is fetched each time; rows already loaded are never re-read, and the whole file is never downloaded.

2. Preview limit is a cell budget, not a row count. The preview table renders every cell into the DOM (no virtualization), so its cost scales with rows × columns, not rows alone. A flat 1,000-row cap therefore over-protects narrow tables and under-protects wide ones. Instead, the preview is bounded by a 50,000-cell budget: the row limit is floor(50,000 / columnCount), with an absolute backstop of 10,000 rows so a very narrow table still can't flood the DOM. This adapts to table shape:

Columns Preview row limit
≤ 5 10,000 (backstop)
10 5,000
20 2,500
100 500

Rows are atomic: the budget is floored to whole rows up front, so the rendered cell count never exceeds 50,000 and no partial rows are shown. The same limit applies to both Parquet (Load max) and CSV/TSV previews.

3. Download full dataset. When the preview limit is reached but the file still has more rows than are shown, the footer surfaces a Download full dataset link so the user has a clear escape hatch to the complete data (opens the signed URL for remote artifacts).

4. CSV/TSV parity. CSV/TSV previews now report the exact total row count and column count in the header, matching Parquet. Counting is done with a streaming parse (every row is counted, only the preview rows are retained). Remote CSVs get a Download full dataset link to the signed URL; inline CSV values trigger a direct file download.

image.png

Related Issue and Pull requests

Type of Change

  • Improvement

Checklist

  • I have tested this does not break current pipelines / runs functionality
  • I have tested the changes on staging

Test Instructions

  1. Open a run with a Parquet artifact that has more than 100 rows and open its preview.
  2. Confirm a top-100 preview loads, then use Load more / Load max and confirm additional rows load without a full-file download.
  3. Confirm the preview stops at the cell budget: for a wide table (e.g. 50+ columns) Load max stops well below 10,000 rows, and once the limit is reached a Download full dataset link appears.
  4. Open a CSV/TSV artifact and confirm the header shows exact row/column counts, and that the download affordance works for both remote and inline CSV.

Additional Comments

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

🎩 Preview

A preview build has been created at: 07-29-feat_parquet_improvements_2/227a763

Copy link
Copy Markdown
Collaborator Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@camielvs
camielvs force-pushed the 07-29-feat_parquet_improvements_2 branch from e484306 to 360e7b9 Compare July 30, 2026 00:19
@camielvs
camielvs force-pushed the 07-21-feat_improve_parquet_viewer branch from 436b6ef to c472ea6 Compare July 30, 2026 00:19
@camielvs camielvs changed the title feat: Parquet Improvements 2 feat: Improve Table Previews Jul 30, 2026
@camielvs
camielvs force-pushed the 07-29-feat_parquet_improvements_2 branch from 360e7b9 to 5613e32 Compare July 30, 2026 00:46
@camielvs
camielvs force-pushed the 07-21-feat_improve_parquet_viewer branch 2 times, most recently from d3eb937 to 99d7a62 Compare July 30, 2026 14:57
@camielvs
camielvs force-pushed the 07-29-feat_parquet_improvements_2 branch 2 times, most recently from 1abf10f to f8a11f0 Compare July 30, 2026 16:02
@camielvs
camielvs marked this pull request as ready for review July 30, 2026 16:24
@camielvs
camielvs requested a review from a team as a code owner July 30, 2026 16:24
@camielvs
camielvs force-pushed the 07-29-feat_parquet_improvements_2 branch 2 times, most recently from 9d07e68 to 2c40fb0 Compare July 30, 2026 17:54
@camielvs
camielvs force-pushed the 07-21-feat_improve_parquet_viewer branch from 99d7a62 to ec1e716 Compare July 30, 2026 18:23
@camielvs
camielvs force-pushed the 07-29-feat_parquet_improvements_2 branch from 2c40fb0 to 227a763 Compare July 30, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant