feat: add pandas DataFrame integration for event analysis #164
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add support for converting event streams to pandas DataFrames for data analysis and exploration. The events_to_dataframe() function accepts an AsyncGenerator of events and returns a DataFrame with all event fields as columns. - Add eventsourcingdb.pandas module with events_to_dataframe() function - Add comprehensive test suite following TDD approach - Add pandas as optional dependency group - Update README with usage examples and flattening guide 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
When converting an empty event stream, the DataFrame now includes all expected columns instead of being completely empty. This is achieved by explicitly defining the columns parameter when creating the DataFrame. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Use conditional logic to create DataFrame with columns parameter only when the event list is empty, avoiding type checking issues with pyright while maintaining correct behavior. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Add type: ignore annotation for the columns parameter when creating an empty DataFrame, as pyright's pandas stubs don't recognize list[str] as a valid type despite it working correctly at runtime. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Set DataFrame columns using pd.Index after creation instead of passing columns parameter to constructor, resolving pyright type checking errors while maintaining correct functionality. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Revert to using columns parameter with correct pyright ignore syntax (reportArgumentType) instead of attempting to set columns property after creation, which fails for empty DataFrames. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for converting event streams to pandas DataFrames, enabling data analysis and exploration of events.
Key changes:
eventsourcingdb.pandasmodule withevents_to_dataframe()functionAsyncGenerator[Event, None]topd.DataFramewith all event fieldspip install eventsourcingdb[pandas])Design decisions:
datafield remains as dict for flexibilitypd.json_normalize()Example Usage
Test Plan
CI/CD
Tests will run automatically via GitHub Actions. The PR includes:
🤖 Generated with Claude Code