Merge branch 'main' into fix-duplicate-pr-labels-20250803-143123

Author: choldgraf

.github/workflows/tests.yaml

@@ -36,7 +36,7 @@ jobs:
         include:
           # Only test oldest supported and latest python version to reduce
           # GitHub API calls, as they can get rate limited
-          - python-version: 3.9
+          - python-version: "3.10"
           - python-version: 3.x
 
     steps:

README.md

@@ -1,28 +1,23 @@
 # github-activity
 
-Generate simple markdown changelogs for GitHub repositories written in Python.
-
 [![continuous-integration](https://github.com/executablebooks/github-activity/actions/workflows/tests.yaml/badge.svg)](https://github.com/executablebooks/github-activity/actions/workflows/tests.yaml)
 
-This package provides a CLI to do two things:
-
-**Scrape all GitHub activity over a period of time for a repository**.
-Given a GitHub org, repository, an initial git reference or date, use the [GitHub GraphQL API](https://developer.github.com/v4/) to return a Pandas DataFrame of all issue and PR activity for this time period.
-
-**Render this as a markdown changelog**.
-Convert this DataFrame to markdown that is suitable for generating changelogs or community updates.
+Generate markdown changelogs for GitHub repositories with more control over types of contributions and metadata used to create the changelogs.
 
 For an example, see the [changelog of this package](<[https://](https://github-activity.readthedocs.io/en/latest/changelog)>).
 
-## Use this tool
+See [the GitHub Activity Documentation](https://github-activity.readthedocs.io) for our documentation.
+
+## Install and use this tool
 
-Use this tool via the command line like so:
+Install and use this tool via the command line like so:
 
 ```bash
+pip install github-activity
 github-activity [<org>/<repo>] --since <date or ref> --until <date or ref>
 ```
 
-See [the User Guide for details on how to install and use this tool](https://github-activity.readthedocs.io/en/latest/use).
+See [the GitHub Activity Documentation](https://github-activity.readthedocs.io) for our documentation.
 
 ## Contribute to this package
 

docs/index.md

@@ -1,44 +1,44 @@
 # github-activity
 
-Generate simple markdown changelogs for GitHub repositories written in Python.
+Generate markdown changelogs for GitHub repositories with more control over the kinds of contributions that are included.
 
-This package does two things:
+GitHub Activity allows you to include more than just "PR author" in your changelogs, such as PR reviewers and issue commenters. This allows you to give credit to a wider group of contributors around your project. Below are some examples, see [](#how-does-this-tool-define-contributions-in-the-reports) for more information.
 
-1. Given a GitHub org, repository, an initial git reference or date, use the
-   [GitHub GraphQL API](https://developer.github.com/v4/) to return a DataFrame
-   of all issue and PR activity for this time period.
-2. A CLI to render this activity as markdown, suitable for generating changelogs or
-   community updates.
+- PR Authors
+- PR Reviewers and Mergers
+- Issue and PR commenters
 
-## Installation
+It also allows you split PRs into sections in your changelog, using either PR labels or PR title metadata (e.g., `[ENH]`). See [](#prefixes-and-tags) for more information.
 
-The easiest way to install this package is to do so directly from GitHub with `pip`:
+GitHub Activity uses the [GitHub GraphQL API](https://docs.github.com/en/graphql), along with some basic pagination and caching to efficiently pull data from GitHub.
 
-```
-pip install github-activity
+```{seealso}
+See [the JupyterHub Team changelog](https://github.com/jupyterhub/jupyterhub/blob/5.3.0/docs/source/reference/changelog.md) for an example of this tool in action.
 ```
 
-```{toctree}
-use
-contribute
-changelog
-```
+## Installation and basic usage
+
+The easiest way to install this package is to do so directly from GitHub with `pip`:
 
-(how-does-this-tool-define-contributions-in-the-reports)=
+```bash
+pip install github-activity
+```
 
-## How we define contributors in the reports
+You can then use it like so:
 
-GitHub Activity tries to automatically determine the unique list of contributors within a given window of time.
-There are many ways to define this, and there isn't necessarily a "correct" method out there.
+```bash
+github-activity [<org>/<repo>] --since <date or ref> --until <date or ref>
+```
 
-We try to balance the two extremes of "anybody who shows up is recognized as contributing" and "nobody is recognized as contributing".
-We've chosen a few rules that try to reflect sustained engagement in issues/PRs, or contributions in the form of help in _others'_ issues or contributing code.
+## Why use this tool?
 
-Here are the rules we follow for finding a list of contributors within a time window. A contributor is anyone who has:
+We created `github-activity` because there is a lot that goes into building open source tools than just making a pull request. This tool tries to surface more diverse contributions around a release, like reviews, comments, etc. It tries to paint a more complete picture of all the work that goes into building open source software.
 
-- Contributed to a PR merged in that window (includes opening, merging, committing, commenting, or committing)
-- Commented on >= 2 issues that weren't theirs
-- Commented >= 6 times on any one issue
-- Known bot accounts are generally not considered contributors
+You might want to use this tool if you're hoping to give credit and attribution to more people in your open source community. This gives your community a feeling of more appreciation, and can create more incentives for others to contribute.
 
-We'd love feedback on whether this is a good set of rules to use.
+```{toctree}
+:maxdepth: 2
+use
+contribute
+changelog
+```

docs/use.md

@@ -1,5 +1,10 @@
 # User guide
 
+This tool has two main user interfaces:
+
+1. **A python library**: Given a GitHub org, repository, an initial git reference or date, use the [GitHub GraphQL API](https://developer.github.com/v4/) to return a DataFrame of all issue and PR activity for this time period.
+2. **A Command Line Interface** to render this activity as markdown, suitable for generating changelogs or community updates.
+
 These sections describe how to control the major functionality of this tool.
 
 ## Generate a markdown changelog
@@ -19,8 +24,7 @@ The `[<org>/<repo>]` argument is **optional**.
 If you do not give it, then `github-activity` will attempt to infer this value by running `git remote -v` and using either `upstream` or `origin` (preferring `upstream` if both are available).
 
 The (optional) arguments in `--since` (or `-s`) and `--until` (or `-u`) can either be
-a date, or a ref (such as a commit hash or tag). `github-activity` will pull the activity
-between the dates corresponding to these values.
+a date, or a ref (such as a commit hash or tag). `github-activity` will pull the activity between the dates corresponding to these values.
 
 ```{margin}
 There are many other options with the `github-activity` CLI, run `github-activity -h`
@@ -41,7 +45,22 @@ You can find the [resulting markdown here](sample_notebook_activity).
 For repositories that use multiple branches, it may be necessary to filter PRs by a branch name.  This can be done using the `--branch` parameter in the CLI.   Other git references can be used as well in place of a branch name.
 ```
 
-### Split PRs by tags and prefixes
+## Choose a date or a tag to filter activity
+
+By default, `github-activity` will pull the activity _after_ the latest GitHub release or git tag. You can choose to manually control the date ranges as well.
+
+To specify a **start date**, use the `-s` (or `--since`) parameter. To specify an **end date**, use the `-u` or `--until` parameter.
+
+Each of these accepts either:
+
+1. A date string. This can be anything that [`dateutil.parser.parse`](https://dateutil.readthedocs.io/en/stable/parser.html) accepts.
+2. A git `ref`. For example, a `commit hash` or a `tag`.
+
+If no `-u` parameter is given, then all activity until today will be included.
+
+(prefixes-and-tags)=
+
+## Split PRs by tags and prefixes
 
 Often you wish to split your PRs into multiple categories so that they are easier
 to scan and parse. You may also _only_ want to keep some PRs (e.g. features, or API
@@ -68,8 +87,70 @@ left-most column above.
 
 If a pull request has multiple labels that match different categories, it will appear in **only the first matching section** based on the order of categories processed. For example, a PR labeled with both `api-change` and `enhancement` will appear only in the "API and Breaking Changes" section, not in "Enhancements made". The categories are processed in the same order as they show above.
 
+## Include Pull Request reviewers and commenters in your changelog
+
+By default, GitHub Activity will include anybody that _reviews_ or _comments_ in a pull request in the item for that PR. This is included in a list of authors at the end of each item. See [the JupyterHub Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) for examples.
+
+## Include a list of all contributors at the end of your changelog
+
+By default, this tool will include a long list of contributors at the end of your changelog. This is the unique set of all contributors that contributed to the release.
+
+(how-does-this-tool-define-contributions-in-the-reports)=
+
+### How we define contributors in a changelog
+
+GitHub Activity tries to automatically determine the unique list of contributors within a given window of time.
+There are many ways to define this, and there isn't necessarily a "correct" method out there.
+
+We try to balance the two extremes of "anybody who shows up is recognized as contributing" and "nobody is recognized as contributing".
+We've chosen a few rules that try to reflect sustained engagement in issues/PRs, or contributions in the form of help in _others'_ issues or contributing code.
+
+Here are the rules we follow for finding a list of contributors within a time window. A contributor is anyone who has:
+
+- Contributed to a PR merged in that window (includes opening, merging, committing, or commenting)
+- Commented on >= 2 issues that weren't theirs
+- Commented >= 6 times on any one issue
+- Known bot accounts are generally not considered contributors
+
+We'd love feedback on whether this is a good set of rules to use.
+
+## Strip PR type metadata from the changelog titles
+
+If you follow the [title prefix convention used to split PRs](#prefixes-and-tags), you can remove these prefixes when you generate your changelog, so that they don't clutter the output.
+
+To strip title prefix metadata, use the `--strip-brackets` flag.
+
+For example, `[DOC] Add some documentation` will be rendered as `Add some documentation`.
+
+## Change the heading level for your changelog items
+
+To change the starting heading level for changelog items, use the `--heading-level N` flag. Where `N` is the starting heading level (e.g., `2` corresponds to `##`).
+
+This is useful if you want to _embed_ your changelog into a larger one (e.g., `CHANGELOG.md`).
+
+## Include issues in your changelog
+
+To include closed issues in your changelog, use the `--include-issues` flag.
+
+## Include opened issues in your changelog
+
+To include Issues and Pull Requests that were _opened_ in a time period, use the `--include-opened` flag.
+
 (use:token)=
 
+## Remove bots from the changelog
+
+`github-activity` ships with a known list of bot usernames, but your project may use ones not on our list.
+To ignore additional usernames from the changelog, use the `--ignore-contributor` flag:
+
+```
+github-activity ... --ignore-contributor robot-one --ignore-contributor 'robot-two*'
+```
+
+Wildcards are matched as per [filename matching semantics](https://docs.python.org/3/library/fnmatch.html#fnmatch.fnmatch).
+
+If this is a generic bot username, consider contributing it back to [our list](https://github.com/executablebooks/github-activity/blob/main/github_activity/github_activity.py#L73).
+
 ## Use a GitHub API token
 
 `github-activity` uses the GitHub API to pull information about a repository's activity.
@@ -106,3 +187,63 @@ To do so, follow these steps:
 - Assign the token to an environment variable called `GITHUB_ACCESS_TOKEN`.
   If you run `github-activity` and this variable is defined, it will be used.
   You may also pass a token via the `--auth` parameter (though this is not the best security practice).
+
+## Use the Python API
+
+You can do most of the above from Python as well.
+This is not as well-documented as the CLI, but should have most functionality available.
+
+### Generate markdown changelogs with the Python API
+
+For generating markdown changelogs from Python, here's an example:
+
+```
+from github_activity import generate_activity_md
+
+markdown = generate_activity_md(
+    target="executablebooks/github-activity",
+    since="2023-01-01",
+    until="2023-12-31",
+    kind=None,
+    auth="your-github-token",
+    tags=None,
+    include_issues=True,
+    include_opened=True,
+    strip_brackets=True,
+    heading_level=1,
+    branch=None,
+)
+
+# Print or save the markdown
+print(markdown)
+```
+
+### Return GitHub Activity queries as a DataFrame
+
+For scraping GitHub and returning the data as a DataFrame, here's an example:
+
+```python
+from github_activity import get_activity
+
+# Get activity data as a DataFrame
+from github_activity import get_activity
+
+df = get_activity(
+    target="executablebooks/github-activity",
+    since="2023-01-01",
+    until="2023-12-31",
+    auth="your-github-token",
+    kind=None,
+    cache=None
+)
+```
+
+In some cases, metadata will be nested inside the resulting dataframe.
+There are some helper functions for this. For example, to extract nested comments inside the activity dataframe:
+
+```python
+from github_activity import get_activity, extract_comments
+
+df = get_activity(...)
+comments_df = extract_comments(df['comments'])
+```

github_activity/cli.py

@@ -21,6 +21,7 @@
     "include-opened": False,
     "strip-brackets": False,
     "all": False,
+    "ignore-contributor": [],
 }
 
 parser = argparse.ArgumentParser(description=DESCRIPTION)
@@ -130,6 +131,11 @@
     action="store_true",
     help=("""Whether to include all the GitHub tags"""),
 )
+parser.add_argument(
+    "--ignore-contributor",
+    action="append",
+    help="Do not include this GitHub username as a contributor in the changelog",
+)
 
 # Hidden argument so that target can be optionally passed as a positional argument
 parser.add_argument(
@@ -214,6 +220,7 @@ def main():
         include_opened=bool(args.include_opened),
         strip_brackets=bool(args.strip_brackets),
         branch=args.branch,
+        ignored_contributors=args.ignore_contributor,
     )
 
     if args.all: