Add generate_playtime_report collection setting to gate playtime reports (PP-4346)

[dbernstein]

Description

Replaces the current behavior of generating reports for every OPDS2 and OPDSForDistributors protocol with an explicit per-collection opt-in flag that controls whether a collection's playtime data is included in the monthly Google Drive report.

Changes

New setting: generate_playtime_report (bool, default False)

Added via the PlaytimeReportSettings mixin to:

• OPDS2APISettings — the settings class used exclusively by OPDS2API (OPDS 2.0 Import)
• OPDSForDistributorsSettings — the settings class used by OPDSForDistributorsAPI

The field is not present on OPDS2ImporterSettings (the shared base class), which means OPDS2WithODLSettings (ODL 2.0) does not inherit it and the setting does not appear on ODL 2.0 collection forms. OPDS2APISettings is a thin subclass of OPDS2ImporterSettings that adds the mixin.

The field is sysadmin-facing and rendered as a SELECT ("Generate playtime report for audio books"). All other protocols are ineligible by design.

Migration

Sets generate_playtime_report = True for all existing OPDS 2.0 / OPDS for Distributors collections whose data_source name begins with "Blackstone" or "Unlimited". All other collections default to False and require no update. I verified in Redshift that using the aforementioned constraints will match all Blackstone and Unlimited Listens data sources and no others.

Downgrade removes the key entirely (absence is equivalent to the Pydantic default of False).

_fetch_distinct_eligible_data_source_names

Now returns only data source names for collections where generate_playtime_report is explicitly True. The PlaytimeSummary fallback (which was the root cause of folder proliferation — it included every historical data source regardless of intent) has been removed.

Behavioral note — deleted collections: Because eligibility is now determined entirely by a collection's settings, a deleted collection produces no report regardless of how much historical playtime data it accumulated. Without a live collection there are no collection settings, and therefore no generate_playtime_report flag to evaluate. This is intentional: report generation should reflect an active, administrator-approved configuration, not the presence of stale summary rows. If we need to go back and find playtime summaries for deleted collections, the data will still be there.

Motivation and Context

The Partner Success team observed large numbers of Google Drive folders being generated for every data source in the system. The underlying cause was a combination of:

1. _fetch_distinct_eligible_data_source_names returning all data sources that ever appeared in PlaytimeSummary records, and
2. A prior change expanding the set of eligible protocols beyond the two Blackstone-platform integrations.

The flag-based approach gives sysadmins explicit control, makes the opt-in visible and auditable in the collection settings UI, and removes the unbounded fallback.

JIRA: PP-4346

How Has This Been Tested?

• test_fetch_distinct_eligible_data_source_names (8 parametrised cases): verifies that only flagged OPDS 2.0/OFD collections are included; unflagged collections, ineligible protocols, and mixed scenarios all behave correctly.
• test_fetch_distinct_eligible_data_source_names_deleted_collection: verifies the new no-fallback behaviour — a deleted collection is excluded even if it had prior playtime data.
• test_generate_playtime_reports: updated to set generate_playtime_report=True on active collections and verify that the deleted collection4 (ds_d) no longer produces a report (3 files instead of 4).
• test_generate_playtime_report_folder_lock_contention: verifies that the task completes and logs a warning when the folder-creation lock cannot be acquired (from PR Reduce Google Drive folder race condition with Redis lock, jitter, and pick-oldest (PP-4434) #3377, now included here after rebase).
• All 14 affected tests pass.

Checklist

• I have updated the documentation accordingly.
• All new and existing tests passed.

[claude]

Claude finished @dbernstein's task in 7m 47s —— View job

---

Code Review

• Gather context and understand the request
• Read changed files and analyze the diff
• Review the migration file
• Review the settings/model changes
• Review the playtime entries task logic
• Review the tests
• Post findings

---

Summary

The PR cleanly replaces an unbounded-fallback design with an explicit opt-in generate_playtime_report flag. The OPDS2APISettings subclass approach (commit 6f03f92) correctly resolves jonathangreen's pre-merge concern: OPDS2WithODLSettings still inherits directly from OPDS2ImporterSettings (no playtime field), while OPDS2API switches to OPDS2APISettings which adds PlaytimeReportSettings as a mixin. The migration correctly targets only collections whose data_source begins with "Blackstone" or "Unlimited", the is True identity check for the JSONB boolean is intentional and correct, and the removal of the PlaytimeSummary fallback is well-motivated and clearly documented. Test coverage for the new behaviour is thorough.

Details

Minor: src/palace/manager/integration/license/opds/settings/playtime_report.py:22

The field name generate_playtime_report has a different grammatical structure from the analogous field in opds1/settings.py (include_in_inventory_report), which follows a <verb>_in_<report> convention. jonathangreen's first-ranked suggestion was include_in_playtime_report to match that pattern. The author chose option 2 (generate_playtime_report) and jonathangreen has approved — this is a naming preference rather than a bug, so flagging only for visibility.

circulation/src/palace/manager/integration/license/opds/settings/playtime_report.py

Line 22 in 6f03f92

generate_playtime_report: Annotated[

Nit: src/palace/manager/celery/tasks/playtime_entries.py:338-345

The c.data_source guard (which calls circulation_api() → instantiates the provider) is evaluated for every collection that passes the generate_playtime_report is True check, incurring a session-cached but still repeated per-collection call to build the API instance. For the typical small set of flagged collections this is harmless; if the eligible set grows, pushing the flag check into the SQL query (WHERE settings @> '{"generate_playtime_report": true}'::jsonb) would avoid loading unflagged collections entirely. Not a change needed now.

circulation/src/palace/manager/celery/tasks/playtime_entries.py

Lines 338 to 345 in 6f03f92

	collection_ds_names = {
	c.data_source.name
	for c in eligible_collections
	if c.integration_configuration.settings_dict.get("generate_playtime_report")
	is True
	and c.data_source
	and c.data_source.name is not None
	}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.40%. Comparing base (06619fc) to head (6f03f92).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3407   +/-   ##
=======================================
  Coverage   93.40%   93.40%           
=======================================
  Files         504      505    +1     
  Lines       46234    46239    +5     
  Branches     6310     6310           
=======================================
+ Hits        43184    43190    +6     
+ Misses       1981     1980    -1     
  Partials     1069     1069           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dbernstein]

I'm not sure why Claude added the "or" clause.

[dbernstein]

I'm going to change that.

[dbernstein]

@jonathangreen : I reworked this one according to your suggestion re using a configuration setting on the collection rather than hardcoding the data source names in the report generator.

[jonathangreen]

I'm approving this one, but I'd to get the OPDS2WithODLSettings issue resolved before merging. Also added a minor comment to consider about variable naming.

[jonathangreen]

Because OPDS2WithODLSettings subclasses OPDS2ImporterSettings, adding PlaytimeReportSettings here also exposes is_generate_playtime_report on ODL 2.0 collection configuration forms. Since OPDS2+ODL collections are filtered out, it doesn't actually enable it for them, but it does create a misleading config setting.

[jonathangreen]

Minor: I really don't like the name is_generate_playtime_report. is_generate_playtime_report is off in two ways:

1. is_ prefix: nothing else in the settings models uses it; the convention is bare verbs.
2. is_ + generate reads as "is generate": grammatically broken. is_ wants an adjective/state (is_enabled), not a verb.

opds1/settings.py has a setting include_in_inventory_report, which serves a similar purpose, and follows the conventions we've been using more closely.

My suggestion would be to rename this to either, in order of preference:

1. include_in_playtime_report
2. generate_playtime_report