Skip to content

feat(MCP): Push MCP usage via OTel#7746

Merged
emyller merged 8 commits into
mainfrom
feat/mcp-usage
Jun 11, 2026
Merged

feat(MCP): Push MCP usage via OTel#7746
emyller merged 8 commits into
mainfrom
feat/mcp-usage

Conversation

@emyller

@emyller emyller commented Jun 10, 2026

Copy link
Copy Markdown
Contributor
  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Contributes to https://github.com/Flagsmith/flagsmith-private/issues/152

Filter incoming requests from MCP servers and emit an mcp.tool.called log to OTel containing 1. the original W3C baggage from the request, and 2. the organisation ID.

Telemetry can then be exported so we can track usage.

How did you test this code?

New unit + integration tests, and full local harness setup using an InfluxDB data storage as telemetry export target.

@emyller emyller self-assigned this Jun 10, 2026
@github-actions github-actions Bot added api Issue related to the REST API docs Documentation updates labels Jun 10, 2026
@vercel

vercel Bot commented Jun 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Preview Jun 11, 2026 4:07pm
flagsmith-frontend-preview Ignored Ignored Preview Jun 11, 2026 4:07pm
flagsmith-frontend-staging Ignored Ignored Preview Jun 11, 2026 4:07pm

Request Review

@github-actions github-actions Bot added feature New feature or request and removed docs Documentation updates labels Jun 10, 2026
@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.56%. Comparing base (aa90ffc) to head (033f911).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7746      +/-   ##
==========================================
+ Coverage   98.54%   98.56%   +0.01%     
==========================================
  Files        1452     1459       +7     
  Lines       55821    56122     +301     
==========================================
+ Hits        55011    55315     +304     
+ Misses        810      807       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 10, 2026
@emyller emyller marked this pull request as ready for review June 10, 2026 18:33
@emyller emyller requested review from a team as code owners June 10, 2026 18:33
@emyller emyller requested review from khvn26 and removed request for a team June 10, 2026 18:33
@emyller emyller changed the title feat(MCP): Push MCP usage to OTel feat(MCP): Push MCP usage via OTel Jun 10, 2026
@github-actions github-actions Bot removed the feature New feature or request label Jun 10, 2026
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-7746 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api-test:pr-7746 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-frontend:pr-7746 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-7746 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-7746 Finished ✅ Results

@github-actions github-actions Bot added the feature New feature or request label Jun 10, 2026
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  44.5 seconds
commit  ce54bc9
info  🔄 Run: #17403 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  38.5 seconds
commit  ce54bc9
info  🔄 Run: #17403 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  42.7 seconds
commit  ce54bc9
info  🔄 Run: #17403 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  39.1 seconds
commit  ce54bc9
info  🔄 Run: #17403 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  34.4 seconds
commit  b139c3c
info  🔄 Run: #17411 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  44.5 seconds
commit  b139c3c
info  🔄 Run: #17411 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  34.5 seconds
commit  b139c3c
info  🔄 Run: #17411 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  43.2 seconds
commit  b139c3c
info  🔄 Run: #17411 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  34.1 seconds
commit  f0bcd67
info  🔄 Run: #17412 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  39.7 seconds
commit  f0bcd67
info  🔄 Run: #17412 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  34.2 seconds
commit  f0bcd67
info  🔄 Run: #17412 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  41 seconds
commit  f0bcd67
info  🔄 Run: #17412 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  36.5 seconds
commit  033f911
info  🔄 Run: #17438 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  33 seconds
commit  033f911
info  🔄 Run: #17438 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  41.2 seconds
commit  033f911
info  🔄 Run: #17438 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  55.7 seconds
commit  033f911
info  🔄 Run: #17438 (attempt 1)

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Visual Regression

19 screenshots compared. See report for details.
View full report

@khvn26 khvn26 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not have user.organisations.first() heuristics at all than allow faulty data. I think it's important to provide exact data.

I would still much prefer setting the org id to a span attribute in the actual code that retrieves it for permission handling. It's the idiomatic OTel way IMO, with less potential failure modes and overall LoC to maintain.

I understand the need to provide org id explicitly so it's documented in the catalogue. However, I'd consider spending time on codegen + log tooling to allow adding select span attributes as OTLP event attributes (using e.g. a lazy getter callable or a sentinel object as a value). This would allow us to pave a way to provide common Flagsmith logging context (e.g. org id, user uuid, plan name, persona type etc) exactly from the source without having to maintain specific plumbing for each case.

Feel free to dismiss my review if you get another one while I'm OOO.

Comment thread api/telemetry/middleware.py Outdated
Comment thread api/telemetry/middleware.py Outdated
@emyller

emyller commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

I'd rather not have user.organisations.first() heuristics at all than allow faulty data. I think it's important to provide exact data.

I agree. Would you please point out, in this PR, why you believe the code is not producing exact data?

I would still much prefer setting the org id to a span attribute in the actual code that retrieves it for permission handling. It's the idiomatic OTel way IMO, with less potential failure modes and overall LoC to maintain.

a15009a — note that the permission layer does not cover all endpoints used by MCP, leading us to the single-org user fallback. Fixing that has a large blast radius, which I experimented with, and prefer leaving to a separate issue.

I understand the need to provide org id explicitly so it's documented in the catalogue. However, I'd consider spending time on codegen + log tooling to allow adding select span attributes as OTLP event attributes (using e.g. a lazy getter callable or a sentinel object as a value). This would allow us to pave a way to provide common Flagsmith logging context (e.g. org id, user uuid, plan name, persona type etc) exactly from the source without having to maintain specific plumbing for each case.

I understand the potential of span enrichment, and I think it warrants proper scoping.

@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 11, 2026
@github-actions github-actions Bot added the docs Documentation updates label Jun 11, 2026
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 11, 2026
Zaimwa9
Zaimwa9 previously approved these changes Jun 11, 2026

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small NIT if ever there are additional back and forth but from my point of view the implementation is good and opened topics have been resolved or deferred.
Approving to unblock, leaving to your appreciation whether to wait on Kim' confirmation on the points he raised

Comment thread api/telemetry/middleware.py Outdated
@khvn26

khvn26 commented Jun 11, 2026

Copy link
Copy Markdown
Member

Would you please point out, in this PR, why you believe the code is not producing exact data?

Good point, I misread it initially. Yeah, optimising for the common case seems like the right call to me.

the permission layer does not cover all endpoints used by MCP

Aw shucks, I assumed otherwise. Thanks for uncovering this. Looks like a good bit of info for #7035.

and I think it warrants proper scoping

Yes, agreed, there are decisions to make - for example, we might want to implement our own context layer instead of using spans directly.

khvn26
khvn26 previously approved these changes Jun 11, 2026
@emyller emyller dismissed stale reviews from khvn26 and Zaimwa9 via 033f911 June 11, 2026 16:07
@github-actions github-actions Bot added docs Documentation updates feature New feature or request and removed feature New feature or request docs Documentation updates labels Jun 11, 2026

@matthewelwell matthewelwell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving based on Kim's prior review and minor additional change.

@emyller emyller merged commit 5d410f2 into main Jun 11, 2026
33 checks passed
@emyller emyller deleted the feat/mcp-usage branch June 11, 2026 17:09
@emyller

emyller commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Looks like a good bit of info for #7035.

For reference:

(LLM content)

Tool Org id
list_organization_groups span-originated
list_organization_invites span-originated
create_organization_invite span-originated
get_project span-originated
update_project span-originated
list_project_environments span-originated
list_environments span-originated
list_project_features single-org fallback
get_feature_flag span-originated
update_feature span-originated
create_feature span-originated
get_feature_evaluation_data single-org fallback
get_feature_code_references span-originated
get_feature_external_resources single-org fallback
get_feature_health_events span-originated
list_feature_multivariate_options span-originated
create_feature_multivariate_option span-originated
update_feature_multivariate_option span-originated
delete_feature_multivariate_option span-originated
list_project_segments span-originated
get_project_segment span-originated
update_project_segment span-originated
create_project_segment span-originated
list_feature_segments single-org fallback
delete_feature_segment span-originated
create_segment_override span-originated
update_environment_feature_state span-originated
update_feature_state span-originated
get_environment_feature_versions span-originated
create_environment_feature_version span-originated
get_environment_feature_version_states span-originated
create_environment_feature_version_state span-originated
update_environment_feature_version_state span-originated
publish_environment_feature_version span-originated
list_organizations single-org fallback
list_projects_in_organization single-org fallback

30 of 36 (83%) span-originated
(End of LLM content)

Tools from private modules aren't included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants